Method for detecting that a human body remains in a lying position after a fall, fall detection method and related device

Abstract

This application relates to the field of intelligent monitoring technology, and discloses a method for detecting a human body maintaining a falling posture, a fall detection method, and related devices. First, N frames of infrared images are acquired, and the first N frames of these N infrared images... F The frame is an infrared image of the fall, followed by N. k The frame is an infrared image of the fall, N F +N k = N. Multiple tracking paths are obtained by tracking hotspots in N frames of infrared images. The target path of the human body is obtained from these multiple tracking paths. Based on the target path, it is determined whether the human body maintains a fallen, lying position. In this embodiment, the target path corresponding to the human body is determined by tracking N frames of infrared images, including the falling process and the human body after the fall. Based on the characteristics of the hotspots in the target path, combined with the characteristics of the hotspots corresponding to when the human body maintains a fallen, lying position, it is possible to accurately analyze whether the human body maintains a fallen, lying position. If the human body maintains a fallen, lying position, the fall detection result is correct; if the human body does not maintain a fallen, lying position, the fall detection result is incorrect.

Description

Technical Field

[0001] This application relates to the field of intelligent monitoring technology, and in particular to a method for detecting a human body maintaining a fallen lying posture, a fall detection method, and related devices. Background Technology

[0002] Fall detection technology helps to detect falls in children and the elderly in a timely manner, preventing more serious consequences. Currently, most fall detection methods are designed for specific scenarios, such as sports, and are suitable for younger people, or rely on wearable devices to determine if a fall has occurred. These methods have limitations: firstly, they are less effective for the elderly with slower movements; secondly, wearable devices need to be worn constantly, which is inconvenient and unsuitable for special situations like bathing. In private places such as bathrooms and bedrooms, to balance the need for privacy and monitoring, low-resolution infrared devices are often used instead of high-resolution RGB cameras as monitoring tools. Infrared thermal imaging cameras only collect the target's temperature information, resulting in lower resolution and ensuring user privacy.

[0003] In some solutions known to the inventors of this application, high-frame-rate, low-resolution infrared thermal imaging equipment is used to acquire temperature distribution data in a scene and generate infrared images for capturing and analyzing whether a person has fallen. However, since the accuracy of fall detection models built based on low-resolution infrared images is difficult to meet the requirements of commercial applications, fall detection models often have a lot of false alarms, and frequent false alarms will erode user trust. Summary of the Invention

[0004] The main technical problem solved by the embodiments of this application is to provide a method, a fall detection method and related device for detecting whether a human body is in a fallen lying position, which can accurately detect whether a human body is in a fallen lying position, and further calibrate the fall detection results, which is beneficial to reduce fall misjudgments.

[0005] In a first aspect, embodiments of this application provide a method for detecting a human body maintaining a fallen, lying position, including:

[0006] Acquire N frames of infrared images, and select the first N frames from the N frames of infrared images. F The frame is an infrared image of the fall, followed by N. k The frame is an infrared image of the fall, N F +N k =N;

[0007] The hot spot in N frames of infrared images is tracked to obtain multiple tracking paths. The tracking paths are used to record the positional features and / or morphological changes of the hot spot in the time domain.

[0008] acquire a target path in which the human body is located from the plurality of tracking paths, the target path including the human body heat spot or a main part of the human body heat spot;

[0009] determine whether the human body maintains a falling lying posture according to a feature of the human body heat spot in the target path.

[0010] In some embodiments, the aforementioned tracking of the heat spots in the N frames of infrared images obtains the plurality of tracking paths, including:

[0011] traversing the N frames of infrared images, if a first target heat spot in a current frame of infrared images overlaps with a second target heat spot in a previous frame of infrared images, the first target heat spot is added to each tracking path in which the second target heat spot is located; wherein the first target heat spot is any one of the heat spots in the current frame of infrared images;

[0012] if a third target heat spot in the previous frame of infrared images splits into s heat spots in the current frame of infrared images, each tracking path in which the third target heat spot is located is copied to obtain s paths, and the s heat spots are added to each tracking path in the s paths one by one; wherein the third target heat spot is any one of the heat spots in the previous frame of infrared images;

[0013] after the N frames of infrared images are traversed, the plurality of tracking paths are obtained.

[0014] In some embodiments, the method further includes:

[0015] if a tracking path is not added to a heat spot for a preset number of consecutive frames, the tracking path is removed from the plurality of tracking paths;

[0016] if a new heat spot appears in the current frame of infrared images, a new tracking path is generated with the new heat spot as a starting point, and the new tracking path is added to the plurality of tracking paths; wherein the new heat spot is a heat spot that does not appear in the previous frame of infrared images.

[0017] In some embodiments, the target path in which the human body is located is acquired from the plurality of tracking paths, including:

[0018] traversing the plurality of tracking paths, and screening out tracking paths in which the number of heat spots is greater than or equal to N k +x, to obtain a plurality of first candidate paths, wherein x is an integer greater than or equal to 1;

[0019] traversing the plurality of first candidate paths, and screening out first candidate paths in which a first distance between the last N k heat spots and the first x heat spots is greater than or equal to a first distance threshold, to obtain a plurality of second candidate paths;

[0020] traversing the plurality of second candidate paths, and screening out second candidate paths in which a second distance between the last N kThe area of each hot spot and the maximum second candidate path, to obtain a plurality of third candidate paths;

[0021] According to the plurality of third candidate paths, the target path is determined.

[0022] In some embodiments, the aforementioned determination of the target path according to the plurality of third candidate paths comprises:

[0023] According to the N frames of infrared images, the static hot spots in the N frames of infrared images are obtained;

[0024] Traverse the plurality of third candidate paths, and screen out the third candidate paths in which the overlap between the last N k hot spots is higher than the overlap threshold, to obtain a plurality of fourth candidate paths;

[0025] Traverse the plurality of fourth candidate paths, and screen out the fourth candidate path with the maximum first distance between the last N k hot spots and the first x hot spots as the target path.

[0026] In some embodiments, the aforementioned first distance between the last N k hot spots and the first x hot spots is calculated in the following manner:

[0027] Obtain the average area of the last N k hot spots;

[0028] Select the target hot spot with the area closest to the average area from the last N k hot spots;

[0029] Determine the first distance as the distance between the target hot spot and the first x hot spots.

[0030] In some embodiments, the aforementioned determination of the first distance as the distance between the target hot spot and the first x hot spots comprises:

[0031] Obtain the distance between the 4 vertices of the target hot spot and the 4 vertices of the first x hot spots, to obtain 4x vertex distances;

[0032] Obtain the distance between the centroid of the target hot spot and the centroid of the first x hot spots, to obtain x centroid distances;

[0033] Take the average of the 4x vertex distances and the x centroid distances as the first distance.

[0034] In some embodiments, the aforementioned obtaining of the static hot spots in the N frames of infrared images according to the N frames of infrared images comprises:

[0035] Obtain the segmentation threshold corresponding to the N frames of infrared images;

[0036] For each of the N frames of infrared images, data greater than or equal to a segmentation threshold is set to 1, and data less than the segmentation threshold is set to 0, to obtain N frames of binary images;

[0037] The first N F frames of the N frames of binary images are added to obtain an overlap image;

[0038] The position of the connected color block with the maximum value in the overlap image is taken as the position of the static hot spot, and the position of the static hot spot is mapped in the N frames of infrared images to obtain the static hot spot.

[0039] In some embodiments, before the step of obtaining N frames of binary images by setting data greater than or equal to a segmentation threshold to 1 and setting data less than the segmentation threshold to 0 for each of the N frames of infrared images, the method further comprises:

[0040] Gaussian filtering is performed on the N frames of infrared images frame by frame.

[0041] In some embodiments, the aforementioned determination of whether the human body maintains a fall lying posture comprises:

[0042] The last N k hot spots of the target path that intersect with the static hot spot are segmented to remove the static hot spot to obtain N k human body hot spots;

[0043] According to the characteristics of the N k human body hot spots, it is determined whether the human body maintains a fall lying posture, wherein the characteristics of the human body hot spots include temperature distribution characteristics and / or position characteristics.

[0044] In some embodiments, the aforementioned segmentation of the last N k hot sources of the target path that intersect with the static hot source to remove the static hot source to obtain N k human body hot sources comprises:

[0045] The last N k hot spots of the target path that intersect with the static hot spot are segmented to remove the static hot spot to obtain N k human body hot spots.

[0046] In some embodiments, the aforementioned determination of whether the human body maintains a fall lying posture according to the characteristics of the last N k human body hot spots comprises:

[0047] The high-temperature concentration points of each of the N k human body hot spots are determined to obtain N k high-temperature concentration points;

[0048] The high-temperature concentration points of each of the Nk a center point of each of the personal heat spots, obtaining N k center points;

[0049] According to the distance between each two of the N k high temperature concentration points and / or the distance between each two of the N k center points, it is determined whether the human body keeps the lying posture after falling down.

[0050] In some embodiments, the foregoing determining whether the human body keeps the lying posture after falling down according to the distance between each two of the N k high temperature concentration points and / or the distance between each two of the N k center points includes:

[0051] If there is a distance between two high temperature concentration points greater than or equal to a second distance threshold, and there is a distance between two center points greater than or equal to the second distance threshold, it is determined that the human body does not keep the lying posture after falling down.

[0052] If there is a distance between two high temperature concentration points greater than or equal to a third distance threshold, or there is a distance between two center points greater than or equal to the third distance threshold, it is determined that the human body does not keep the lying posture after falling down, wherein the third distance threshold is greater than the second distance threshold.

[0053] In some embodiments, the method further includes:

[0054] If the distance between any two high temperature concentration points and the distance between any two center points are both less than or equal to the second distance threshold, it is determined whether the difference between the maximum heat spot length and the minimum heat spot length in the N k personal heat spots exceeds a preset length threshold;

[0055] If not, it is determined that the human body keeps the lying posture after falling down.

[0056] In a second aspect, the embodiments of the present application provide a falling down detection method, including:

[0057] After preliminarily detecting that the human body falls down, the method for detecting whether the human body keeps the lying posture after falling down as in the first aspect is used to detect whether the human body keeps the lying posture after falling down.

[0058] If the human body keeps the lying posture after falling down, it is determined that the human body falls down and an early warning of falling down is output.

[0059] If the human body does not keep the lying posture after falling down, it is determined that the falling down is misjudged.

[0060] In a third aspect, the embodiments of the present application provide an electronic device, including:

[0061] at least one processor, and

[0062] The memory is in communication connection with the at least one processor, wherein

[0063] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0064] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are used to enable a computer device to perform the method of the first aspect.

[0065] The method for detecting whether a human body keeps a lying posture after falling provided by some embodiments of the present application first acquires N frames of infrared images, the first N frames of infrared images are infrared images in a falling process, and the last N frames of infrared images are infrared images after falling. F N frames of infrared images are acquired, the first N frames of infrared images are infrared images in a falling process, and the last N frames of infrared images are infrared images after falling. k N frames of infrared images are acquired, the first N frames of infrared images are infrared images in a falling process, and the last N frames of infrared images are infrared images after falling. F N frames of infrared images are acquired, the first N frames of infrared images are infrared images in a falling process, and the last N frames of infrared images are infrared images after falling. k N frames of infrared images are acquired, the first N frames of infrared images are infrared images in a falling process, and the last N frames of infrared images are infrared images after falling. BRIEF DESCRIPTION OF DRAWINGS

[0066] One or more embodiments are illustrated by way of example in the figures that are part of this document, and which illustrate by way of example the principles of the embodiments. The embodiments are not limited to the examples of drawings. Elements having the same reference numerals in the figures indicate like elements unless expressly stated otherwise. The figures in the drawings are not to scale and are provided for purposes of explanation only.

[0067] Figure 1 It is a schematic diagram of infrared images in some embodiments of the present application;

[0068] Figure 2 It is a schematic diagram of infrared images in some embodiments of the present application;

[0069] Figure 3 It is a schematic diagram of infrared images in some embodiments of the present application;

[0070] Figure 4 It is a schematic diagram of infrared images in some embodiments of the present application;

[0071] Figure 5 Flowchart of the method for detecting the lying posture of a human body in some embodiments of the present application;

[0072] Figure 6 Schematic diagram of heat spot tracking in some embodiments of the present application;

[0073] Figure 7 Schematic diagram of vertex distance and centroid distance in some embodiments of the present application;

[0074] Figure 8 Schematic diagram of filtering discrete color blocks of a binary image in some embodiments of the present application;

[0075] Figure 9 Schematic diagram of obtaining a static heat source in some embodiments of the present application;

[0076] Figure 10 Schematic diagram of heat spot segmentation before and after Gaussian filtering in some embodiments of the present application;

[0077] Figure 11 Schematic diagram of a binary image in some embodiments of the present application;

[0078] Figure 12 Schematic diagram of binary images before and after segmentation to remove static heat spots in some embodiments of the present application;

[0079] Figure 13 Schematic diagram of segmentation of a static heat spot in some embodiments of the present application;

[0080] Figure 14 Schematic diagram of segmentation of a static heat spot in some embodiments of the present application;

[0081] Figure 15 Schematic diagram of the length of a human body heat spot in some embodiments of the present application;

[0082] Figure 16 Flowchart of the method for detecting a fall in some embodiments of the present application. DETAILED DESCRIPTION

[0083] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0084] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0085] It should be noted that the various features of the embodiments of the present application can be combined with each other, and are within the protection scope of the present application, if there is no conflict. In addition, although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. In addition, the terms "first", "second", "third" and the like used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same function and effect.

[0086] Unless otherwise defined, all technical and scientific terms used in the specification are the same as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not used to limit the present application. The term "and / or" used in the specification includes any and all combinations of one or more related listed items.

[0087] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict.

[0088] In order to facilitate the understanding of the method provided by the embodiments of the present application, first, the terms involved in the embodiments of the present application are introduced:

[0089] (1) Infrared thermal imaging technology

[0090] Infrared thermal imaging technology uses photoelectric technology to detect the infrared signal of a specific wave band of the thermal radiation of an object, converts the signal into an infrared imaging image that can be distinguished by human vision, and can further calculate the temperature value. The value of each pixel point in the infrared imaging image is the temperature value of the object in the corresponding world coordinate system. Thus, the infrared thermal imaging technology enables humans to overcome visual barriers, so that people can "see" the temperature distribution of the object surface.

[0091] For example, an infrared camera is installed on the ceiling of a room to capture and collect infrared videos of the human body in the room. It can be understood that the infrared video includes a plurality of continuous infrared images. Please refer to Figure 1 , Figure 1 A frame of infrared image collected when a low-resolution (24*32) infrared camera is installed on the ceiling above the ground. Figure 1A group of irregular highlighted areas is the temperature distribution of a human body lying on the ground, and the temperature of the human body is higher than that of other background areas.

[0092] Before introducing the embodiments of the present application, the fall detection method known to the inventors of the present application is briefly introduced to facilitate the understanding of the subsequent embodiments of the present application.

[0093] In some schemes, a method and system for action recognition in infrared video are provided. The method comprises the following steps: S1: collecting infrared video data; S2: inputting the obtained infrared video data into a pre-set dilated three-dimensional convolutional neural network (a pre-trained detection model), and recognizing the infrared video data by using the dilated three-dimensional convolutional neural network; and S3: obtaining the recognition result of the infrared video data.

[0094] In this scheme, the fall action is recognized to detect the fall. If the fall action is found, it is determined that the human body falls. However, the accuracy of the trained detection model is difficult to meet the requirements of commercial applications due to the influence of the training data which cannot be exhausted. The detection model usually has more false reports of falls, and frequent false reports will lack the trust of users.

[0095] In some schemes, a bedside fall detection method is provided. Based on a thermoelectric infrared array sensor, a thermal image of a bedside area is obtained. Based on a constraint condition, the thermal image is processed to obtain a processed thermal image. Based on the processed thermal image, a human body area is locked. The human body area is subjected to fall recognition and judgment to obtain a judgment result. After the fall is judged, the body movement in the human body area is monitored based on a radar sensor, and the body movement time is counted. Based on the body movement time, the human body area is subjected to auxiliary judgment of fall recognition.

[0096] In this scheme, an additional radar sensor is introduced to assist in determining whether the human body maintains the fall posture after the fall is detected by the infrared sensor, which increases the cost of the equipment and the complexity of installation and debugging.

[0097] In some schemes, after the human body is preliminarily detected to fall, a difference method is used to determine whether the human body still lies down after falling, to assist in judging the fall detection result. However, as shown in Figure 2 Due to unstable environmental temperature, noise in the data collected by the equipment, and other problems, the same segmentation threshold is used to segment the continuous 3 frames of infrared images, and the difference of the extracted static human body heat spots is large. Therefore, it is impossible to use a simple difference method to determine whether the human body maintains the fall lying posture after falling.

[0098] To solve the above problems, some embodiments of the present application provide a method for detecting whether a human body maintains a fall lying posture, a fall detection method, and related devices. The method for detecting whether a human body maintains a fall lying posture first acquires N frames of infrared images. The first N infrared images are used as training data to train a detection model. The detection model is used to determine whether a human body maintains a fall lying posture.F Frame is infrared image in falling process, N k Frame is infrared image after falling, N F +N k =N. Track hot spots in N frames of infrared images to obtain multiple tracking paths. Obtain a target path where the human body is located from the multiple tracking paths. According to the target path, determine whether the human body maintains a lying posture after falling. In this embodiment, the target path corresponding to the human body is determined by tracking N frames of infrared images including the falling process and the human body after falling. Based on the characteristics of the hot spots in the target path and in combination with the characteristics of the hot spots corresponding to the human body maintaining the lying posture after falling, it can be accurately analyzed whether the human body maintains the lying posture after falling. If the human body maintains the lying posture after falling, it indicates that the falling detection result is correct, and if the human body does not maintain the lying posture after falling, it indicates that the falling detection result is incorrect. That is, after the human body is preliminarily detected to fall, the falling detection result is further corrected in this way, which is beneficial to reduce falling misjudgment.

[0099] The electronic device for implementing the method for detecting whether the human body maintains the lying posture after falling or the falling detection method provided by the embodiments of the present application is described below. It can be understood that the electronic device has a computing processing capability, and the electronic device in the embodiments of the present application can be a monitoring device for on-site monitoring, or a terminal or a server in communication connection with the monitoring device.

[0100] In some embodiments, the monitoring device is an infrared thermal imaging device, such as Figure 3 As shown in FIG. 1, the infrared thermal imaging device is installed on the ceiling corresponding to the target area in the room, at a distance h from the ground, and faces the target area. The infrared thermal imaging device is in communication connection with the electronic device. Thus, the infrared imaging device can send the N frames of infrared images collected to the electronic device for implementing the method for detecting whether the human body maintains the lying posture after falling or the falling detection method.

[0101] Please refer to Figure 4 , the electronic device 100 includes a processor 101 and a memory 102 in communication connection. Here, the communication connection can be connected by a bus, Figure 4 which is exemplarily illustrated by a bus connection. It can be understood that Figure 4 the electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0102] Based on the communication connection between the electronic device and the infrared thermal imaging device, when the infrared thermal imaging device collects N frames of infrared imaging images in succession, the N frames of infrared imaging images are sent to the electronic device. Thus, the processor can obtain the N frames of infrared imaging images.

[0103] The processor 101 is configured to support the electronic device 100 to perform the corresponding functions in the method for detecting a human body maintaining a falling lying posture or the falling detection method. The processor 101 can be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0104] The memory 102, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method for detecting a human body maintaining a falling lying posture or the falling detection method in the embodiments of the present application. The processor 101 can implement the method for detecting a human body maintaining a falling lying posture or the falling detection method in any one of the method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 102.

[0105] The memory 102 can include a volatile memory (VM), such as a random access memory (RAM); the memory 1002 can also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); and the memory 102 can further include a combination of the above types of memories.

[0106] It is understood that electronic devices also include other supporting hardware and software. Hardware may include antennas, various sensors, microphones, etc. Software may include operating systems, which are programs that manage and control the hardware and software resources of electronic devices. Software may also include various applications (apps). Other parts of the electronic device not involved in the embodiments of this application will not be described here.

[0107] It is worth noting that in some embodiments, the electronic device can also be integrated with the infrared thermal imaging device as a monitoring device. In some embodiments, the electronic device can also be a terminal or server, etc. In this application embodiment, no limitation is made on the form of the electronic device, as long as it can acquire infrared image sequences and has computing processing capabilities.

[0108] As can be understood from the above, the method for detecting a human body maintaining a fallen lying posture provided in this application embodiment can be implemented by various types of electronic devices with processing capabilities, such as being executed by the processor of an electronic device or by other devices with computing capabilities. Other devices with computing capabilities can be smart terminals or servers that are communicatively connected to the electronic device.

[0109] The target heat source segmentation method provided in this application is described below with reference to exemplary applications and implementations of the electronic devices provided in the embodiments of this application. Please refer to... Figure 5 , Figure 5 This is a flowchart illustrating a method for detecting a human body maintaining a fallen, lying position, as provided in an embodiment of this application. It is understood that the execution entity of this method for detecting a human body maintaining a fallen, lying position can be one or more processors of an electronic device.

[0110] like Figure 5 As shown, the method S100 may specifically include the following steps:

[0111] S10: Acquire N frames of infrared images. The first N frames of these N frames... F The frame is an infrared image of the fall, followed by N. k The frame is an infrared image of the fall, N F +N k =N.

[0112] N-frame infrared images are infrared videos captured by an infrared camera in a real-world application scenario, which can reflect changes in human movement over time. In some embodiments, the infrared camera sends the captured infrared video to an electronic device. When the electronic device detects a human fall based on these infrared videos, it acquires N-frame infrared images of the vicinity before and after that moment. These N-frame infrared images cover the human's fall and subsequent movements.

[0113] In the N frames of infrared images, the first N frames are infrared images during the falling process, and the last N frames are infrared images after falling. F k In the N frames of infrared images, the first N frames are infrared images during the falling process, and the last N frames are infrared images after falling. F k In the N frames of infrared images, the first N frames are infrared images during the falling process, and the last N frames are infrared images after falling.

[0114] S20: Track the hot spots in the N frames of infrared images to obtain a plurality of tracking paths.

[0115] Based on the N frames of infrared images being time series data, if the human body moves in this time period, it is reflected in the N frames of infrared images, and the human body hot spot will move between frames in the N frames of infrared images. For static interference heat sources, such as hot water basin, heated toilet, hot towel, a hot or cold water, sunlight, etc., which are reflected in the N frames of infrared images, the static hot spots will not move between frames in the N frames of infrared images.

[0116] It can be understood that there can be multiple different hot spots in the first frame of infrared image in the N frames of infrared images. Each hot spot is tracked frame by frame in the N frames of infrared images to obtain a plurality of tracking paths. The tracking path is a time series data of a hot spot, including a plurality of hot spots arranged in time sequence. In some cases, the hot spot corresponding to the human body can change position in the N frames of infrared images, and can also overlap with the static hot spot. In some cases, as the human body behaves on the surrounding objects, such as moving or introducing static heat source, the static hot spot in the N frames of infrared images will also change position. In some cases, the target area collected by the infrared camera can enter new personnel or pets, so that new hot spots will be added in the N frames of infrared images.

[0117] Therefore, the hot spots in the N frames of infrared images are tracked to obtain a plurality of tracking paths. Each tracking path records the position characteristics or morphological changes of a hot spot in the time domain, that is, each tracking path records the position characteristics or action changes of an object (human body or static heat source) in the time domain.

[0118] In some embodiments, the foregoing step S20 specifically comprises:

[0119] S21: Traverse the N frames of infrared images, and if a first target hot spot in the current frame of infrared image overlaps with a second target hot spot in the last frame of infrared image, add the first target hot spot to each tracking path in which the second target hot spot is located. Wherein the first target hot spot is any one of the hot spots in the current frame of infrared image.

[0120] ​​For any hotspot (the first target hotspot) in the current frame of the infrared image, it is matched with a hotspot in the previous frame of the infrared image. If the matched hotspot overlaps with the second target hotspot in the previous frame of the infrared image at a pixel position, the first target hotspot is added to each tracking path where the second target hotspot is located. Here, overlap means that there is an intersection at a pixel position.

[0121] like Figure 6 As shown, in the first frame of the infrared image, p1 is a human hotspot, and s1 is a static hotspot. Two paths, R1 and R2, are initialized, containing hotspots R1:{p1} and R2:{s1}, respectively. In the second frame of the infrared image, the human hotspot and the static hotspot merge into a single hotspot sp2. Since sp2 overlaps with both p1 and s1 from the previous frame, sp2 is matched to the paths containing p1 and s1, resulting in paths R1:{p1,sp2} and R2:{s1,sp2}.

[0122] S22: If the third target hot spot in the previous infrared image splits into s hot spots in the current infrared image, then copy each tracking path where the third target hot spot is located to obtain s paths, and add the s hot spots one by one to each tracking path in s paths.

[0123] The third target hotspot is any hotspot in the previous frame of the infrared image. For any hotspot in the previous frame (the third target hotspot), fission occurs in the current infrared image, meaning one hotspot becomes *s* hotspots. In some embodiments, *s* can be 2 or 3, etc. For example, if the third target hotspot is tracked in two tracking paths, the two tracking paths containing the third target hotspot are copied into *s* copies, resulting in *s* tracking paths, each including the previous two tracking paths. Then, the *s* hotspots are added one-to-one to each of the *s* tracking paths.

[0124] like Figure 6 As shown, in the third frame of the infrared image, the human body leaves the static heat source, which is reflected in the infrared image as the hot spot sp2 splitting into two hot spots p3 and s3. Since hot spot p3 intersects with hot spot sp2 in the second frame, path R1 becomes R1:{p1,sp2,p3}, and path R2 becomes R2:{s1,sp2,p3}. Similarly, hot spot s3 also intersects with hot spot sp2 in the second frame. At this time, since R1 has already matched hot spot p3 in the third frame, it cannot match hot spot s3 again. To ensure complete tracking, the original path R1 is copied to become R3, and R3 tracks s3. Similarly, the original R2 is copied to become R4, thus obtaining the new paths R3:{p1,sp2,s3} and R4:{s1,sp2,s3}.

[0125] In some embodiments, the human thermal spot p3 has some regions with lower temperature due to temperature fluctuation, and when the thermal spot is segmented by using the segmentation threshold, the human body and the leg are segmented, forming thermal spots p41 and p42, and the thermal spots p41 and p42 respectively have intersection with the thermal spot p3 in the third frame. Therefore, R1 and R2 need to be split into paths R5 and R6 respectively, and the following paths are obtained:

[0126] R1:{p1,sp2,p3,p41},R2:{s1,sp2,p3,p41}

[0127] R5:{p1,sp2,p3,p42},R6:{s1,sp2,p3,p42}

[0128] The thermal spot s4 intersects with the thermal spot s3 in the third frame, and therefore, the paths R3 and R4 become:

[0129] R3:{p1,sp2,s3,s4},R4:{s1,sp2,s3,s4}

[0130] As can be seen from the above process, the human body p1 and the static heat source s1 in the scene form 6 thermal spot tracking paths after 4 frames of dynamic changes of the human body, and the 6 tracking paths can clearly express the origin of the thermal spot block in the current frame.

[0131] S23: After the N frames of infrared images are traversed, a plurality of tracking paths are obtained.

[0132] The above is only an example of 4 frames of infrared images, 1 human body heat source and 1 static heat source, and it can be understood that in actual scenes, there can be one or more human body heat sources and one or more static heat sources. The segmentation threshold is used to segment the thermal spots in the N frames of infrared images, and after removing the background interference, the thermal spots corresponding to each heat source in the scene are obtained. In some embodiments, for each frame of infrared image in the N frames of infrared images, data greater than or equal to the segmentation threshold is set to 1, and data less than the segmentation threshold is set to 0, to obtain N binary images.

[0133] Then, the N binary images are traversed, and each thermal spot is tracked by using the method in steps S21 and S22 to obtain a plurality of tracking paths. Therefore, the tracking paths can reflect the dynamic changes of each thermal spot in the time domain.

[0134] In some embodiments, the method S100 further comprises: if a tracking path is not added to the hot spot for a preset number of consecutive frames, removing the tracking path from the plurality of tracking paths. If a new hot spot appears in the current frame of the infrared image, generating a new tracking path starting from the new hot spot, and adding the new tracking path to the plurality of tracking paths; wherein the new hot spot does not appear in the hot spots in the previous frame of the infrared image.

[0135] wherein the preset number of frames is a pre-set empirical value, for example, can be 2 frames or 5 frames, etc. For example, if a certain tracking link does not match the hot spot for 2 consecutive frames, it means that the tracking path fails to track the hot spot, and thus the tracking path is removed from the plurality of tracking paths. Removing these tracking failed paths can effectively reduce invalid data, and is conducive to making subsequent calculation more lightweight.

[0136] It can be understood that if the target area captured by the infrared camera can introduce new personnel, pets or static heat sources, new hot spots will be added in the N frames of infrared images. For the hot spot that suddenly appears in the middle, a new tracking path is generated starting from the hot spot, and the new tracking path is added to the plurality of tracking paths. Thus, each heat source can be tracked effectively, and the plurality of tracking paths corresponding to the N frames of infrared images are more accurate.

[0137] It can be understood that a tracking path records hot spots arranged in time sequence, i.e. each pixel point of the hot spot needs to be recorded. In some embodiments, the two-dimensional coordinates of the pixel points in the binary image are represented by one-dimensional coordinates. In some embodiments, the one-dimensional coordinate of the pixel point (i, j) is i*the width of the infrared image + j. For example, for an infrared image of 24*32 size, the one-dimensional coordinate of the pixel point (2, 4) is 2*32+4=68. Each pixel point in the binary image has a one-dimensional coordinate, and the pixel points are expanded by row, such as the pixel points in the 2nd row are spliced at the rightmost of the pixel points in the 1st row, and so on, so as to expand the two-dimensional binary image into a one-dimensional sequence.

[0138] Thus, the hot spots in the binary image can be represented by one-dimensional coordinate sequence and feature data, for example, p1=[68, 69, 99, 100,...]. Representing the hot spots by one-dimensional coordinate sequence facilitates the operation of the hot spots. In some embodiments, whether there is an intersection between two hot spots is determined by judging whether there is the same value in the two one-dimensional coordinate sequences.

[0139] In some embodiments, in addition to recording the one-dimensional coordinate sequence corresponding to the hot spot pixel points, the tracking path also records the feature data of the hot spot, such as the four vertices and the center of mass coordinates of the circumscribed rectangle of the hot spot, which facilitates subsequent determination of the shape position feature of the hot spot, and the position and shape change in time sequence, etc. For example, for N frames of binary images N th, after heat spot tracking, M heat spot tracking paths R are obtained M , and feature data F corresponding to each heat spot in the M paths M In some embodiments, the data examples of the tracking path R1 and the corresponding feature F1 are as follows:

[0140] R1:{[68,69,99,100,...],[68,69,99,100,101,102,...],......}

[0141] F1:{[(2,3),(2,5),(3,3),(3,5),(2.5,4)],[(2,3),(2,6),(3,3),(3,6),(2.5,4.5)],......}

[0142] In this embodiment, the two-dimensional binary image is unfolded into a one-dimensional sequence, and the pixel points in the infrared image are represented by one-dimensional coordinates, which facilitates the operation of the heat spot and is conducive to improving the accuracy of heat spot tracking.

[0143] S30: Obtain a target path in which the human body is located from the multiple tracking paths.

[0144] In the embodiment shown in Figure 6 , the paths R1 and R2 can fully reflect the position or shape change of the human body heat spot in the time domain, the paths R3 and R4 can fully reflect the change of the static heat spot in the time domain, and the paths R5 and R6 have defects in reflecting the human body heat spot. Therefore, here, the paths R1 and R2 are the target path in which the human body is located. That is, each heat spot in the target path includes the human body heat spot or the main part of the human body heat spot.

[0145] In order to subsequently determine whether the human body maintains the lying posture after falling, the target path in which the human body is located is obtained from the multiple tracking paths, so as to analyze the shape features of the human body based on the target path and determine whether the lying posture after falling is maintained.

[0146] In some embodiments, the foregoing step S30 specifically includes:

[0147] S31: Traverse the multiple tracking paths, and select a tracking path in which the number of heat spots is greater than or equal to N k +x, to obtain multiple first candidate paths, wherein x is an integer greater than or equal to 1.

[0148] It can be understood that x is a pre-set empirical value, 1≤x<N k . x is actually the number of frames that need to be selected from N F frames. In some embodiments, x can be 3.

[0149] In order to determine whether the human body maintains the lying posture after falling, NF At least 3 frames corresponding to the hot spot in the frame judge the dynamic human body hot spot, and then N K The hot spot in a tracking path needs at least NK+x. If the number of hot spots in a tracking path is less than the number, the tracking path is directly excluded from the plurality of tracking paths to obtain a plurality of first candidate paths.

[0150] It can be understood that due to tracking loss or dynamic change of heat source, some invalid tracking paths with a number of hot spots less than N k +x may be caused. In this embodiment, these invalid tracking paths are excluded to improve the calculation efficiency and reduce invalid operations.

[0151] S32: Traverse the plurality of first candidate paths, and select the first candidate paths whose first distances between the last N k hot spots and the first x hot spots are greater than or equal to the first distance threshold to obtain a plurality of second candidate paths.

[0152] It can be understood that in the target path, the first x hot spots can reflect the standing posture of the human body before falling down or falling down, and the last N k hot spots reflect the posture after falling down. The shape and position of the hot spots of the human body change greatly from before falling down to after falling down, that is, the distance between the last N K hot spots and the first x hot spots needs to be greater than or equal to the first distance threshold to meet the moving condition. In some embodiments, the first distance threshold can be 2, representing 2 pixel points.

[0153] If the distance between the last N K hot spots and the first x hot spots is less than the first distance threshold, it means that the hot spots in the target path do not have a falling process, and the falling detection result is a false positive. In some embodiments, when the first distance between the last N k hot spots and the first x hot spots in all tracking paths is less than the first distance threshold, it is determined that the falling detection result is a false positive.

[0154] Therefore, in order to select the target path, based on the shape and position change characteristics of the human body hot spots before and after falling in the target path, the first distance threshold is used to exclude the first candidate paths whose hot spot changes do not meet the shape and position characteristics before and after falling. The first candidate paths whose first distances between the last N k hot spots and the first x hot spots are greater than or equal to the first distance threshold are selected as the second candidate paths. After the plurality of first candidate paths are traversed, a plurality of second candidate paths are obtained.

[0155] In this embodiment, the first distance threshold is used to eliminate the first candidate path that does not conform to the shape position feature before and after falling down, so that the range of the obtained second candidate path is smaller, which is conducive to determining the target path. In addition, the invalid tracking path is eliminated, which is conducive to improving the calculation efficiency and reducing invalid operation.

[0156] In some embodiments, the first distance between the last N k spots and the first x spots is calculated in the following way:

[0157] S321: Obtain the average area of the last N k spots.

[0158] S322: Select a target spot with an area closest to the average area from the last N k spots.

[0159] S323: Determine the first distance as the distance between the target spot and the first x spots.

[0160] It can be understood that the area of a spot can be the number of pixel points it occupies. In some embodiments, the area of a spot can be determined by counting the length of its one-dimensional coordinate sequence.

[0161] After obtaining the areas of the last N k spots, the average area of the N k spots is calculated. Then, a target spot L with an area closest to the average area is selected from the last N k spots. Finally, the distance between the target spot and the first x spots is calculated as the first distance.

[0162] In this embodiment, the target spot with an area closest to the average area is less disturbed and can represent the spot in the lying state. Using the target spot as a reference to calculate the first distance makes the first distance more accurate.

[0163] In some embodiments, the aforementioned step S323 specifically includes:

[0164] (1) Obtain the distances between the 4 vertices of the target spot and the 4 vertices of the first x spots, respectively, to obtain 4x vertex distances.

[0165] (2) Obtain the distances between the centroids of the target spot and the centroids of the first x spots, respectively, to obtain x centroid distances.

[0166] (3) Take the average of the 4x vertex distances and the x centroid distances as the first distance.

[0167] Please refer to Figure 7, the 4 vertices of the target hot spot are the 4 vertices of the circumscribed rectangle, and the center of the target hot spot is the center of the circumscribed rectangle. The 4 vertex distances and the center distance between each of the first x hot spots and the target hot spot are calculated respectively. As shown in Figure 7 , d1, d2, d3, and d4 are vertex distances, and cd is the center distance.

[0168] It can be understood that the x hot spots correspond to 4x vertex distances and x center distances. Then, the mean of the 4x vertex distances and the x center distances is calculated, and this mean is taken as the first distance.

[0169] In this embodiment, by calculating the mean of the multiple vertex distances and center distances as described above, an accurate first distance can be calculated.

[0170] S33: Traverse the multiple second candidate paths to filter out the second candidate path with the largest area sum of the last N k hot spots, to obtain multiple third candidate paths.

[0171] It can be understood that, referring again to Figure 6 , the human body hot spot p3 in the 3rd frame is split into hot spots p41 and p42 due to temperature fluctuations causing some areas to have lower temperatures, and the segmentation threshold is used for hot spot segmentation, causing the person's body and legs to be segmented.

[0172] Considering the problem of splitting the human body hot spot, in this embodiment, the second candidate path with the largest area sum of the last N k hot spots is filtered out. It can be understood that there can be multiple second candidate paths with the largest area sum, for example, the area sums of the tracking paths R1 and R2 are equal. Therefore, multiple third candidate paths are obtained.

[0173] By filtering the second candidate path with the largest area sum as described above, the multiple third candidate paths obtained include the tracking path where the main part of the human body hot spot is located, and exclude the tracking path where the small part of the human body hot spot is located, to reduce the interference caused by the small part of the human body hot spot that is split.

[0174] S34: Determine the target path according to the multiple third candidate paths.

[0175] It can be understood that after the multiple tracking paths are filtered by the steps S31 to S33 described above, the remaining multiple third candidate paths are the candidate paths where the human body hot spot is located. Therefore, the target path can be determined from the multiple third candidate paths.

[0176] Considering that the last N KThe personal body heat spot can intersect with some static heat spots. If the personal body heat spot and the static heat spot have a high overlap degree, the recognition of the body lying posture can be affected. Based on this, in some embodiments, the third candidate path with a high overlap degree between the personal body heat spot and the static heat spot is eliminated.

[0177] In some embodiments, the foregoing step S34 specifically includes:

[0178] S341: Obtain static heat spots in the N frames of infrared images according to the N frames of infrared images.

[0179] It can be understood that the static heat spot does not change in position in the time domain, that is, the pixel position of the static heat spot in the N frames of infrared images is approximately the same. Therefore, the heat spot that does not change in position in the N frames of infrared images is the static heat spot.

[0180] In some embodiments, the foregoing step S341 specifically includes: obtaining a segmentation threshold corresponding to the N frames of infrared images. For each frame of infrared image in the N frames of infrared images, data greater than or equal to the segmentation threshold is set to 1, and data less than the segmentation threshold is set to 0, to obtain N frames of binary images. The first N frames of the N frames of binary images are added to obtain an overlap image. The position of the connected color block with the largest value in the overlap image is taken as the position of the static heat spot. The position of the static heat spot is mapped in the N frames of infrared images to obtain the static heat spot. F

[0181] The segmentation threshold is a temperature threshold for distinguishing heat spots and backgrounds. In some embodiments, the segmentation threshold can be a temperature threshold preset by a person skilled in the art according to the environmental temperature. In some embodiments, the segmentation threshold can also be obtained by a threshold determination algorithm. For example, for any one frame of the N frames of infrared images, the frame is subjected to translation processing, and the translated image is subjected to difference calculation with the frame of infrared image to obtain a heat spot edge region. The mean or mode of the pixel points in the heat spot edge region in the frame of infrared image is taken as the threshold of the frame of infrared image. Finally, the mean of the thresholds of the N frames of infrared images is taken as the segmentation threshold corresponding to the N frames of infrared images. In this embodiment, the segmentation threshold is determined by the edge pixel region of the heat source by means of translation processing and difference calculation, so that the segmentation threshold is not constrained by the area size of the heat source in the infrared imaging image, and the adaptability is wide.

[0182] After obtaining the segmentation threshold corresponding to the N frames of infrared images, refer to Figure 8 ​In step (a) of the infrared image, data greater than or equal to the segmentation threshold are set to 1, and data less than the segmentation threshold are set to 0, resulting in a binary image. In some embodiments, considering that the color patch corresponding to the human body cannot be less than 4 pixels, connected color patches less than 4 pixels in the binary image are deleted to remove interference, resulting in the image shown below. Figure 8 The binary image shown in (b) is shown in the image.

[0183] The first N frames of binary images F Frames are added together to obtain an overlapping image. See also... Figure 9 Taking two frames of binary images as an example, N is used for illustration. F After the binary images are summed pixel by pixel, the value corresponding to the overlapping part of the hot spot in the resulting overlapping image is 2, the value corresponding to the non-overlapping part of the hot spot is 1, and the value corresponding to the background part is 0.

[0184] Understandably, the more hotspots overlap in an NF frame, the larger the value of the corresponding connected patch in the overlapping image. That is, the connected patch with the largest value in the overlapping image indicates that it appeared in all the binary images used for addition, suggesting that the heat source corresponding to these patches is stationary and has not moved. The hotspot corresponding to this overlap is in N... F Since the position of the hot spot does not change within the frame, it is considered a static hot spot. Therefore, the location of the connected color block with the largest value in the overlapping image is taken as the location of the static hot spot. In some embodiments, color blocks with an area of ​​less than 4 pixels are filtered out from the connected color block with the largest value, resulting in the final connected color block, which is taken as the location of the static hot spot. Finally, this location of the static hot spot is mapped onto N frames of infrared images to obtain the static hot spot. In other words, obtaining the pixel location of the static hot spot in the infrared image is sufficient to obtain the static hot spot.

[0185] In this embodiment, static hot spots are extracted by superimposing multiple frames of binary images to reduce interference in determining human hot spots.

[0186] In some implementations, the method further includes performing Gaussian filtering on each of the N frames of infrared images before binarization.

[0187] It is understandable that the N frames of infrared images are captured by an infrared camera and contain a lot of noise, with the value of each pixel fluctuating around its actual value. In this embodiment, Gaussian filtering is applied to the N frames of infrared images to allow the pixels to return to their actual values ​​as much as possible, making the values ​​of adjacent pixels relatively smooth.

[0188] like Figure 10 As shown, compared with direct hot spot segmentation using infrared images, the infrared images processed by Gaussian filtering produce more solid hot spots with fewer gaps, and most zero heat dissipation spots are smoothed out by filtering.

[0189] In this embodiment, by performing Gaussian filtering on N frames of infrared images, then performing thermal binaryzation and hot spot extraction, the position of the determined hot spot is more accurate, so that the corresponding hot spot is more compact, and the scattered thermal noise is reduced.

[0190] S342: traverse the plurality of third candidate paths, and screen out the third candidate path in which there is no hot spot in the last N k hot spots whose overlap degree with the static hot spot is higher than an overlap threshold, to obtain a plurality of fourth candidate paths.

[0191] It is considered that the last N K hot spots in the third candidate path may intersect with the static hot spot, and if the overlap degree between the human body hot spot and the static hot spot is high, the recognition of the human body lying posture will be affected.

[0192] Here, the third candidate path in which there is no hot spot in the last N k hot spots whose overlap degree with the static hot spot is higher than an overlap threshold is screened out, to obtain a plurality of fourth candidate paths. The overlap threshold can be a pre-set proportional empirical value. If there is one or more hot spots in the last N k hot spots in the third candidate path whose overlap degree with the static hot spot is higher than the overlap threshold, the third candidate path is removed, and the remaining third candidate path is taken as the fourth candidate path.

[0193] In some embodiments, the overlap degree between two hot spots can be calculated by using the following formula:

[0194] Overlap degree = number of overlapping pixels between two hot spots / number of pixels of the larger hot spot between the two hot spots.

[0195] In this embodiment, by using the overlap threshold to remove the third candidate path whose overlap degree with the static hot spot is high, the remaining fourth candidate path is more effective, and the interference caused by the overlapping hot spot is reduced.

[0196] S343: traverse the plurality of fourth candidate paths, and screen out the fourth candidate path in which the first distance between the last N k hot spots and the first x hot spots is the largest, as the target path.

[0197] Finally, the fourth candidate path in which the first distance between the last N k hot spots and the first x hot spots is the largest is screened out from the plurality of fourth candidate paths, as the target path.

[0198] The first distance can be calculated in the manner described in step S32 above, which will not be repeated here. In this embodiment, the first distance is further compared in size. It can be understood that the first distance is the largest, indicating that the moving range of the hot spot is also the largest, and it is most likely a human body. Therefore, the fourth candidate path with the largest first distance is taken as the target path, so that the target path is more accurate.

[0199] S40: Determine whether the human body remains in the lying posture according to the target path.

[0200] It can be understood that the target path includes a human body hot spot. Based on the characteristics of the hot spot in the target path, in combination with the corresponding hot spot characteristics when the human body remains in the lying posture after falling, it can be accurately analyzed whether the human body remains in the lying posture after falling. If the human body remains in the lying posture, it indicates that the falling detection result is correct, and if the human body does not remain in the lying posture, it indicates that the falling detection result is incorrect. That is, after preliminarily detecting that the human body falls, the falling detection result is further corrected in this manner, which is beneficial to reduce the falling misjudgment.

[0201] In some embodiments, the foregoing step S40 specifically includes:

[0202] S41: Segmenting and removing the static hot spot from the hot spot in the last N k hot spots of the target path that has intersection with the static hot spot, to obtain the human body hot spot. k

[0203] Please refer to Figure 11 , the human body hot spot in the last N k hot spots of the target path can have intersection with the static hot spot. Figure 11 (a) and (b) in the figure are the hot spots of the same person in different frames when lying still. Figure 11 (a) in the figure has intersection with a hot spot due to data interference problem, and cannot be completely segmented. Figure 11 (b) in the figure, the human body hot spot does not have intersection with the static hot spot.

[0204] It can be understood that if the human body hot spot and the static hot spot have intersection and are not completely segmented, it will cause large deviation of the center of mass of the human body hot spot. If the center of mass of the human body hot spot deviates, it will be considered that the human body has moved and does not remain in the lying posture. Obviously, this is incorrect. In fact, the human body remains in the lying posture after falling. Therefore, segmenting the static hot spot and the human body hot spot is beneficial to the accuracy of the subsequent lying posture after falling. As Figure 12 shown, after segmenting and removing the static hot spot from (a) in the figure, the human body hot spot shown in (b) in the figure is obtained. Figure 12 Figure 12

[0205] ​​​In some embodiments, the foregoing step S41 specifically comprises: using the edge line of the circumscribed rectangle of the static hot spot to segment the hot spots in the last N k hot sources of the target path that have intersection with the static hot spot, to obtain N k personal body hot spots.

[0206] Referring to Figure 13 , first, the overlapping part C k of the body hot spots in the last N p hot spots of the target path is obtained. p It can be understood that the overlapping part C p of the body hot spots can be used to mark the common position of the body. If the number of pixel points of the overlapping part C k is 0, it is determined that the body does not maintain a lying posture, or the body has moved after falling.

[0207] As shown in (a) of Figure 13 , for any one hot spot B in the last N p hot spots of the target path that has intersection with the static hot spot, the edge line of the circumscribed rectangle of the static hot source is used as 4 segmentation lines, and the positions of the overlapping pixels C p are judged in the order of left, right, top and bottom. First, it is judged whether there are pixel points of C p on the left side of the segmentation line 1. If yes, the part of the hot spot B belonging to the right side of the segmentation line 1 is cut off to form the body hot spot as shown in (b) of Figure 13 , so as to complete the segmentation of the body hot spot and the static hot source. If there are no pixel points of C p on the left side of the segmentation line 1, it is judged whether there are pixel points of C p on the right side of the segmentation line 2. If yes, the part of the hot spot B belonging to the left side of the segmentation line 2 is cut off. If there are no pixel points of C p on the right side of the segmentation line 2, it is judged whether there are pixel points of C p on the top side of the segmentation line 3. If yes, the part of the hot spot B belonging to the bottom side of the segmentation line 3 is cut off. If there are no pixel points of C p on the top side of the segmentation line 3, it is judged whether there are pixel points of C p on the bottom side of the segmentation line 4. If yes, the part of the hot spot B belonging to the top side of the segmentation line 4 is cut off.

[0208] In some embodiments, referring to (a) of Figure 14 , if the overlapping pixels C p cover the area of 2 or more segmentation lines, the hot spot B is directly used to cut off the static hot source to obtain the body hot spot as shown in (b) of Figure 14 , so as to complete the segmentation of the body hot spot and the static hot spot.

[0209] When the last N k thermal spots of the target path intersect with the static thermal spot, the static thermal spot is segmented in the above manner, and N k personal thermal spots are obtained.

[0210] In this embodiment, the last N k thermal spots of the target path are respectively calculated with the static thermal spot, and for the thermal spots that intersect with the static thermal spot, the static thermal spot is segmented to remove the static thermal spot, and the personal thermal spots are obtained. Thus, N k personal thermal spots can reflect the real shape of the human body, which is conducive to the accuracy of the subsequent fall lying posture judgment result.

[0211] S42: Determine whether the human body maintains the fall lying posture according to the characteristics of the N k personal thermal spots.

[0212] The characteristics of the personal thermal spots include temperature distribution characteristics and / or position characteristics. The temperature distribution characteristics refer to the distribution of the temperature corresponding to the personal thermal spots. For example, the upper torso corresponds to a relatively high temperature area in the personal thermal spot, and the limbs correspond to a relatively low temperature area in the personal thermal spot. Based on the characteristics of the fall lying posture, the upper torso is stationary, and the limbs can move. Thus, by analyzing whether the high-temperature area in the personal thermal spot moves, it can be determined whether the human body maintains the fall lying posture. The position characteristics refer to the pixel position of the personal thermal spot. It can be understood that the pixel position of the personal thermal spot reflects the position of the human body in space. Based on the characteristics of the fall lying posture, the spatial position of the upper torso does not change or the spatial position of the limbs changes little. Thus, by analyzing the position characteristics of the personal thermal spot, it can be determined whether the human body maintains the fall lying posture.

[0213] Based on the characteristics of the N k personal thermal spots, the real shape of the human body can be reflected, and thus, in combination with the thermal spot characteristics corresponding to the human body maintaining the fall lying posture, it can be accurately analyzed whether the human body maintains the fall lying posture.

[0214] In some embodiments, the foregoing step S42 specifically includes:

[0215] S421: Determine the high-temperature concentration point of each personal thermal spot in the N k personal thermal spots, and obtain N k high-temperature concentration points.

[0216] S422: Determine the centroid point of each personal thermal spot in the N k personal thermal spots, and obtain N k centroid points.

[0217] S423: According to the distance between each two of the N k high-temperature concentration points and / or the distance between each two of the N kThe distance between each two of the N center points determines whether the human body keeps the lying posture after falling down.

[0218] It can be understood that, if N k If the high temperature concentration point and / or the center of each human body thermal spot in the human body thermal spot has a large deviation, it indicates that the human body has moved during this period and has not kept the lying posture after falling down. If N k If the high temperature concentration point and / or the center of each human body thermal spot in the human body thermal spot has a large deviation, it indicates that the human body has moved during this period and has not kept the lying posture after falling down. If N

[0219] In this embodiment, the high temperature concentration point of each human body thermal spot is first calculated. In some embodiments, the high temperature concentration point of the human body thermal spot is calculated using the following formula:

[0220]

[0221]

[0222]

[0223] wherein T i is the temperature value of the i-th pixel point in the human body thermal spot, T max is the maximum temperature value in the human body thermal spot, T min is the minimum temperature value in the human body thermal spot, W i is the weight of the temperature value of the i-th pixel point, h is the number of pixel points in the human body thermal spot, and (hx, hy) is the high concentration point of the human body thermal spot.

[0224] The above formula takes into account the weight of the temperature of each pixel point, so that the calculated high concentration point is more accurate. The high concentration point of each of the N k human body thermal spots is calculated using the above formula, and accurate N k high concentration points can be obtained.

[0225] Then, the center of each human body thermal spot is calculated. In some embodiments, the center of the human body thermal spot can be calculated using the following formula:

[0226]

[0227]

[0228] wherein (x i , y i ) is the two-dimensional pixel coordinate of the i-th pixel point in the human body thermal spot, (cx, cy) is the center of the human body thermal spot, and h is the number of pixel points in the human body thermal spot.

[0229] The center of each of the Nk the center of mass of the personal body heat spot, the accurate N k center of mass can be obtained.

[0230] Finally, according to the distance between each two of the N k high temperature concentration points and / or the distance between each two of the N k center of mass points, it is determined whether the human body maintains the fall lying posture. For example, if the distance between the high temperature concentration points is too large or the distance between the centers of mass is too large, it is determined that the human body moves and does not maintain the fall lying posture. If the distance between the high temperature concentration points is relatively small or the distance between the centers of mass is relatively small, it is determined that the human body maintains the fall lying posture.

[0231] In some embodiments, the foregoing step S423 specifically comprises:

[0232] If the distance between two high temperature concentration points is greater than or equal to a second distance threshold, and the distance between two centers of mass is greater than or equal to the second distance threshold, it is determined that the human body does not maintain the fall lying posture. If the distance between two high temperature concentration points is greater than or equal to a third distance threshold, or the distance between two centers of mass is greater than or equal to the third distance threshold, it is determined that the human body does not maintain the fall lying posture, wherein the third distance threshold is greater than the second distance threshold.

[0233] The second distance threshold can be an empirical value set by a person skilled in the art according to the angle of view or image resolution of the infrared camera. In some embodiments, the second distance threshold can be 2 pixel points.

[0234] For the distance between each two of the N k high temperature concentration points and the distance between each two of the N k center of mass points, if the distance between two high temperature concentration points is greater than or equal to a second distance threshold, and the distance between two centers of mass is greater than or equal to the second distance threshold, it is determined that the human body moves, and then it is determined that the human body does not maintain the fall lying posture.

[0235] It can be understood that, in some embodiments, if there is no distance greater than or equal to the second distance threshold in the high temperature concentration points and there is no distance greater than or equal to the second distance threshold in the centers of mass, that is, the distance between any two high temperature concentration points and the distance between any two centers of mass are less than or equal to the second distance threshold, it is determined that the human body maintains the fall lying posture.

[0236] The second distance threshold can be an empirical value set by a person skilled in the art according to the angle of view or image resolution of the infrared camera, and the third distance threshold is greater than the second distance threshold. For the distance between each two of the N k high temperature concentration points and the distance between each two of the N kIf the distance between any two high-temperature concentration points and the distance between any two center points are both less than or equal to the second distance threshold, it is determined that the human body does not have a large movement. In order to identify whether the human body has a body movement such as curling.

[0237] In some embodiments, the method S100 further comprises:

[0238] S50: If the distance between any two high-temperature concentration points and the distance between any two center points are both less than or equal to the second distance threshold, it is determined that the human body does not have a large movement. k whether the difference between the maximum length of the heat spot and the minimum length of the heat spot exceeds a preset length threshold.

[0239] S60: If the difference does not exceed the preset length threshold, it is determined that the human body maintains the lying posture after falling.

[0240] If the distance between any two high-temperature concentration points and the distance between any two center points are both less than or equal to the second distance threshold, it is determined that the human body does not have a large movement. In order to identify whether the human body has a body movement such as curling.

[0241] Referring to Figure 15 , each square block in the human heat spot represents a pixel point, and the heat spot length d is the diagonal length of the human heat spot. After obtaining N k heat spot lengths, it is determined whether the difference between the maximum length of the heat spot and the minimum length of the heat spot exceeds a preset length threshold. If the difference does not exceed the preset length threshold, it is determined that the human body does not have a large movement or a body movement, so that the human body maintains the lying posture after falling.

[0242] The preset length threshold is an empirical value for determining whether a body movement occurs, and can be set according to actual conditions by those skilled in the art.

[0243] In this embodiment, under the condition that it is determined that the human body does not have a large movement, the heat spot length is further used for evaluation to determine whether the human body has a body movement. When the human body does not have a body movement, it is determined that the human body maintains the lying posture after falling, so that the result is more accurate.

[0244] In summary, the method for detecting whether a human body maintains a lying posture after falling provided by some embodiments of the present application first obtains N frames of infrared images. The first N F frames of infrared images are infrared images during falling, the last N k frames of infrared images are infrared images after falling, and N F +N k=N. The hot spot in N frames of infrared images is tracked to obtain a plurality of tracking paths. A target path in which the human body is located is obtained from the plurality of tracking paths. Whether the human body maintains the falling lying posture is determined according to the target path. In this embodiment, the target path corresponding to the human body is determined by tracking N frames of infrared images including the falling process and the human body after falling. Based on the characteristics of the hot spot in the target path and the corresponding hot spot characteristics when the human body maintains the falling lying posture, whether the human body maintains the falling lying posture can be accurately analyzed. If the human body maintains the falling lying posture, it indicates that the falling detection result is correct. If the human body does not maintain the falling lying posture, it indicates that the falling detection result is incorrect. That is, after the human body is preliminarily detected to fall, the falling detection result is further corrected in this way, which is beneficial to reduce the falling misjudgment.

[0245] The method for detecting whether the human body maintains the falling lying posture in some embodiments of the present application is applied to the falling detection to assist in verifying the falling detection result. The falling detection method in the embodiments of the present application can be implemented by various types of electronic devices with computing processing capability, such as a smart terminal and a server.

[0246] The falling detection method provided by the embodiments of the present application is described below in combination with an exemplary application and implementation of a terminal provided by the embodiments of the present application. Referring to Figure 16 , Figure 16 is a flowchart of the falling detection method provided by the embodiments of the present application. The method S200 includes the following steps:

[0247] S201: After the human body is preliminarily detected to fall, the method for detecting whether the human body maintains the falling lying posture in any of the above embodiments is used to detect whether the human body maintains the falling lying posture.

[0248] S202: If the human body maintains the falling lying posture, it is determined that the human body falls and a falling warning is output.

[0249] S203: If the human body does not maintain the falling lying posture, it is determined that the human body misjudges falling.

[0250] In some embodiments, a pre-trained falling detection model can be used to detect the infrared image sequence. If the human body is preliminarily detected to fall, N frames of infrared images (the first N F frames are infrared images in the falling process, and the last N k frames are infrared images after falling) are obtained, and the method for detecting whether the human body maintains the falling lying posture in any of the above embodiments is used to detect whether the human body maintains the falling lying posture.

[0251] If the human body maintains the falling lying posture, it indicates that the falling detection result is correct. If the human body does not maintain the falling lying posture, it indicates that the falling detection result is incorrect. That is, after the human body is preliminarily detected to fall, the falling detection result is further corrected in this way, which is beneficial to reduce the falling misjudgment.

[0252] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separated, and the units displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0253] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus a general hardware platform, and of course can also be realized by hardware. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0254] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; under the idea of the present application, the technical features in the above examples or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of the different aspects of the present application as described above. In order to be brief, they are not provided in detail; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of detecting that a human body is holding a fallen lying position, characterized in that, include: N frames of infrared images are acquired, the first N F frames of infrared images in the N frames of infrared images are infrared images in a falling process, and the last N k frames of infrared images in the N frames of infrared images are infrared images after falling, N F +N k =N; The hot spots in the N frames of infrared images are tracked to obtain multiple tracking paths, wherein the tracking paths are used to record the positional features and / or morphological changes of the hot spots in the time domain; The target path where the human body is located is obtained from the multiple tracking paths, and the target path includes the human body hot spot or the main part of the human body hot spot; Based on the characteristics of the human body heat spots in the target path, determine whether the human body is in a fallen lying position; The step of tracking hot spots in the N frames of infrared images to obtain multiple tracking paths includes: Traverse the N frames of infrared images. If the first target hot spot in the current frame of infrared image overlaps with the second target hot spot in the previous frame of infrared image, then add the first target hot spot to each tracking path where the second target hot spot is located; wherein, the first target hot spot is any hot spot in the current frame of infrared image. If the third target hot spot in the previous frame infrared image splits into s hot spots in the current frame infrared image, then each tracking path where the third target hot spot is located is copied to obtain s paths, and the s hot spots are added one-to-one to each tracking path in the s paths; wherein, the third target hot spot is any hot spot in the previous frame infrared image; After the N frames of infrared images have been traversed, the multiple tracking paths are obtained.

2. The method of claim 1, wherein, The method further includes: If a tracking path is not added to the hot spot for a preset number of consecutive frames, then the tracking path is removed from the multiple tracking paths; If a new hot spot appears in the current frame of the infrared image, a new tracking path is generated starting from the new hot spot, and the new tracking path is added to the multiple tracking paths; wherein, the new hot spot is a hot spot that did not appear in the previous frame of the infrared image.

3. The method of claim 1, wherein, The step of obtaining the target path where the human body is located from the multiple tracking paths includes: Traverse the plurality of tracking paths, and screen out tracking paths with a number of hot spots greater than or equal to N k tracking paths of x, wherein x is an integer greater than or equal to 1; traverse the plurality of first candidate paths, and filter out the first candidate paths with a first distance between the last N hot spots and the first x hot spots being greater than or equal to a first distance threshold, to obtain a plurality of second candidate paths. k traverse the plurality of first candidate paths, and filter out the first candidate paths with a first distance between the last N hot spots and the first x hot spots being greater than or equal to a first distance threshold, to obtain a plurality of second candidate paths. traverse the plurality of second candidate paths, and screen out the last N k hot spots and the largest second candidate path to obtain a plurality of third candidate paths; The target path is determined based on the multiple third candidate paths.

4. The method of claim 3, wherein, Determining the target path based on the plurality of third candidate paths includes: Based on the N frames of infrared images, obtain the static hot spots in the N frames of infrared images; Traverse the multiple third candidate paths and filter out the last N. k A third candidate path is obtained for hot spots that do not have an overlap with the static hot spot that is higher than the overlap threshold, resulting in multiple fourth candidate paths. Traverse the multiple fourth candidate paths and filter out the last N. k The fourth candidate path with the largest first distance among the x hot spots is taken as the target path.

5. The method according to claim 3 or 4, characterized in that, The last N k The first distance between the last N hot spots and the first x hot spots is calculated in the following way: acquiring an average area of the last N k hot spots; selecting a target hot spot with an area closest to the average area from the last N k hot spots. The first distance is determined to be the distance between the target hot spot and the first x hot spots.

6. The method of claim 5, wherein, Determining the first distance as the distance between the target hotspot and the first x hotspots includes: Obtain the distances between the four vertices of the target hot spot and the four vertices of the previous x hot spots, thus obtaining the 4x vertex distances; Obtain the distances between the centroid of the target hot spot and the centroids of the previous x hot spots to obtain x centroid distances; The average of the 4x vertex distances and the x centroid distances is taken as the first distance.

7. The method of claim 4, wherein, The step of obtaining static hot spots in the N frames of infrared images includes: Obtain the segmentation threshold corresponding to the N frames of infrared images; For each of the N infrared images, data that is greater than or equal to the segmentation threshold is set to 1, and data that is less than the segmentation threshold is set to 0, thus obtaining N binary images; The first N frames of the N binary images F Frames are added together to obtain an overlapping image; The position of the largest numerical connected color block in the superimposed image is taken as the position of the static hot spot, and the position of the static hot spot is mapped in the N frames of infrared images to obtain the static hot spot.

8. The method of claim 7, wherein, Before the step of setting data greater than or equal to the segmentation threshold to 1 and setting data less than the segmentation threshold to 0 to obtain N frames of binary images, the method further comprises: Gaussian filtering is performed on the N frames of infrared images frame by frame.

9. The method of claim 4, wherein, The method further comprises: N hot spots of the target path that have intersection with the static hot spot, and the static hot spot is segmented and removed, to obtain N k personal body hot spots k personal body hot spots According to the N k characteristics of the personal thermal spot, determining whether the human body keeps a fall lying position, wherein the characteristics of the personal thermal spot include temperature distribution characteristics and / or position characteristics.

10. The method of claim 9, wherein, the target path, and the N k thermal sources intersecting with the static thermal spot, removing the static thermal spot, obtaining N k personal thermal spots, comprising: Using the outer rectangular edge of the static hotspot, the last N of the target path are... k The hot spots that intersect with the static hot spots among the hot spots are divided into the static hot spots to obtain the N hot spots. k Individual body heat spots.

11. The method of claim 9, wherein, said according to the post-N k characteristics of the personal body heat patch, determining whether the human body maintains a fall lying position, comprising: determining the N k a high temperature concentration point for each of the N k high temperature concentration points; determining the N k a center of mass point for each of the N k personal thermal spots, obtaining N According to the distance between each two of the N k high-temperature concentration points and / or the distance between each two of the N k centroid points, it is determined whether the human body maintains a fall-down lying posture.

12. The method of claim 11, wherein, The N k high-temperature concentration points and / or the distance between each of the N k centroid points, to determine whether the human body maintains a fall-down lying posture, comprising: If the distance between the two high-temperature concentration points is greater than or equal to a second distance threshold, and the distance between the two centroid points is greater than or equal to the second distance threshold, it is determined that the human body does not maintain the fall lying posture. If the distance between the two high-temperature concentration points is greater than or equal to a third distance threshold, or the distance between the two centroid points is greater than or equal to the third distance threshold, it is determined that the human body does not maintain the fall lying posture, wherein the third distance threshold is greater than the second distance threshold.

13. The method of claim 12, wherein, The method further comprises: If the distance between any two high-temperature concentration points and the distance between any two mass center points are both less than or equal to the second distance threshold, it is determined that the N k whether the difference between the maximum spot length and the minimum spot length in the personal body heat spot exceeds a preset length threshold; If the distance between the two high-temperature concentration points is greater than or equal to a second distance threshold, and the distance between the two centroid points is greater than or equal to the second distance threshold, it is determined that the human body does not maintain the fall lying posture.

14. A fall detection method characterized by, If the human body maintains the fall lying posture, it is determined that a fall has occurred and a fall warning is output. If the human body does not maintain the fall lying posture, it is determined that a false fall has occurred. The method further comprises: at least one processor, and 15. An electronic device, comprising: a memory connected in communication with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-14. The computer readable storage medium stores computer executable instructions for causing a computer device to perform the method of any one of claims 1-14. ​ 16. A computer-readable storage medium, characterized in that, ​

Images