Human body abnormal posture recognition method, device and computer readable storage medium
By extracting the human edge contour and head feature points, combining area and movement trajectory changes, the abnormal posture of the human body is quickly recognized, solving the problem of large amount of calculation in traditional technology, and improving the real-time and accuracy of recognition.
Patent Information
- Application Number
- CN202011595393.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-29
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-12-29
AI Technical Summary
Traditional human abnormal posture recognition technology has the problem of large amount of calculation and poor real-time performance in two-dimensional and three-dimensional conversions. The existing technology focuses more on improving model accuracy rather than reducing calculation, making it difficult to achieve real-time monitoring.
By obtaining real-time video frame images, extracting the human edge contour, finding the head position and tracking feature points, calculating the ratio of the head and upper body area, judging the posture based on horizontal and vertical cross-sectional areas, and using the movement trajectory changes of the head reference point to judge abnormal postures , send an alarm signal.
It reduces the amount of calculation of abnormal posture recognition in human body, improves the real-time and accuracy of recognition, and is suitable for human body monitoring in practical applications.
Smart Images

Figure CN114758147B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to a method and device for recognizing abnormal human postures, and a computer-readable storage medium. Background Art
[0002] As medical technology advances and the birth rate decreases year by year, the problem of population aging becomes more and more serious. The young and middle-aged people have a heavier responsibility to care for their elders. Therefore, it becomes more and more important to use monitoring equipment such as cameras to assist in caring for the elders.
[0003] Traditional intelligent monitoring detects abnormal postures by detecting the motion state of each key node of the human body. However, there are obvious visual differences between the key nodes of the two-dimensional human body and the key nodes of the three-dimensional human body, and different parts of the body will have a visual shortening effect. In addition, because the monitoring data transmitted through the network is too large, the key node detection is time-consuming and difficult to achieve real-time performance. The training process through the network is slow, which is not conducive to the training of monitoring data in practical applications.
[0004] Reducing the amount of monitoring data and thus reducing the amount of model calculations can solve the above problems, but the existing technology only focuses on how to improve the accuracy of the model and lacks relevant technologies for reducing the amount of calculations. Summary of the invention
[0005] In view of this, an object of the present invention is to provide a method, device and computer-readable storage medium for identifying abnormal human posture, which can reduce the amount of calculation for identifying abnormal human posture.
[0006] An embodiment of the present invention provides a method for recognizing abnormal posture of a human body, the method comprising: acquiring a real-time video frame image; extracting a human body edge contour from the real-time video frame image; finding the head position from the human body edge contour, finding the head feature points according to the symmetry of the human body and the head position, and tracking the head feature points to determine the head posture of the human body edge contour; selecting a head reference point according to the change state of the head posture; calculating the area according to the pixel points in the human body edge contour, respectively obtaining the head area and the upper body area, and calculating the ratio of the head area to the upper body area; calculating the horizontal cross-sectional area and the vertical cross-sectional area of the human body edge contour; determining whether the ratio of the head area to the upper body area falls within a preset ratio value interval, and when determining that the ratio of the head area to the upper body area falls within the preset ratio value interval, preliminarily determining that the human body posture is normal, and when determining that the ratio of the head area to the upper body area does not fall within the preset ratio value interval, preliminarily determining that the human body posture is abnormal; determining whether the horizontal cross-sectional area and the vertical cross-sectional area are equal to a first threshold of the horizontal cross-sectional area, a second threshold of the horizontal cross-sectional area, and a first threshold of the vertical cross-sectional area. A relationship between a threshold and a second threshold of a vertical cross-sectional area, wherein the second threshold of the horizontal cross-sectional area is greater than the first threshold of the horizontal cross-sectional area and the second threshold of the vertical cross-sectional area is greater than the first threshold of the vertical cross-sectional area. When it is judged that the horizontal cross-sectional area is less than or equal to the first threshold of the horizontal cross-sectional area and the vertical cross-sectional area is between the first threshold of the vertical cross-sectional area and the second threshold of the vertical cross-sectional area, it is preliminarily judged that the posture of the human body is normal. When it is judged that the horizontal cross-sectional area is between the first threshold of the horizontal cross-sectional area and the second threshold of the horizontal cross-sectional area and the vertical cross-sectional area is less than or equal to the first threshold of the vertical cross-sectional area, it is preliminarily judged that the posture of the human body is abnormal. When it is preliminarily judged that the posture of the human body is abnormal, the motion trajectory change of the head reference point is further obtained according to the adjacent previous frame of the video frame image, and the displacement distance value and the displacement time difference of the head reference point in the vertical direction are obtained according to the motion trajectory change; and whether an abnormal posture occurs is judged according to the displacement distance value and the displacement time difference. When the displacement distance value is greater than the preset displacement threshold and the displacement time difference is less than the preset time threshold, it is judged that the abnormal posture occurs and an alarm signal is issued.
[0007] An embodiment of the present invention further provides a device, which includes a memory and a processor, wherein the memory is used to store at least one instruction, and the processor is used to implement the method for identifying abnormal human posture when executing the at least one instruction.
[0008] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the method for identifying abnormal human posture is implemented.
[0009] Compared with the prior art, the method, device and computer-readable storage medium for identifying abnormal human posture provided by the present invention can reduce the amount of calculation for identifying abnormal human posture. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 FIG. 4 is a schematic diagram of the structure of a device according to an embodiment of the present invention.
[0011] Figure 2 Flow chart of a method for recognizing abnormal human posture according to an embodiment of the present invention.
[0012] Main component symbols
[0013] Device 100 processor 102 Memory 104 Camera 110 step S202~S220
[0014] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0015] In order to facilitate the understanding and implementation of the present invention by those skilled in the art of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the present invention provides many applicable inventive concepts, which can be implemented in a variety of specific forms. Those skilled in the art of the present invention can use the details described in these embodiments or other embodiments and other available structural, logical and electrical changes to implement the invention without departing from the spirit and scope of the present invention.
[0016] The present invention specification provides different embodiments to illustrate the technical features of different implementations of the present invention. Among them, the configuration of each component in the embodiment is for illustrative purposes and is not intended to limit the present invention. And the partial repetition of the figure numbers in the embodiment is to simplify the description and does not mean the correlation between different embodiments. Among them, the same component numbers used in the drawings and the description represent the same or similar components. The drawings in this specification are simplified and are not drawn to exact proportions.
[0017] Furthermore, in describing some embodiments of the present invention, the specification describes the method and (or) program of the present invention in a specific order of steps. However, since the method and program are not necessarily implemented according to the specific order of steps described, they are not limited to the specific order of steps described. A person skilled in the art of the present invention will know that other orders are also possible implementation methods. Therefore, the specific order of steps described in the specification is not used to limit the scope of the patent application. Furthermore, the scope of the patent application for the method and (or) program of the present invention is not limited to the order of execution steps written therein, and a person skilled in the art of the present invention will understand that adjusting the order of execution steps does not deviate from the spirit and scope of the present invention.
[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.
[0019] See also Figure 1 , which is a schematic diagram of the structure of a device 100 according to an embodiment of the present invention. Figure 1 As shown, the device 100 includes at least one processor 102 and a memory 104. Those skilled in the art should understand that Figure 1 The structure of the device 100 shown does not constitute a limitation of the embodiment of the present invention, and the device 100 may include less or more other hardware or software than shown, or different component settings. In this embodiment, the device may be a desktop, a mobile phone, a tablet computer, or a server. It should be noted that the device 100 is only an example, and other existing or future devices that may appear should also be included in the protection scope of the present invention if they can be adapted to the present invention.
[0020] In some embodiments, the device 100 includes a terminal that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices.
[0021] In one embodiment, the processor 102 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units, microprocessors, digital processing chips, graphics processors, and various control chips. The at least one processor 102 is the control core (Control Unit) of the device 100, and uses various interfaces and lines to connect the various components of the entire device 100, and executes or executes the programs or modules stored in the memory 104, and calls the data stored in the memory 104 to execute various functions of the device 100 and process data, such as the function of performing abnormal posture recognition of the human body.
[0022] In some embodiments, the memory 104 is used to store program code and various data, including storing program instructions for realizing abnormal posture recognition of human body. The memory 104 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable storage medium that can be used to carry or store data.
[0023] In one embodiment, the device 100 also establishes a communication connection with one or more cameras 110 via a video cable, a wired network or a wireless network to perform real-time transmission of video images captured by the camera 110. The camera 110 includes a hemispherical camera, a high-definition ball camera, a pan / tilt / zoom (PTZ) camera, and a combination of various cameras.
[0024] Although not shown, the device 100 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 102 through a power management device, so that the power management device can manage charging, discharging, and power consumption. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The device 100 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0025] See also Figure 2, which is a flow chart of a method for recognizing abnormal human postures in one embodiment of the present invention. In this embodiment, the method for recognizing abnormal human postures can be applied to the device 100. For the device 100 that needs to recognize abnormal human postures, the function for recognizing abnormal human postures provided by the method of the present invention can be directly integrated on the device 100, or run on the device 100 in the form of a software development kit (SDK). Figure 2 As shown, the method for identifying abnormal human posture specifically includes the following steps. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.
[0026] Step S202, obtaining real-time video frame images captured by the camera.
[0027] Step S204, extracting the edge contour of the human body from the real-time video frame image. In this embodiment, the real-time video frame image is subjected to a differential operation with an adjacent previous video frame image to obtain motion information, and then the information is used to extract a moving target from the background of the real-time video frame image, and the moving target is matched with a human body template to determine whether it is a human body. If the moving target is determined to be a human body, the edge contour of the human body is extracted from the real-time video frame image according to the motion information.
[0028] Step S206, selecting a head reference point according to the change of head movement. In this embodiment, according to the human figure features, the head position is found from the human body edge contour according to a certain ratio, and the head feature points are further selected according to the symmetry of the human body and the head position, and the head feature points are tracked to determine the head posture. Specifically, since the relative positions of the eyes, ears, and back of the head in the head are fixed, the eyes, ears, or the center of the back of the head can be used as head feature points, and the head feature points are tracked to determine the head posture of the human body edge contour. In one embodiment, the head posture includes facing the camera, facing the camera sideways, or facing the back of the camera. When the human head is facing the camera, the eyes are selected as the head reference points; when the head is facing the camera sideways, the ears are selected as the head reference points; when the head is facing the back of the camera, the center of the back of the head is selected as the head reference point. According to the change of head posture, different head reference points are selected to avoid the problem of reference point loss caused by the change of head posture.
[0029] Next, steps S208 to S210 and steps S212 to S214 may be executed simultaneously or one after another, and the execution order of step S208 and step S212 is not limited.
[0030] Step S208, calculate the area with the pixel points in the human body edge contour, and judge the change of the human body three-dimensional posture according to the head contour area and the upper body contour area. Specifically, the head contour area and the upper body contour area (including the head contour area) are calculated with pixel points from the head contour and the upper body contour in the human body edge contour. In normal human body posture, the head contour area accounts for a certain proportion of the upper body contour area, for example, the head contour area accounts for one third of the upper body contour area. In one embodiment, a ratio value, such as 0.33, can be preset, and a ratio value interval, such as 0.3 to 0.36, can be preset. In one embodiment, since the head and the upper body may face the camera, sideways to the camera or back to the camera, different area adjustment coefficients α can be set according to the head contour facing posture, sideways posture and back to the posture, where 0<α<1, and different area adjustment coefficients β can be set according to the upper body contour facing posture, sideways posture and back to the posture, where 0<β<1. According to the head posture, select the corresponding area adjustment coefficient, multiply the head contour area by the adjustment coefficient to obtain the adjusted head contour area. According to the upper body contour posture, a corresponding area adjustment coefficient is selected, and the upper body contour area is multiplied by the adjustment coefficient to obtain the adjusted upper body contour area.
[0031] Step S210, when the calculated ratio of the head contour area to the upper body contour area falls outside the preset ratio value interval, it is preliminarily judged that the human body posture is abnormal, and then step S216 is executed; when the calculated ratio of the head contour area to the upper body contour area falls within the preset ratio value interval, it is preliminarily judged that the human body posture is normal, and return to step S202.
[0032] Step S212, judging the posture of the human body according to the horizontal cross-sectional area and the vertical cross-sectional area within the edge contour of the human body. Specifically, different cross-sectional area thresholds can be set according to the monitored object, for example, different cross-sectional area thresholds can be set according to age, gender, and body shape. For example, when standing normally, bending over, or half squatting, the horizontal cross-sectional area should be less than or equal to the first horizontal cross-sectional area threshold X1, and the vertical cross-sectional area should be between the first vertical cross-sectional area threshold Y1 and the second vertical cross-sectional area threshold Y2, wherein the second vertical cross-sectional area threshold Y2 is greater than the first vertical cross-sectional area threshold Y1; when the posture is lying down, the horizontal cross-sectional area should be between the first horizontal cross-sectional area threshold X1 and the second horizontal cross-sectional area threshold X2, wherein the second horizontal cross-sectional area threshold X2 is greater than the first horizontal cross-sectional area threshold X1, and the vertical cross-sectional area should be less than or equal to the first vertical cross-sectional area threshold Y1.
[0033] Step S214, when it is judged that the horizontal cross-sectional area is between the first threshold value X1 of the horizontal cross-sectional area and the second threshold value X2 of the horizontal cross-sectional area, wherein the second threshold value X2 of the horizontal cross-sectional area is greater than the first threshold value X1 of the horizontal cross-sectional area, and the vertical cross-sectional area is less than or equal to the first threshold value Y1 of the vertical cross-sectional area, it is preliminarily judged that the human body posture is abnormal, and then step S216 is executed; when it is judged that the horizontal cross-sectional area is less than or equal to the first threshold value X1 of the horizontal cross-sectional area, and the vertical cross-sectional area is between the first threshold value Y1 of the vertical cross-sectional area and the second threshold value Y2 of the vertical cross-sectional area, wherein the second threshold value Y2 of the vertical cross-sectional area is greater than the first threshold value Y1 of the vertical cross-sectional area, it is preliminarily judged that the human body posture is normal, and the process returns to step S202.
[0034] Step S216, obtaining the change of the motion trajectory of the head reference point by comparing with the adjacent previous video frame image.
[0035] Step S218, according to the change of the motion trajectory, obtain the displacement distance value and displacement time difference of the head reference point in the vertical direction. When the displacement distance value is greater than the preset displacement threshold, further determine whether the displacement time difference is less than the preset time threshold. If the displacement time difference is less than the preset time threshold, it means that the time interval between the two frames before and after the obvious posture change is very short, it can be determined that an abnormal posture has occurred, and execute step S220; if the displacement time difference is greater than or equal to the predetermined threshold, it means that the time interval between the two frames before and after the obvious posture change is within the normal posture change time range, return to step S202, and continue to obtain the next frame of video frame image. Specifically, different default displacement thresholds can be set according to the monitored object, for example, different displacement thresholds can be set according to age, gender, and body shape.
[0036] S220, determining that an abnormal posture occurs and sending an alarm signal. In this embodiment, the abnormal posture is falling.
[0037] In one embodiment, multiple cameras 110 are arranged in a monitoring area, each camera 110 has a separate field of view, and the field of view of two cameras 110 may or may not overlap, and the multiple cameras 110 cooperate to identify abnormal postures of the human body in the monitoring area. In this embodiment, the device 100 can predict the movement direction of the human body based on the change of the movement trajectory of the head reference point, and further preferentially select the closer camera 110 to perform the detection based on the movement direction. Figure 2 In another embodiment, the device 100 can also track the reference point of the human head to determine the camera 110 corresponding to the location of the monitored object in real time, and give priority to switching to the camera 110 to execute Figure 2 process steps.
[0038] In summary, the human body abnormal posture recognition method, device and computer-readable storage medium of the present invention can select different reference points of the head according to the change of head movement, solve the problem of misjudgment caused by the loss of reference points due to human body movement, and reduce and simplify the calculation process. The method of calculating the area by pixel points is combined with the time difference of the contour area change, the horizontal and vertical cross-sectional area change and the head reference point motion trajectory change to perform abnormal posture recognition, and the sudden change of posture change can be accurately judged.
[0039] It is worth noting that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A method for identifying abnormal human posture, characterized in that: The method comprises: Get real-time video frame images; Extracting the human body edge contour from the real-time video frame image; Find the head position from the human body edge contour, find the head feature points according to the human body symmetry and the head position, and track the head feature points to determine the head posture of the human body edge contour; Selecting a head reference point according to the head posture change state; Calculate the area according to the pixel points within the human body edge contour, obtain the head area and the upper body area respectively, and calculate the ratio of the head area to the upper body area; Calculate the horizontal cross-sectional area and the vertical cross-sectional area of the human body edge contour; Determine whether the ratio of the head area to the upper body area falls within a preset ratio value interval. When it is determined that the ratio of the head area to the upper body area falls within the preset ratio value interval, preliminarily determine that the human body posture is normal; when it is determined that the ratio of the head area to the upper body area does not fall within the preset ratio value interval, preliminarily determine that the human body posture is abnormal; Determine the relationship between the horizontal cross-sectional area and the vertical cross-sectional area and a first threshold value of the horizontal cross-sectional area, a second threshold value of the horizontal cross-sectional area, a first threshold value of the vertical cross-sectional area, and a second threshold value of the vertical cross-sectional area, wherein the second threshold value of the horizontal cross-sectional area is greater than the first threshold value of the horizontal cross-sectional area and the second threshold value of the vertical cross-sectional area is greater than the first threshold value of the vertical cross-sectional area, and when it is determined that the horizontal cross-sectional area is less than or equal to the first threshold value of the horizontal cross-sectional area and the vertical cross-sectional area is between the first threshold value of the vertical cross-sectional area and the second threshold value of the vertical cross-sectional area, preliminarily determine that the posture of the human body is normal, and when it is determined that the horizontal cross-sectional area is between the first threshold value of the horizontal cross-sectional area and the second threshold value of the horizontal cross-sectional area and the vertical cross-sectional area is less than or equal to the first threshold value of the vertical cross-sectional area, preliminarily determine that the posture of the human body is abnormal; When it is preliminarily determined that the human body posture is abnormal, further obtaining the motion trajectory change of the head reference point according to the adjacent previous frame of video frame image, and obtaining the displacement distance value and displacement time difference of the head reference point in the vertical direction according to the motion trajectory change; and Whether an abnormal posture occurs is determined based on the displacement distance value and the displacement time difference. When the displacement distance value is greater than a preset displacement threshold and the displacement time difference is less than a preset time threshold, it is determined that an abnormal posture occurs and an alarm signal is issued.
2. The method for recognizing abnormal human posture according to claim 1, characterized in that: The step of extracting the human body edge contour from the real-time video frame image also includes: Performing a differential operation on the real-time video frame image and the adjacent previous video frame image to obtain motion information; Extracting a moving target from the background of the real-time video frame image according to the motion information; The moving target is matched with a human body template to determine whether the moving target is a human body. When the moving target is determined to be a human body, the human body edge contour is extracted from the real-time video frame image according to the motion information.
3. The method for recognizing abnormal human posture according to claim 1, characterized in that: The head posture includes facing the camera, facing the side of the camera, and facing away from the camera.
4. The method for recognizing abnormal human posture as claimed in claim 3, characterized in that: The step of selecting a head reference point according to the head posture change state also includes: When the head posture is facing the camera, the eyes are selected as the head reference point; When the head posture is sideways to the camera, the ear is selected as the head reference point; and, When the head posture is facing away from the camera, the center of the back of the head is selected as the head reference point.
5. The method for recognizing abnormal human posture according to claim 1, characterized in that: The first threshold of the horizontal cross-sectional area, the second threshold of the horizontal cross-sectional area, the first threshold of the vertical cross-sectional area, and the second threshold of the vertical cross-sectional area are set according to the age, gender, and body shape of the monitored object.
6. The method for recognizing abnormal human posture according to claim 1, characterized in that: The preset displacement threshold is set according to the age, gender and body shape of the monitored object.
7. The method for recognizing abnormal human posture according to claim 1, characterized in that: The method further comprises: Tracking the head reference point to determine the camera corresponding to the location of the monitored object in real time; Priority is given to switching to the corresponding camera to obtain real-time video frame images.
8. A device, characterized in that: The device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for recognizing abnormal human posture as described in any one of claims 1 to 7 are implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for recognizing abnormal human posture as described in any one of claims 1 to 7 are implemented.
