Thermal imaging-based monitoring system for detecting abnormal behavior of elderly people

Abstract

The application relates to the technical field of intelligent monitoring, in particular to an old person behavior abnormality monitoring system based on thermal imaging, which comprises a processor and a memory, the processor executes a computer program stored in the memory to realize the following steps: according to the long axis included angle, the barycenter and the axis ratio of the minimum circumscribed ellipse of a target region on a to-be-judged image, the included angle change representation value of a local window corresponding to a first suspected falling image, the barycenter offset degree corresponding to the to-be-judged image and the ellipse deformation degree corresponding to the to-be-judged image are obtained, and the falling behavior of a monitored object is monitored according to the included angle change representation value, the barycenter offset degree and the ellipse deformation degree. Moreover, the application can improve the accuracy of monitoring the falling behavior of the monitored object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent monitoring, in particular to an old person behavior abnormality monitoring system based on thermal imaging. BACKGROUND

[0002] With the aggravation of global aging trend, the abnormal behavior monitoring of the old people gradually becomes an important research direction in the health care field, especially the falling behavior of the old people, that is, falling is one of the main factors endangering the health of the old people, because the old people are prone to hip fracture, intracranial hemorrhage, myocardial infarction and other complications due to osteoporosis and weak organ compensatory capacity, and these symptoms cannot be treated in time and will lead to irreversible serious consequences such as death and paralysis, so it is crucial to identify the falling behavior of the old people in time, and the monitoring of the behavior of the old people is mainly applied to smart home, nursing home and medical institutions.

[0003] In the prior art, an infrared thermal imaging camera is usually used to obtain a single frame monitoring image of a monitored object, and then the collected single frame monitoring image is input into a classification model, and the model outputs the behavior type of the monitored object in the single frame monitoring image, but when the falling behavior of the old people is monitored based on single frame image and classification model, the behavior dynamic factor is not considered, so the phenomenon of false detection or misidentification may occur, for example, if the behavior of the old people in a frame image is falling, then the behavior of the old people in the image may be identified as normal behavior such as quickly bending down to pick up an object, slowly sitting down or slowly getting up when the existing method is used for identification, that is, the existing monitoring method of the falling behavior of the old people is prone to false detection, so when the falling behavior of the old people is monitored or identified, how to improve the accuracy of monitoring or identification is a problem to be solved. SUMMARY

[0004] In order to solve the above problems, the present application provides an old person behavior abnormality monitoring system based on thermal imaging, and the technical solution adopted is as follows:

[0005] An embodiment of the present application provides an old person behavior abnormality monitoring system based on thermal imaging, comprising a processor and a memory, and the processor executes a computer program stored in the memory to realize the following steps:

[0006] An infrared thermal imaging camera is used to obtain a to-be-analyzed monitoring video of a monitored object, the to-be-analyzed monitoring video is composed of to-be-analyzed images, a target region on the to-be-analyzed images and a first suspected falling image in the to-be-analyzed monitoring video are obtained, and the target region on the to-be-analyzed images is a bounding box corresponding to the monitored object in the to-be-analyzed image;

[0007] Obtain a local window corresponding to the first suspected falling image, and record all images in the local window as to-be-judged images, and obtain an angle change characteristic value of the local window corresponding to the first suspected falling image, a gravity center offset degree corresponding to the to-be-judged image, and an ellipse deformation degree corresponding to the to-be-judged image according to a long axis included angle of a minimum circumscribed ellipse of a target region on the to-be-judged image, a gravity center, and an axis ratio, the long axis included angle being an included angle of a long axis of the corresponding minimum circumscribed ellipse and a horizontal ground.

[0008] Monitor a falling behavior of the monitored object according to the angle change characteristic value, the gravity center offset degree, and the ellipse deformation degree.

[0009] Beneficial effects: The application first acquires a to-be-analyzed monitoring video of a monitored object by using an infrared thermal imaging camera, and acquires a target region on the to-be-analyzed image and a first suspected falling image in the to-be-analyzed monitoring video; then obtains a local window corresponding to the first suspected falling image, and records all images in the local window as to-be-judged images, and obtains an angle change characteristic value of the local window corresponding to the first suspected falling image, a gravity center offset degree corresponding to the to-be-judged image, and an ellipse deformation degree corresponding to the to-be-judged image according to a long axis included angle of a minimum circumscribed ellipse of a target region on the to-be-judged image, a gravity center, and an axis ratio; finally, a falling behavior of the monitored object is monitored according to the angle change characteristic value, the gravity center offset degree, and the ellipse deformation degree. Moreover, the application can improve the accuracy of monitoring the falling behavior of the monitored object according to the angle change characteristic value, the gravity center offset degree, and the ellipse deformation degree. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, below, a brief introduction will be given to the drawings needed in the embodiments or the prior art description. Obviously, the drawings in the following description are only some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0011] Figure 1 A flowchart of the method for monitoring abnormal behaviors of the elderly based on thermal imaging. DETAILED DESCRIPTION

[0012] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, and not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art also belong to the scope of protection of the embodiments of the present application.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0014] This embodiment provides a thermal imaging-based system for monitoring abnormal behavior in the elderly, including a processor and a memory. The processor executes a computer program stored in the memory to implement a thermal imaging-based method for monitoring abnormal behavior in the elderly. Figure 1 As shown, this method for monitoring behavioral abnormalities in the elderly based on thermal imaging includes the following steps:

[0015] Step S001: Use an infrared thermal imaging camera to acquire a monitoring video of the monitored object to be analyzed. The monitoring video to be analyzed consists of images to be analyzed. Acquire the target area on the images to be analyzed and the first suspected fall image in the monitoring video to be analyzed.

[0016] Because normal behaviors such as quickly bending over to pick up items, slowly sitting down, or slowly standing up can easily lead to misidentification or false detection when identifying or detecting falls in the elderly, this can result in the failure to detect falls in a timely manner. Failure to detect falls in a timely manner can lead to irreversible and serious consequences, such as death or paralysis. Therefore, the main purpose of this embodiment is to improve the accuracy of identifying or monitoring falls in the elderly, thereby enabling timely treatment when a fall occurs. Furthermore, while current methods for monitoring the behavior of the elderly are applied in multiple scenarios, such as smart homes, nursing homes, and medical institutions, this embodiment, for ease of understanding, selects only one scenario for monitoring. For example, this embodiment mainly focuses on monitoring the fall behavior of elderly people living at home, and this embodiment monitors any elderly person living at home; that is, the monitored subject described in the following embodiment is the same elderly person.

[0017] In this embodiment, when monitoring fall behavior of elderly people living at home, infrared thermal imaging cameras need to be deployed in the elderly person's residence. Since infrared thermal imaging cameras are not affected by lighting conditions and can work stably at night or in low-light environments, while effectively protecting user privacy and preventing the leakage of sensitive information such as facial recognition, infrared thermal imaging cameras have become an important device for monitoring the behavior of the elderly. The placement and number of infrared thermal imaging cameras need to be determined by relevant personnel based on the size and structure of the residence, but it is required that the elderly person can be detected by the infrared thermal imaging cameras whenever they move in any location in their residence. In addition, for ease of understanding, this embodiment will use the video captured by any infrared thermal imaging camera to determine whether the monitored elderly person has fallen. That is, all videos or images that appear in this embodiment later are collected by the same infrared thermal imaging camera.

[0018] Next, the video captured by the infrared thermal imaging camera during the current monitoring period is acquired and recorded as the initial video. The initial video consists of multiple frames, and all images in the initial video are recorded as initial images. Then, all initial images are preprocessed, and the preprocessed images are recorded as the images to be screened corresponding to the initial images. Then, the monitored objects in the images to be screened are identified using a target detection algorithm, and the bounding boxes of the monitored objects are obtained on the images to be screened. The bounding box regions of the monitored objects on the images to be screened are recorded as the target regions on the corresponding images to be screened. The monitored objects are elderly people. Since the initial video is only a video captured by one camera, there may be cases where the images to be screened do not contain the monitored objects. That is, the monitored objects do not always move in the same position, and images without monitored objects do not have analytical value. Therefore, in this embodiment, the initial video that does not contain the target regions is removed. The video composed of the remaining images to be screened after removal is recorded as the monitoring video to be analyzed for the monitored objects, and all images in the monitoring video to be analyzed are recorded as images to be analyzed. That is, the monitoring video to be analyzed is composed of images to be analyzed.

[0019] In this embodiment, the implementer needs to set the camera's frame rate and the duration of the current monitoring period according to the actual situation. For example, in this embodiment, the frame rate can be set to 25 FPS and the duration of the current monitoring period can be set to 1 minute. The preprocessing in this embodiment involves image enhancement, Gaussian filtering smoothing, histogram equalization, etc. In addition, in this embodiment, the implementer also needs to select the type of target detection algorithm according to the actual situation. For example, in this embodiment, the YOLO target detection algorithm can be used to obtain the bounding box of the monitored object in the image. The process of using the target detection algorithm to obtain the bounding box of the monitored object in the image is a well-known technology, so it will not be described in detail in this embodiment.

[0020] Since the image to be analyzed is a filterable image, and as described above, the monitoring video to be analyzed consists of filterable images containing target regions, and the target region is the bounding box of the monitored object, it can be seen from the above process that the target region on the image to be analyzed has been obtained.

[0021] After obtaining the monitoring video to be analyzed and the target areas on each image to be analyzed within the monitoring video, the first suspected fall image in the monitoring video to be analyzed is obtained, and the specific acquisition process is as follows:

[0022] First, a trained classification model is obtained. Then, the target regions on the images to be analyzed are input into the trained classification model. The model outputs the behavior type corresponding to the target region on each image to be analyzed, and the behavior type corresponding to the target region on the image to be analyzed is the behavior type of the monitored object in the target region on the corresponding image to be analyzed. Then, based on the behavior type corresponding to the target region on the image to be analyzed, images to be analyzed that belong to the behavior type of falling are obtained from the monitoring video to be analyzed, and all images to be analyzed that belong to the behavior type of falling are recorded as the first suspected falling images.

[0023] In this embodiment, the classification model is mainly used to identify the behavior type of the monitored object in the image. The behavior types in this embodiment mainly include two categories: normal behavior and falling behavior. Normal behavior includes, but is not limited to, bending over, standing up, and sitting down, while falling behavior includes, but is not limited to, falling forward, falling backward, and falling sideways. In specific applications, the implementer needs to select the type of classification model according to the actual situation. For example, in this embodiment, the MobileNet-FallNet model can be selected for behavior type identification. MobileNet-FallNet is an improvement based on MobileNet, a lightweight deep neural network designed for mobile and embedded devices. The MobileNet-FallNet model mainly uses the MobileNet network for feature extraction first. That is, the input image is converted into a compact feature vector by the MobileNet network. The feature vector contains key information such as human contour and posture, providing a foundation for subsequent classification tasks. Then, the features extracted by MobileNet are input into a fully connected layer, and the probability distribution of different behavior types is calculated through the Softmax function. Based on the probability magnitude, the final output is the prediction result of the behavior category. Furthermore, since the model training process is a well-known technology, it will not be described in detail in this embodiment.

[0024] Therefore, this embodiment can obtain the monitoring video to be analyzed, the target area on each image to be analyzed in the monitoring video to be analyzed, and the first suspected fall image in the monitoring video to be analyzed through the above process. The first suspected fall image may contain images that do not belong to the fall behavior type. That is, the first suspected fall image may contain images whose behavior type belongs to normal behavior, such as bending over or sitting down. Therefore, further analysis and screening are required.

[0025] Step S002: Obtain the local window corresponding to the first suspected fall image, and record all images in the local window as images to be judged. Based on the major axis angle, centroid and axis ratio of the minimum circumscribed ellipse of the target region on the image to be judged, obtain the angle change characterization value of the local window corresponding to the first suspected fall image, the centroid offset of the image to be judged, and the degree of ellipse deformation of the image to be judged.

[0026] Based on the above description, it can be seen that the first suspected fall image obtained above may contain images of normal behavior. This phenomenon is because the input for behavior type identification is a single-frame image, which only considers the static information of the monitored object. However, in order to improve the accuracy of behavior monitoring, this embodiment will combine the dynamic information of the monitored object for identification. That is, this embodiment will combine the features of the monitored object in different behavioral processes to detect fall behavior. Next, this embodiment will obtain the local window corresponding to the first suspected fall image. The purpose of obtaining the local window is to combine the dynamic information of the monitored object. In this embodiment, the specific process of obtaining the local window corresponding to the first suspected fall image is as follows:

[0027] For any first suspected fall image: In the monitoring video to be analyzed, a preset number of images to be analyzed are acquired whose acquisition time is earlier than the first suspected fall image. The window formed by arranging the first suspected fall image and the preset number of images to be analyzed whose acquisition time is earlier than the first suspected fall image in chronological order of acquisition time is recorded as the local window corresponding to the first suspected fall image. In specific applications, the implementer needs to set the value of the preset number according to the actual situation. In this embodiment, the preset number is required to be related to the acquisition frame rate. That is, the more images acquired per second, the larger the value of the preset number. For example, if the acquisition frame rate is 30fps, then the value of the preset number is set to 30. If the acquisition frame rate is 60fps, then the value of the preset number is set to 60.

[0028] In addition, when the number of images to be analyzed in front of the first suspected fall image is less than a preset number, all the images to be analyzed in the monitoring video to be analyzed that are in front of the first suspected fall image, and the window formed by the images to be analyzed, are used as the local window corresponding to the first suspected fall image.

[0029] Therefore, this embodiment can obtain the local window corresponding to each first suspected fall image through the above process. In this embodiment, all images in each local window are recorded as images to be judged. After obtaining the local window corresponding to each first suspected fall image, the minimum bounding ellipse of the target region on each image to be judged in the local window is obtained. The process of obtaining the minimum bounding ellipse of the target region is a known technique. Then, the major axis angle, centroid, and axis ratio of the minimum bounding ellipse of the target region on each image to be judged are obtained. The major axis angle of the minimum bounding ellipse is the angle between the major axis of the minimum bounding ellipse and the horizontal ground. The centroid of the minimum bounding ellipse is the intersection of the major axis and the minor axis of the minimum bounding ellipse. The axis ratio of the minimum bounding ellipse is the normalized value of the ratio of the major axis and the minor axis of the minimum bounding ellipse. For example, if the coordinates of the two intersection points of the major axis of any minimum bounding ellipse and the ellipse are respectively... and If the major axis of the smallest circumscribed ellipse is a1 and the minor axis is a2, then the included angle between the major axes of the smallest circumscribed ellipse is . The included angle of the major axis of the smallest circumscribed ellipse is and the axial ratio is . Norm() is the normalization function, and arctan() is the arctangent function. The value range is from 0 to 1; in addition, in this embodiment, the coordinate values ​​of the points refer to the coordinate values ​​of the corresponding points in the plane coordinate system, and the horizontal axis of the plane coordinate system is the horizontal ground and the vertical axis is perpendicular to the horizontal ground.

[0030] Then, based on the major axis angle, centroid, and axial ratio of the minimum circumscribed ellipse of the target region in the local window corresponding to each first suspected fall image, the angle change characterization value of the local window corresponding to each first suspected fall image, the centroid shift degree of each target image in the local window, and the ellipse deformation degree of each target image in the local window are obtained. In this embodiment, the angle change characterization value, centroid shift degree, and ellipse deformation degree are obtained based on the major axis angle, centroid, and axial ratio of the minimum circumscribed ellipse of the target region because the major axis angle, centroid, and axial ratio of the minimum circumscribed ellipse of the target region will change differently when the monitored object is in different behavioral states. Therefore, the subsequent analysis of the obtained angle change characterization value, centroid shift degree, and ellipse deformation degree can further improve the recognition of fall behavior. For example, when at rest... In a standing position, the major axis of the smallest circumscribed ellipse of the monitored object's outline is nearly perpendicular to the ground, the aspect ratio is close to 1, and the center of gravity hardly moves. The aspect ratio is close to 1 because the monitored object's outline usually has a large length-to-width ratio. During a fall, the angle of the major axis of the smallest circumscribed ellipse of the monitored object's outline will shift significantly and gradually tend towards horizontal. The center of gravity of the smallest circumscribed ellipse of the monitored object's outline will show a phenomenon of slow movement followed by accelerated movement. That is, the skewness of the center of gravity of the smallest circumscribed ellipse of the monitored object's outline first increases slowly and then increases sharply. At the time of the fall, the aspect ratio of the smallest circumscribed ellipse of the monitored object's outline is close to 0. During a slow sitting down or bending over to stand up, the angle of the major axis of the smallest circumscribed ellipse of the monitored object's outline changes within a small range, and the angle of the major axis is close to 45 degrees. The aspect ratio of the smallest circumscribed ellipse of the monitored object's outline also changes little, and the aspect ratio is close to 0.5.

[0031] In this embodiment, the angle change characterization value of the local window corresponding to each first suspected fall image will be obtained based on the major axis angle of the minimum circumscribed ellipse of the target region in each image to be judged within the local window corresponding to each first suspected fall image. The specific process for obtaining the angle change characterization value of the local window corresponding to each first suspected fall image is as follows:

[0032] For any first suspected fall image W: First, obtain the target angle sequence of the local window corresponding to the first suspected fall image W. The t-th target angle in the target angle sequence is the absolute value of the difference between the major axis angle of the (t+1)-th image to be judged and the major axis angle of the t-th image to be judged in the local window corresponding to the first suspected fall image W. The major axis angle of the (t+1)-th image to be judged is the major axis angle of the minimum circumscribed ellipse of the target region on the (t+1)-th image to be judged in the local window corresponding to the first suspected fall image. The absolute value of the difference between the major axis angles of the t-th images to be judged is the minimum circumscribed ellipse angle of the target region on the t-th image to be judged in the local window corresponding to the first suspected fall image. The major axis of the minor circumscribed ellipse is included; then the cosine value of each target angle in the target angle sequence is obtained, and the sequence formed by the cosine values ​​of all target angles in the target angle sequence is recorded as the cosine value sequence of the local window corresponding to the first suspected fall image, that is, the t-th cosine value in the cosine value sequence of the local window corresponding to the first suspected fall image is the cosine value of the t-th target angle in the target angle sequence; then the mean of the cosine value sequence of the local window corresponding to the first suspected fall image W is obtained, and it is used as the angle change characterization value of the local window corresponding to the first suspected fall image W; and in this embodiment, the specific calculation process of the angle change characterization value of the local window corresponding to the first suspected fall image W is as follows:

[0033]

[0034] in, Let be the angle change representation value of the local window corresponding to the first suspected fall image W, T be the number of target angles in the target angle sequence of the local window corresponding to the first suspected fall image W, and cos() be the cosine function. Let be the value of the t-th target angle in the target angle sequence; and when The larger the value, the more likely it is to indicate that... The larger, and The larger the value, the greater the rate of change of the major axis angle within the local window corresponding to the first suspected fall image W. A larger rate of change of the major axis angle indicates a higher probability that the behavior type of the detected object in the first suspected fall image W belongs to the fall behavior type. Conversely, a smaller rate of change of the major axis angle indicates a higher probability that the behavior type of the detected object in the first suspected fall image W belongs to the fall behavior type. The smaller the value, the lower the probability that the behavior of the detected object in the first suspected fall image W belongs to the fall behavior type.

[0035] Therefore, this embodiment can obtain the angle change characterization value of the local window corresponding to each first suspected fall image through the above process. After obtaining the angle change characterization value, the centroid offset of each image to be judged in the local window corresponding to each first suspected fall image is obtained based on the centroid of the minimum circumscribed ellipse of the target region on each image to be judged in the local window corresponding to each first suspected fall image. The specific process for obtaining the centroid offset of each image to be judged is as follows:

[0036] For any first suspected fall image W, the b-th image to be judged within the local window B:

[0037] First, in local window B, acquire the image to be judged that is to the left and adjacent to the b-th image to be judged, and record it as the left adjacent image corresponding to the b-th image to be judged. Acquire the image to be judged that is to the right and adjacent to the b-th image to be judged, and record it as the right adjacent image of the b-th image to be judged. Then, record the minimum bounding ellipse of the target region on the b-th image to be judged as the ellipse to be analyzed. Record the minimum bounding ellipse of the target region on the left adjacent image of the b-th image to be judged as the left ellipse, and record the minimum bounding ellipse of the target region on the right adjacent image of the b-th image to be judged as the right ellipse. Then, acquire the distance between the centroid of the ellipse to be analyzed and the centroid of the left ellipse, and record it as the first distance. Acquire the distance between the centroid of the ellipse to be analyzed and the centroid of the right ellipse, and record it as the second distance. The method for calculating the distance between the two centroids is the same as the method for calculating the distance between two points. The coordinates of the centroids refer to the coordinates in the plane coordinate system. Next, determine whether the value of b is 1. If it is, then... If the value of b is M, then the centroid offset of the b-th image to be judged is used; otherwise, it is determined whether the value of b is M. If it is M, then... If the value of b is not equal to 1 and M, then the centroid offset of the b-th image to be judged is used. The centroid offset is used for the b-th image to be judged, where D2 is the second distance, D1 is the first distance, c is a preset first constant, Norm() is the normalization function, and the preset first constant is to prevent the denominator from being zero. M is the total number of images in the local window B. Furthermore, this embodiment mainly analyzes the changes in the centroid using the centroid offset, which reflects the probability that the behavior type of the detected object in the first suspected fall image belongs to the fall behavior type.

[0038] Furthermore, in specific applications, the implementer needs to set the value of the preset first constant according to the actual situation. For example, in this embodiment, the preset first constant can be set to... .

[0039] Therefore, this embodiment can obtain the centroid offset of the image to be judged through the above process. Next, this embodiment will obtain the degree of ellipse deformation of each image to be judged in the local window corresponding to each first suspected fall image based on the axis ratio of the minimum circumscribed ellipse of the target region in each image to be judged in the local window corresponding to each first suspected fall image. That is, the specific process of obtaining the degree of ellipse deformation of the image to be judged is as follows:

[0040] For any first suspected fall image W, the b-th image to be judged within the local window B:

[0041] First, the aspect ratios of the minimum bounding ellipse of the target region in the b-th image to be judged, the minimum bounding ellipse of the target region in the left adjacent image of the b-th image to be judged, and the minimum bounding ellipse of the target region in the right adjacent image of the b-th image to be judged are respectively denoted as the first aspect ratio, the second aspect ratio, and the third aspect ratio. Then, the difference between the third aspect ratio and the first aspect ratio is obtained and denoted as the first aspect ratio difference, and the difference between the first aspect ratio and the second aspect ratio is obtained and denoted as the second aspect ratio difference. The ratio of the first aspect ratio difference to the first aspect ratio is denoted as the first feature ratio, and the ratio of the second aspect ratio difference to the second aspect ratio is denoted as the second feature ratio. Next, it is determined whether the value of b is 1. If it is, then... If the value of b is M, then the degree of ellipse deformation corresponding to the b-th image to be judged is determined. Otherwise, the value of b is determined. If it is M, then the value of b is determined. If the degree of ellipse deformation corresponds to the b-th image to be judged, then check if the value of b is not equal to 1 and M. If it is, then... The degree of ellipse deformation corresponding to the b-th image to be judged is represented by R3, where R3 is the third axis ratio, R2 is the second axis ratio, and R1 is the first axis ratio. The first characteristic ratio, This is the second feature ratio. Furthermore, in this embodiment, the deformation of the ellipse within the local window is primarily analyzed based on the degree of ellipse deformation. The deformation of the ellipse within the local window reflects the probability that the behavior type of the detected object in the first suspected fall image belongs to the fall behavior type.

[0042] Therefore, this embodiment can obtain the degree of ellipse deformation corresponding to the image to be judged through the above process; in addition, when falling, the change in axis ratio is more drastic, so the degree of ellipse deformation will be greater. When sitting or bending over, the change in axis ratio is more stable, so the degree of ellipse deformation will be smaller.

[0043] Step S003: Monitor the falling behavior of the monitored object based on the angle change characterization value, the center of gravity offset, and the degree of ellipse deformation.

[0044] Next, this embodiment will monitor the fall behavior of the monitored object within the current monitoring time period based on the obtained angle change characterization value, center of gravity shift, and ellipse deformation degree. The specific process of monitoring the fall behavior of the monitored object within the current monitoring time period based on the obtained angle change characterization value, center of gravity shift, and ellipse deformation degree is as follows:

[0045] First, based on the angle change values ​​of the local windows corresponding to each first suspected fall image, the centroid offset of each image to be judged within the local window, and the degree of elliptical deformation of each image to be judged within the local window, initial fall judgment index values ​​for each first suspected fall image are obtained. Then, based on the obtained initial fall judgment index values ​​for each first suspected fall image, the first suspected fall images are filtered, and second suspected fall images and their target fall judgment index values ​​are obtained based on the filtering results. Next, it is determined whether the target fall judgment index values ​​of all second suspected fall images are not greater than a preset second judgment threshold. If... If yes, then the behavior type of the monitored object within the current monitoring period is determined to be normal, and there is no need to alert the family. However, if the target fall judgment index value of all second suspected fall images is not less than or equal to the preset second judgment threshold, it indicates that the monitored object is in a fall behavior type within the current monitoring period, and an alert should be issued to the family so that the monitored object can receive timely treatment. That is, if the target fall judgment index value of the second suspected fall image is greater than the preset second judgment threshold, then the behavior type of the monitored object in the corresponding second suspected fall image is determined to be a fall behavior, and an alert should be issued to the family.

[0046] In addition, in specific applications, implementers need to set a preset first judgment threshold and a preset second judgment threshold according to the actual situation. For example, in this embodiment, both the preset first judgment threshold and the preset second judgment threshold can be set to 0.6.

[0047] In this embodiment, the specific process for obtaining the initial fall determination index value of each first suspected fall image is as follows:

[0048] For any first suspected fall image W:

[0049] First, the local window corresponding to the first suspected fall image W is denoted as local window B. The average value of the centroid offset corresponding to all images to be judged in local window B is denoted as the average centroid offset. The offset difference feature value corresponding to each image to be judged in local window B is obtained, and the average value of the offset difference feature values ​​corresponding to all images to be judged in local window B is denoted as the centroid offset difference characterization value of the local window corresponding to the first suspected fall image W. The offset difference feature value corresponding to the b-th image to be judged in local window B is the normalized value of the absolute value of the difference between the weighted centroid offset and the average centroid offset of the b-th image to be judged. The weighted centroid offset of the b-th image to be judged is the result of multiplying the centroid offset of the b-th image to be judged by the reference weight of the b-th image to be judged. The reference weight of the b-th image to be judged is... c2 is a preset second constant, exp() is an exponential function with base e, and if b is 1, then the reference weight of the b-th image to be judged is... The purpose of setting reference weights is to make the reference value of images acquired later greater.

[0050] Next, the normalized value of the degree of elliptical deformation corresponding to each image to be judged in local window B is obtained and recorded as the normalized elliptical deformation degree corresponding to the image to be judged. Then, the mean of the normalized elliptical deformation degree corresponding to all images to be judged in local window B is recorded as the elliptical deformation characterization value of the local window corresponding to the first suspected fall image W. Finally, the angle change characterization value, centroid shift difference characterization value, and elliptical deformation characterization value of the local window corresponding to the first suspected fall image W are weighted and summed, and the result of the weighted sum is used as the initial fall judgment index value of the first suspected fall image W. The specific calculation expression of the initial fall judgment index value of the first suspected fall image is as follows:

[0051]

[0052] in, Let W be the initial fall determination index value for the first suspected fall image. As the first weight value, This is the second weight value. The third weight value is M, where M is the total number of images in the local window B. Let be the weighted center offset of the b-th image to be judged in local window B. The average offset of the center of gravity. Let b be the degree of ellipse deformation corresponding to the b-th image to be judged in local window B. This is to normalize the degree of ellipse deformation corresponding to the b-th image to be judged.

[0053] In addition, when The larger, The larger and The larger the value of S, the greater the probability that the behavior of the monitored object in the first suspected fall image W belongs to the fall behavior. In specific applications, the implementer needs to set the first weight value, the second weight value, the third weight value, and the preset second threshold value according to the actual situation. For example, in this embodiment, the first weight value, the second weight value, and the third weight value can be set to 0.4, 0.3, and 0.3 respectively, and the preset second threshold can be set to the variance of the centroid offset value corresponding to all images in the local window B.

[0054] In this embodiment, the method for filtering first suspected fall images based on initial fall determination index values ​​and obtaining target fall determination index values ​​for second suspected fall images based on the filtering results includes:

[0055] First, a suspected fall image with an initial fall judgment index value greater than a preset first judgment threshold is obtained and recorded as a second suspected fall image. The behavior type of the monitored object in the first suspected fall image with an initial fall judgment index value not greater than the preset first judgment threshold is considered normal behavior. However, the behavior type of the monitored object in the first suspected fall image with an initial fall judgment index value greater than the preset first judgment threshold may be either bending over or falling. Therefore, after obtaining the second suspected fall image, it is necessary to obtain the target fall judgment index value corresponding to the second suspected fall image based on the histograms of each image in the corresponding local window. The specific process for obtaining the target fall judgment index value corresponding to the second suspected fall image is as follows:

[0056] In the monitoring video to be analyzed, any image of the behavior type belonging to bending over is recorded as a reference image, and a reference local window corresponding to the reference image is obtained. All images to be analyzed in the reference local window are recorded as feature images. The method of obtaining the reference local window is the same as the method of obtaining the local window corresponding to the first suspected fall image.

[0057] First, obtain the LBP histogram of the minimum bounding ellipse of the target region on the feature image, and record it as the local LBP histogram corresponding to the feature image. The horizontal axis of the LBP histogram is the LBP value, and the vertical axis is the frequency of the corresponding LBP value. The average local LBP histogram of all feature images in the reference local window is recorded as the reference local LBP histogram.

[0058] Next, for any second suspected fall image: the local window corresponding to the second suspected fall image is recorded as the local window to be analyzed, and all images in the local window to be analyzed are recorded as the first image. The LBP histogram of the minimum bounding ellipse of the target region on the first image is obtained and recorded as the local LBP histogram of the corresponding first image. The chi-square distance between the local LBP histogram of the first image and the reference local LBP histogram is obtained and recorded as the feature distance representation value of the first image. The normalized value of the ratio of the feature distance representation value of the first image to the number of LBP value categories on the LBP histogram is recorded as the behavior representation value of the corresponding first image. The mean of the behavior representation values ​​of all first images in the local window to be analyzed is used as the target fall judgment index value corresponding to the second suspected fall image. The chi-square distance between the local LBP histogram of the first image and the reference local LBP histogram is... Where G is the number of LBP value categories on the LBP histogram. This represents the frequency corresponding to the g-th LBP value on the local LBP histogram of the first image. To reference the frequency corresponding to the g-th LBP value on the local LBP histogram

[0059] In addition, in specific applications, implementers need to set a preset first judgment threshold and a preset second judgment threshold according to the actual situation. For example, in this embodiment, both the preset first judgment threshold and the preset second judgment threshold can be set to 0.6.

[0060] This concludes the monitoring of fall behavior among elderly people living at home.

[0061] In summary, this embodiment first uses an infrared thermal imaging camera to acquire the monitoring video of the monitored object, and acquires the target area on the image to be analyzed and the first suspected fall image in the monitoring video. Then, it acquires the local window corresponding to the first suspected fall image, and records all images in the local window as images to be judged. Based on the major axis angle, centroid, and axial ratio of the minimum circumscribed ellipse of the target area on the image to be judged, it obtains the angle change characterization value of the local window corresponding to the first suspected fall image, the centroid shift degree of the image to be judged, and the ellipse deformation degree of the image to be judged. Finally, based on the angle change characterization value, centroid shift degree, and ellipse deformation degree, it monitors the fall behavior of the monitored object. Furthermore, this embodiment, by relying on the angle change characterization value, centroid shift degree, and ellipse deformation degree, can improve the accuracy of monitoring the fall behavior of the monitored object.

[0062] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A thermal imaging-based system for monitoring abnormal behavior in the elderly, comprising a processor and a memory, characterized in that, The processor executes the computer program stored in the memory to perform the following steps: An infrared thermal imaging camera is used to acquire a monitoring video of the monitored object to be analyzed. The monitoring video to be analyzed consists of an image to be analyzed. The target area on the image to be analyzed and the first suspected fall image in the monitoring video to be analyzed are acquired. The target area on the image to be analyzed is the bounding box of the monitored object in the corresponding image to be analyzed. Obtain the local window corresponding to the first suspected fall image, and record all images in the local window as images to be judged. Based on the major axis angle, centroid and axis ratio of the minimum circumscribed ellipse of the target area on the image to be judged, obtain the angle change characterization value of the local window corresponding to the first suspected fall image, the centroid offset of the image to be judged, and the degree of ellipse deformation of the image to be judged. The major axis angle of the minimum circumscribed ellipse is the angle between the major axis of the minimum circumscribed ellipse and the horizontal ground. The fall behavior of the monitored object is monitored based on the angle change value, the center of gravity shift, and the degree of ellipse deformation. The method for obtaining the centroid offset of the image to be judged includes: For the b-th image to be judged in the local window B corresponding to any first suspected fall image: the adjacent images to the left and right of the b-th image to be judged in the local window B are respectively denoted as the left adjacent image and the right adjacent image of the b-th image to be judged. The distance between the centroid of the minimum bounding ellipse of the target region on the b-th image to be judged and the centroid of the minimum bounding ellipse of the target region on the left adjacent image is denoted as the first distance. The distance between the centroid of the minimum bounding ellipse of the target region on the right adjacent image and the centroid of the minimum bounding ellipse of the target region on the b-th image to be judged is denoted as the second distance. If b is 1, then... As the centroid offset corresponding to the b-th image to be judged, if b is M, then... As the centroid offset corresponding to the b-th image to be judged, if b is not equal to 1 or M, then... As the centroid offset corresponding to the b-th image to be judged, D1 is the first distance, D2 is the second distance, c1 is the preset first constant, Norm() is the normalization function, and M is the total number of images in the local window B.

2. The thermal imaging-based elderly behavioral abnormality monitoring system as described in claim 1, characterized in that, The method for obtaining the first suspected fall image in the monitoring video to be analyzed includes: The target region on the image to be analyzed is input into the classification model, and the behavior type corresponding to the target region on the image to be analyzed is output. Based on the behavior type corresponding to the target region on the image to be analyzed, images to be analyzed that belong to the behavior type of falling are obtained from the monitoring video to be analyzed, and all images to be analyzed that belong to the behavior type of falling are recorded as the first suspected falling images.

3. The thermal imaging-based elderly behavioral abnormality monitoring system as described in claim 1, characterized in that, The method for obtaining the local window corresponding to the first suspected fall image includes: For any first suspected fall image, in the monitoring video to be analyzed, a window is formed by arranging the first suspected fall image and a preset number of images to be analyzed whose acquisition time is earlier than the first suspected fall image in chronological order of acquisition time. This window is denoted as the local window corresponding to the first suspected fall image.

4. The thermal imaging-based elderly behavior abnormality monitoring system as described in claim 1, characterized in that, The method for obtaining the angle change representation value of the local window corresponding to the first suspected fall image includes: For any first suspected fall image, a sequence of cosine values ​​of the local window corresponding to the first suspected fall image is obtained, and the mean of the cosine value sequence is used as the angular change representation value of the local window corresponding to the first suspected fall image. The t-th cosine value in the cosine value sequence is the cosine value of the t-th target angle in the target angle sequence. The t-th target angle in the target angle sequence is the absolute value of the difference between the major axis angle of the (t+1)-th image to be judged and the major axis angle of the t-th image to be judged in the local window corresponding to the first suspected fall image. The major axis angle of the (t+1)-th image to be judged is the major axis angle of the minimum circumscribed ellipse of the target region on the (t+1)-th image to be judged. The major axis angle of the t-th image to be judged is the major axis angle of the minimum circumscribed ellipse of the target region on the t-th image to be judged.

5. The thermal imaging-based elderly behavior abnormality monitoring system as described in claim 1, characterized in that, The method for obtaining the degree of ellipse deformation corresponding to the image to be judged includes: For the b-th image to be judged: the axis ratio of the minimum bounding ellipse of the target region in the b-th image to be judged, the axis ratio of the minimum bounding ellipse of the target region in the left adjacent image of the b-th image to be judged, and the axis ratio of the minimum bounding ellipse of the target region in the right adjacent image of the b-th image to be judged are respectively denoted as the first axis ratio, the second axis ratio, and the third axis ratio. The difference between the third axis ratio and the first axis ratio is denoted as the first axis ratio difference. The difference between the first axis ratio and the second axis ratio is denoted as the second axis ratio difference. The ratio of the first axis ratio difference to the first axis ratio is denoted as the first feature ratio. The ratio of the second axis ratio difference to the second axis ratio is denoted as the second feature ratio. If b is not equal to 1 or M, then the absolute value of the difference between the first feature ratio and the second feature ratio is recorded as the degree of ellipse deformation corresponding to the b-th image to be judged. If b is equal to 1, then the absolute value of the first feature ratio is recorded as the degree of ellipse deformation corresponding to the b-th image to be judged. If b is equal to M, then the absolute value of the second feature ratio is recorded as the degree of ellipse deformation corresponding to the b-th image to be judged.

6. The thermal imaging-based elderly behavior abnormality monitoring system as described in claim 2, characterized in that, A method for monitoring the fall behavior of the monitored object based on the included angle change value, the center of gravity shift, and the degree of ellipse deformation includes: Based on the angle change characteristic value, the center of gravity offset, and the degree of ellipse deformation, an initial fall determination index value for the first suspected fall image is obtained; based on the initial fall determination index value, the first suspected fall image is filtered, and a target fall determination index value for the second suspected fall image is obtained based on the filtering result; it is determined whether the target fall determination index value of the second suspected fall image is greater than a preset second determination threshold. If so, the behavior type of the monitored object in the corresponding second suspected fall image is determined to be a fall behavior.

7. The thermal imaging-based elderly behavioral abnormality monitoring system as described in claim 6, characterized in that, The method for obtaining the initial fall determination index value of the first suspected fall image includes: Regarding any of the first suspected fall images: The local window corresponding to the first suspected fall image is denoted as local window B, and the average value of the center of gravity offset corresponding to all images to be judged in local window B is denoted as the average center of gravity offset. Obtain the offset difference feature values ​​corresponding to each image to be judged in the local window B, and record the average of the offset difference feature values ​​corresponding to all images to be judged in the local window B as the centroid offset difference characterization value of the local window corresponding to the first suspected fall image. The offset difference feature value corresponding to the b-th image to be judged in the local window B is the normalized value of the absolute value of the difference between the weighted centroid offset of the b-th image to be judged and the average centroid offset. The weighted centroid offset of the b-th image to be judged is the result of multiplying the centroid offset of the b-th image to be judged by the reference weight of the b-th image to be judged. The reference weight of the b-th image to be judged is... c2 is a preset second constant, and Norm() is a normalization function; the normalized value of the degree of elliptical deformation corresponding to each image to be judged in the local window B is recorded as the normalized elliptical deformation degree corresponding to the image to be judged, and the mean of the normalized elliptical deformation degree corresponding to all images to be judged in the local window B is recorded as the elliptical deformation characterization value of the local window corresponding to the first suspected fall image; the weighted sum of the angle change characterization value, the centroid shift difference characterization value, and the elliptical deformation characterization value of the local window corresponding to the first suspected fall image is used as the initial fall judgment index value of the first suspected fall image.

8. The thermal imaging-based elderly behavioral abnormality monitoring system as described in claim 6, characterized in that, A method for filtering the first suspected fall images based on the initial fall determination index value, and obtaining the target fall determination index value for the second suspected fall image based on the filtering results, includes: The first suspected fall image whose initial fall determination index value is greater than the preset first determination threshold is recorded as the second suspected fall image. The target fall determination index value of the second suspected fall image is obtained based on the histogram of each image in the local window corresponding to the second suspected fall image.

9. The thermal imaging-based elderly behavior abnormality monitoring system as described in claim 8, characterized in that, A method for obtaining the target fall determination index value of the second suspected fall image based on the histograms of each image in the local window corresponding to the second suspected fall image includes: In the monitoring video to be analyzed, any image to be analyzed that belongs to the behavior type of bending over is recorded as a reference image, and a reference local window corresponding to the reference image is obtained. All images to be analyzed in the reference local window are recorded as feature images. The method of obtaining the reference local window is the same as the method of obtaining the local window corresponding to the first suspected fall image. Obtain the LBP histogram of the minimum bounding ellipse of the target region on the feature image, and record the average LBP histogram of the LBP histograms corresponding to all feature images in the reference local window as the reference LBP histogram; For any second suspected fall image: The local window corresponding to the second suspected fall image is designated as the local window to be analyzed; all images within the local window to be analyzed are designated as first images; the LBP histogram of the minimum bounding ellipse of the target region on the first image is obtained and designated as the local LBP histogram of the corresponding first image; the chi-square distance between the local LBP histogram of the first image and the reference LBP histogram is obtained and designated as the feature distance representation value of the corresponding first image; the normalized value of the ratio of the feature distance representation value of the first image to the total number of LBP values ​​on the LBP histogram is designated as the behavioral representation value of the corresponding first image; the mean of the behavioral representation values ​​of all first images within the local window to be analyzed is used as the target fall determination index value of the second suspected fall image.

Images