Old people behavior abnormity monitoring system based on thermal imaging

Abstract

The invention relates to the technical field of intelligent monitoring, in particular to an old people behavior abnormity monitoring system based on thermal imaging, which comprises a processor and a memory, the processor executes a computer program stored in the memory so as to realize the following steps: according to a long axis included angle, a gravity center and an axial ratio of a minimum circumscribed ellipse of a target area on a to-be-judged image; and obtaining an included angle change representation value of a local window corresponding to the first suspected fall image, a gravity center shift degree corresponding to the to-be-judged image and an elliptical deformation degree corresponding to the to-be-judged image, and monitoring the fall behavior of the monitored object according to the included angle change representation value, the gravity center shift degree and the elliptical deformation degree. And the accuracy of monitoring the falling behavior of the monitored object can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring, and particularly relates to an elderly behavior anomaly monitoring system based on thermal imaging. Background Art

[0002] With the intensification of the global aging trend, the monitoring of abnormal behaviors of the elderly has gradually become an important research direction in the field of health care. In particular, the fall behavior of the elderly, that is, falling is one of the main factors endangering the health of the elderly. Because the elderly have osteoporosis and weak organ compensation ability, falling is likely to cause complications such as hip fractures, intracranial hemorrhage, and myocardial infarction. If these symptoms cannot be treated in time, it will lead to irreparable serious consequences such as death and paralysis. Therefore, it is crucial to timely identify the fall behavior of the elderly at present, and the current monitoring of the behavior of the elderly is mainly applied in smart homes, nursing homes, and medical institutions.

[0003] In the prior art, an infrared thermal imaging camera is usually used to obtain a single-frame monitoring image of the monitored object, and then the collected single-frame monitoring image is input into a classification model, and the model outputs the behavior type of the monitored object in the single-frame monitoring image. However, when monitoring the fall behavior of the elderly based on a single-frame image and a classification model, the dynamic factors of the behavior are not considered, resulting in false detection or misidentification. For example, if the behavior of an elderly person in a certain frame of image is a fall, then when using the existing method for identification, the behavior of the elderly person in this image may be identified as normal behaviors such as quickly bending down to pick up items, slowly sitting down, or slowly getting up. That is, the existing monitoring methods for the fall behavior of the elderly are prone to false detection. Therefore, when monitoring or identifying the fall behavior of the elderly, how to improve the accuracy of monitoring or identification is an urgent problem to be solved. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides an elderly behavior anomaly monitoring system based on thermal imaging, and the specific technical solution adopted is as follows:

[0005] An embodiment of the present invention provides an elderly behavior anomaly monitoring system based on thermal imaging, including a processor and a memory. The processor executes the computer program stored in the memory to implement the following steps:

[0006] Using an infrared thermal imaging camera to obtain a to-be-analyzed monitoring video of the monitored object, the to-be-analyzed monitoring video is composed of to-be-analyzed images, and obtaining the target area on the to-be-analyzed images and the first suspected fall image in the to-be-analyzed monitoring video. The target area on the to-be-analyzed images is the bounding box corresponding to the monitored object in the to-be-analyzed images;

[0007] Obtain the local window corresponding to the first suspected fall image, and record all the images in the local window as images to be judged. According to the major axis included angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target area on the image to be judged, obtain the included angle change characterization value of the local window corresponding to the first suspected fall image, the centroid offset degree corresponding to the image to be judged, and the ellipse deformation degree corresponding to the image to be judged. The major axis included angle of the minimum circumscribed ellipse is the included angle between the major axis of the corresponding minimum circumscribed ellipse and the horizontal ground;

[0008] Monitor the fall behavior of the monitored object according to the included angle change characterization value, the centroid offset degree, and the ellipse deformation degree.

[0009] Beneficial effects: The present invention first uses an infrared thermal imaging camera to obtain the monitoring video to be analyzed of the monitored object, and obtains the target area on the image to be analyzed and the first suspected fall image in the monitoring video to be analyzed; then obtains the local window corresponding to the first suspected fall image, and records all the images in the local window as images to be judged. According to the major axis included angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target area on the image to be judged, obtain the included angle change characterization value of the local window corresponding to the first suspected fall image, the centroid offset degree corresponding to the image to be judged, and the ellipse deformation degree corresponding to the image to be judged; finally, monitor the fall behavior of the monitored object according to the included angle change characterization value, the centroid offset degree, and the ellipse deformation degree. And based on the included angle change characterization value, the centroid offset degree, and the ellipse deformation degree, the present invention can improve the accuracy of monitoring the fall behavior of the monitored object. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a flowchart of a method for monitoring abnormal behavior of the elderly based on thermal imaging according to the present invention. Detailed Embodiments

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the embodiments of the present invention.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0014] This embodiment provides a monitoring system for abnormal behavior of the elderly based on thermal imaging, including a processor and a memory. The processor executes the computer program stored in the memory to implement a method for monitoring abnormal behavior of the elderly based on thermal imaging, as Figure 1 shown. The method for monitoring abnormal behavior of the elderly based on thermal imaging includes the following steps:

[0015] Step S001, using an infrared thermal imaging camera to obtain a monitoring video to be analyzed of the monitored object. The monitoring video to be analyzed is composed of images to be analyzed, and obtain the target area on the image to be analyzed and the first suspected fall image in the monitoring video to be analyzed.

[0016] Due to normal behaviors such as quickly bending down to pick up items, slowly sitting down or slowly getting up, when identifying or detecting the fall behavior of the elderly in the prior art, it is easy to have misidentifications or false detections, which will lead to the inability to timely detect the fall behavior of the elderly. When the fall behavior of the elderly cannot be detected in time, it will lead to serious irreparable consequences such as death, paralysis, etc. Therefore, the main purpose of this embodiment is to improve the accuracy of identifying or monitoring the fall behavior of the elderly, so as to provide timely treatment when a fall occurs; and because the monitoring of the behavior of the elderly is currently applied in multiple scenarios, such as smart homes, nursing homes and medical institutions, and for the convenience of understanding in this embodiment, only any one scenario is selected for monitoring. For example, this embodiment mainly monitors the fall behavior of the elderly at home, and this embodiment monitors any elderly person at home, that is, the monitored object described later in this embodiment is the same elderly person.

[0017] In this embodiment, when monitoring the fall behavior of the elderly at home, an infrared thermal imaging camera needs to be arranged in the living place of the elderly. And because the infrared thermal imaging camera is not affected by the lighting conditions, it can work stably at night or in low-light environments, and at the same time effectively protects user privacy and avoids the leakage of sensitive information such as face recognition. Therefore, the infrared thermal imaging camera has become an important device for monitoring the behavior of the elderly; and the arrangement position and quantity of the infrared thermal imaging camera need to be set by relevant staff according to the size and structure of the living place, but it is required that when the elderly are moving at any position in their living place, they can be monitored by the infrared thermal imaging camera; in addition, for the convenience of understanding in this embodiment, subsequently, the video captured by any infrared thermal imaging camera will be used as the basis to determine whether the monitored elderly person has fallen, that is, the videos or images that appear later in this embodiment are all collected by the same infrared thermal imaging camera.

[0018] After that, the video captured by the infrared thermal imaging camera during the current monitoring period is obtained and recorded as the initial video. The initial video consists of multiple frames of images, and all the images in the initial video are recorded as initial images. Then, all the initial images are preprocessed, and the preprocessed images are recorded as the images to be screened corresponding to the initial images. Then, a target detection algorithm is used to identify the monitored object in the images to be screened, and the bounding box of the monitored object is obtained on the images to be screened. The bounding box area of the monitored object on the images to be screened is recorded as the target area corresponding to the images to be screened, and the monitored object is an elderly person. And because the above initial video is only the video captured by one camera, there will be a situation where the images to be screened do not contain the monitored object, that is, the monitored object does not always move at the same position, and the images that do not contain the monitored object do not have the value of analysis. Therefore, in the initial video of this embodiment, the images to be screened that do not contain the target area are removed, and the video composed of the remaining images to be screened after the removal is recorded as the monitoring video to be analyzed of the monitored object, and the images in the monitoring video to be analyzed are all recorded as the images to be analyzed, that is, the monitoring video to be analyzed consists of the images to be analyzed.

[0019] In this embodiment, the implementer needs to set the acquisition frame rate of the camera and the time length of the current monitoring period according to the actual situation. For example, in this embodiment, the acquisition frame rate can be set to 25 FPS, and the time length of the current monitoring period can be set to 1 minute. The preprocessing in this embodiment is to perform image enhancement, Gaussian filter smoothing, histogram equalization, etc. on the images. In addition, in this embodiment, the implementer also needs to select the type of the target detection algorithm according to the actual situation. For example, in this embodiment, the YOLO target detection algorithm can be used to obtain the bounding box of the monitored object in the image, and the process of using the target detection algorithm to obtain the bounding box of the monitored object in the image is a well-known technology, so it will not be described in detail in this embodiment.

[0020] Since the images to be analyzed belong to the images to be screened, and it can be seen from the above description that the monitoring video to be analyzed is composed of the images to be screened containing the target area, and the target area is the bounding box of the monitored object, so it can be known from the above process that the target area on the images to be analyzed has been obtained.

[0021] After obtaining the monitoring video to be analyzed and the target area on each image to be analyzed in the monitoring video to be analyzed, the first suspected fall image in the monitoring video to be analyzed is obtained, and the specific obtaining process is as follows:

[0022] First, obtain the trained classification model. Then, input the target regions on the images to be analyzed into the trained classification model respectively. The model outputs the behavior types corresponding to the target regions on each image to be analyzed, and the behavior type corresponding to the target region on the image to be analyzed is the behavior type of the monitored object in the target region on the corresponding image to be analyzed. After that, according to the behavior types corresponding to the target regions on the images to be analyzed, obtain the images to be analyzed in the monitored video to be analyzed whose behavior types belong to the fall behavior, and record all the images to be analyzed belonging to the fall behavior as the first suspected fall images. And when performing behavior type recognition based on the classification model, the input

[0023] And in this embodiment, the classification model is mainly used to identify the behavior types of the monitored objects on the images. And the behavior types in this embodiment mainly include two categories, namely normal behavior and fall behavior. The normal behavior includes but is not limited to bending, standing upright, sitting down, etc., and the fall behavior includes but is not limited to falling forward, falling backward, falling sideways, etc. And in specific applications, the implementer needs to select the type of the classification model according to the actual situation. For example, in this embodiment, the MobileNet-FallNet model can be selected to perform behavior type recognition. The MobileNet-FallNet is an improvement based on MobileNet. MobileNet is a lightweight deep neural network designed for mobile and embedded devices. And this MobileNet-FallNet model mainly first uses the MobileNet network for feature extraction, that is, the input image will first be converted into a compact feature vector through the MobileNet network. The feature vector contains key information such as human body contour and posture, providing a basis for subsequent classification tasks. After that, the features extracted by MobileNet will be input into the fully connected layer, and the probability distribution of different behavior types will be calculated through the Softmax function. Based on the probability size, the final output is the prediction result of the behavior category. In addition, since the training process of the model is a well-known technology, it will not be described in detail in this embodiment.

[0024] Therefore, through the above process, this embodiment can obtain the monitored video to be analyzed, the target regions on each image to be analyzed in the monitored video to be analyzed, and the first suspected fall images in the monitored video to be analyzed. And the first suspected fall images may include images that do not belong to the fall behavior type, that is, the first suspected fall images may include images whose behavior types belong to normal behavior, such as bending and sitting down. Therefore, subsequent re-analysis and screening are required.

[0025] Step S002: Obtain the local window corresponding to the first suspected fall image, and mark all the images in the local window as images to be judged. According to the major axis included angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target region on the image to be judged, obtain the included angle change characterization value of the local window corresponding to the first suspected fall image, the centroid offset degree corresponding to the image to be judged, and the ellipse deformation degree corresponding to the image to be judged.

[0026] Based on the above description, it can be seen that the first suspected fall images obtained above may include images with a behavior type belonging to normal behavior. This phenomenon is because when performing behavior type recognition above, a single-frame image is input, and only the static information of the monitored object is considered. In order to improve the accuracy of behavior monitoring in this embodiment, the dynamic information of the monitored object will be combined for recognition, that is, in this embodiment, the characteristics in different behavior processes of the monitored object will be combined to monitor the fall behavior. Then, in the following, this embodiment will obtain the local window corresponding to the first suspected fall image, and the purpose of obtaining the local window is to combine the dynamic information of the monitored object. Therefore, in this embodiment, the specific process of obtaining the local window corresponding to the first suspected fall image is as follows:

[0027] For any first suspected fall image: In the video to be analyzed and monitored, obtain a preset number of images to be analyzed whose acquisition times are before this first suspected fall image, and mark the window formed by arranging this first suspected fall image and the preset number of images to be analyzed whose acquisition times are before this first suspected fall image in the order of acquisition time as the local window corresponding to this first suspected fall image; In specific applications, the implementer needs to set the value of the preset number according to the actual situation, and in this embodiment, it is required that the preset number is related to the size of the acquisition frame rate, that is, the more images are acquired per second, the larger the value of the preset number. If the acquisition frame rate is 30fps, then the value of the preset number is set to 30. If the acquisition frame rate is 60fps, then the value of the preset number is set to 60.

[0028] In addition, when the number of images to be analyzed before this first suspected fall image is less than the preset number, then use the window formed by all the images to be analyzed before this first suspected fall image in the video to be analyzed and monitored and this image to be analyzed as the local window corresponding to this first suspected fall image.

[0029] Therefore, through the above process, each local window corresponding to the first suspected fall image can be obtained in this embodiment. In this embodiment, all the images in each local window are recorded as images to be judged. After obtaining each local window corresponding to the first suspected fall image, the minimum circumscribed ellipse of the target region on each image to be judged in the local window is obtained, and the process of obtaining the minimum circumscribed ellipse of the target region is a well-known technology. Then, the major axis angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target region on each image to be judged are obtained. The major axis angle of the minimum circumscribed ellipse is the angle between the major axis of the corresponding minimum circumscribed ellipse and the horizontal ground. The centroid of the minimum circumscribed ellipse is the intersection point of the major axis and the minor axis of the corresponding minimum circumscribed ellipse. The axis ratio of the minimum circumscribed ellipse is the normalized value of the ratio of the major axis to the minor axis of the corresponding minimum circumscribed ellipse. For example, if the coordinates of the two intersection points of the major axis of any minimum circumscribed ellipse and the ellipse are (x 1 , y 1 ) and (x 2 , y 2 ), the major axis of this minimum circumscribed ellipse is a1 and the minor axis is a2, then the major axis angle of this minimum circumscribed ellipse is The major axis angle of this minimum circumscribed ellipse is the axis ratio is Norm() is the normalization function, arctan() is the arctangent function, and the value range is from 0 to 1. In addition, the coordinate values of points in this embodiment all refer to the coordinate values of the corresponding points in the plane coordinate system, and the horizontal axis of the plane coordinate system is the horizontal ground and the vertical axis is perpendicular to the horizontal ground.

[0030] Then, according to the major axis angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target region on each to-be-determined image in the local window corresponding to each first suspected fall image, the included angle change characterization value of the local window corresponding to each first suspected fall image, the centroid offset degree corresponding to each to-be-determined image in the local window, and the ellipse deformation degree corresponding to each to-be-determined image in the local window are obtained; and in this embodiment, the reason for obtaining the included angle change characterization value, centroid offset degree, and ellipse deformation degree based on the major axis angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target region is that when the monitored object is in different behavioral states, the major axis angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target region will all change differently. Therefore, further analysis of the obtained included angle change characterization value, centroid offset degree, and ellipse deformation degree can improve the recognition of fall behavior. For example, in the static standing state, the major axis of the minimum circumscribed ellipse of the monitored object's contour is close to perpendicular to the ground, the axis ratio is close to 1, and the centroid hardly moves. The axis ratio is close to 1 because the contour of the monitored object usually shows a large aspect ratio at this time; during the fall process, the major axis angle of the minimum circumscribed ellipse of the monitored object's contour will deviate significantly and gradually tend to be horizontal. The centroid of the minimum circumscribed ellipse of the monitored object's contour will show a phenomenon of slow movement to accelerated movement, that is, the skewness of the centroid of the minimum circumscribed ellipse of the monitored object's contour first increases slowly and then increases sharply, and the axis ratio of the minimum circumscribed ellipse of the monitored object's contour approaches 0 when falling; during the process of slowly sitting down or bending down and getting up, the change range of the major axis angle of the minimum circumscribed ellipse of the monitored object's contour is small, and the major axis angle is close to 45 degrees at this time. The change of the axis ratio of the minimum circumscribed ellipse of the monitored object's contour is also small, and the axis ratio is close to 0.5 at this time.

[0031] In this embodiment, next, according to the major axis angle of the minimum circumscribed ellipse of the target region on each to-be-determined image in the local window corresponding to each first suspected fall image, the included angle change characterization value of the local window corresponding to each first suspected fall image is obtained, and the specific obtaining process of the included angle change characterization value of the local window corresponding to each first suspected fall image is as follows:

[0032] For any first suspected fall image W: First, obtain the target angle sequence of the local window corresponding to the first suspected fall image W. The t-th target angle in the target angle sequence is the absolute value of the difference between the major axis angle of the (t + 1)-th image to be judged in the local window corresponding to the first suspected fall image W and the major axis angle of the t-th image to be judged. The major axis angle of the (t + 1)-th image to be judged is the major axis angle of the minimum circumscribed ellipse of the target region on the (t + 1)-th image to be judged in the local window corresponding to this first suspected fall image, and the absolute value of the difference between the major axis angles of the t-th image to be judged is the major axis angle of the minimum circumscribed ellipse of the target region on the t-th image to be judged in the local window corresponding to this first suspected fall image. Then, obtain the cosine values of each target angle in the target angle sequence, and record the sequence composed of the cosine values of all target angles in the target angle sequence as the cosine value sequence of the local window corresponding to this first suspected fall image. That is, the t-th cosine value in the cosine value sequence of the local window corresponding to this first suspected fall image is the cosine value of the t-th target angle in the target angle sequence. After that, obtain the mean value of the cosine value sequence of the local window corresponding to the first suspected fall image W, and use it as the angle change characterization value of the local window corresponding to the first suspected fall image W. And in this embodiment, the specific calculation process of the angle change characterization value of the local window corresponding to the first suspected fall image W is as follows:

[0033]

[0034] Among them, Δθ is the angle change characterization value of the local window corresponding to the first suspected fall image W, T is the number of target angles in the target angle sequence of the local window corresponding to the first suspected fall image W, cos() is the cosine function, and θ t is the value of the t-th target angle in the target angle sequence; and when cos(θ t ) is larger, it indicates that Δθ is larger, and the larger Δθ is, it indicates that within the local window corresponding to the first suspected fall image W, the change rate of the major axis angle is larger. And when the change rate of the major axis angle is larger, it indicates that the probability that the behavior type of the detected object on the first suspected fall image W belongs to the fall behavior type is greater. On the contrary, when Δθ is smaller, it indicates that the probability that the behavior type of the detected object on the first suspected fall image W belongs to the fall behavior type is smaller.

[0035] Therefore, through the above process, this embodiment can obtain the angle change characterization value of the local window corresponding to each first suspected fall image. After obtaining the angle change characterization value, the centroid offset degree corresponding to each to-be-determined image in the local window corresponding to each first suspected fall image is obtained based on the centroid of the minimum circumscribed ellipse of the target region on each to-be-determined image in the local window. The specific process of obtaining the centroid offset degree corresponding to each to-be-determined image is as follows:

[0036] For the b-th to-be-determined image in the local window B corresponding to any first suspected fall image W:

[0037] First, in the local window B, obtain the to-be-determined image adjacent to and on the left of the b-th to-be-determined image, and denote it as the left adjacent image corresponding to the b-th to-be-determined image. Obtain the to-be-determined image adjacent to and on the right of the b-th to-be-determined image, and denote it as the right adjacent image corresponding to the b-th to-be-determined image. Then, denote the minimum circumscribed ellipse of the target region on the b-th to-be-determined image as the ellipse to be analyzed, denote the minimum circumscribed ellipse of the target region on the left adjacent image of the b-th to-be-determined image as the left ellipse, and denote the minimum circumscribed ellipse of the target region on the right adjacent image of the b-th to-be-determined image as the right ellipse. Then, obtain the distance between the centroid of the ellipse to be analyzed and the centroid of the left ellipse, and denote it as the first distance. Obtain the distance between the centroid of the ellipse to be analyzed and the centroid of the right ellipse, and denote it as the second distance. The calculation method of the distance between two centroids is the same as the calculation method of the distance between two points. The coordinates of the centroid refer to the coordinates in the plane coordinate system. Then, determine whether the value of b is 1. If so, use Norm(D2) as the centroid offset degree corresponding to the b-th to-be-determined image. Otherwise, determine whether the value of b is M. If so, use Norm(D1) as the centroid offset degree corresponding to the b-th to-be-determined image. Otherwise, determine whether the value of b is not equal to 1 and M. If so, use as the centroid offset degree corresponding to the b-th to-be-determined image, where D1 is the second distance, D2 is the first distance, c is a preset first constant, Norm() is a normalization function. The preset first constant is to prevent the denominator from being zero, and M is the total number of images in the local window B. In this embodiment, the subsequent analysis mainly focuses on the centroid offset degree to analyze the change of the centroid. The change of the centroid can reflect the probability that the behavior type of the detected object in the first suspected fall image belongs to the fall behavior type.

[0038] In addition, in specific applications, the implementer needs to set the value of the preset first constant according to the actual situation. For example, in this embodiment, the preset first constant can be set to 10 -6 。

[0039] Therefore, this embodiment can obtain the centroid offset corresponding to the image to be judged through the above process. Next, this embodiment will obtain the ellipse deformation degree corresponding to each image to be judged in the local window corresponding to each first suspected fall image according to the axis ratio of the minimum circumscribed ellipse of the target area on each image to be judged in the local window corresponding to each first suspected fall image. That is, the specific acquisition process of the ellipse deformation degree corresponding to the image to be judged is:

[0040] For the b-th image to be judged in the local window B corresponding to any first suspected fall image W:

[0041] First, the axis ratio of the minimum circumscribed ellipse of the target area on the b-th image to be judged, the axis ratio of the minimum circumscribed ellipse of the target area on the left adjacent image of the b-th image to be judged, and the axis ratio of the minimum circumscribed ellipse of the target area on the right adjacent image of the b-th image to be judged are recorded as the first axis ratio, the second axis ratio, and the third axis ratio, respectively; then the difference between the third axis ratio and the first axis ratio is obtained, and recorded as the first axis ratio difference, the difference between the first axis ratio and the second axis ratio is obtained, and recorded as the second axis ratio difference, and the ratio of the first axis ratio difference to the first axis ratio is recorded as the first feature ratio, and the ratio of the second axis ratio difference to the second axis ratio is recorded as the second feature ratio; then, it is determined whether the value of b is 1. If so, then As the degree of ellipse deformation corresponding to the b-th image to be judged, otherwise, it is determined whether the value of b is M. If so, As the degree of ellipse deformation corresponding to the b-th image to be judged, otherwise, it is determined whether the value of b is not equal to 1 and M. If so, then As the degree of ellipse deformation corresponding to the bth image to be judged, R3 is the third axis ratio, R2 is the second axis ratio, and R1 is the first axis ratio. is the first characteristic ratio, The second characteristic ratio is obtained. In this embodiment, the deformation of the ellipse in the local window is mainly analyzed by the degree of ellipse deformation, and the deformation of the ellipse in the local window can reflect the probability that the behavior type of the detected object in the first suspected fall image belongs to the fall behavior type.

[0042] Therefore, this embodiment can obtain the degree of ellipse deformation corresponding to the image to be judged through the above process; in addition, when the axis ratio changes more drastically during the fall, the degree of ellipse deformation will be greater. When sitting down or bending over, the axis ratio changes more steadily, and the degree of ellipse deformation will be smaller.

[0043] Step S003: monitoring the falling behavior of the monitored object according to the angle change characterization value, the center of gravity offset and the ellipse deformation degree.

[0044] Next, this embodiment will monitor the fall behavior of the monitored object during the current monitoring time period based on the obtained angle change characterization value, center of gravity offset degree, and ellipse deformation degree. The specific process of monitoring the fall behavior of the monitored object during the current monitoring time period based on the obtained angle change characterization value, center of gravity offset degree, and ellipse deformation degree is as follows:

[0045] First, based on the angle change characterization value of the local window corresponding to each first suspected fall image, the center of gravity offset degree corresponding to each image to be judged in the local window, and the ellipse deformation degree corresponding to each image to be judged in the local window, the initial fall determination index value of each first suspected fall image is obtained; then, based on the obtained initial fall determination index values of each first suspected fall image, the first suspected fall images are screened, and according to the screening results, the second suspected fall images and the target fall determination index values of the second suspected fall images are obtained; then, it is judged whether the target fall determination index values of all the second suspected fall images are not greater than the preset second determination threshold. If so, it is determined that the behavior type of the monitored object during the current monitoring time period belongs to the normal behavior type. In this case, there is no need to alarm the family members. When it is judged that the target fall determination index values of all the second suspected fall images are not all less than or equal to the preset second determination threshold, it indicates that the monitored object is in the fall behavior type during the current monitoring time period. In this case, the family members should be alarmed so that the monitored object can receive timely treatment, that is, if the target fall determination index value of the second suspected fall image is greater than the preset second determination threshold, it is determined that the behavior type of the monitored object in the corresponding second suspected fall image belongs to the fall behavior, and the family members should be alarmed.

[0046] In addition, in specific applications, the implementer needs to set the preset first determination threshold and the preset second determination threshold according to the actual situation. For example, in this embodiment, both the preset first determination threshold and the preset second determination threshold can be set to 0.6.

[0047] In this embodiment, the specific process of obtaining the initial fall determination index value of each first suspected fall image is as follows:

[0048] For any first suspected fall image W:

[0049] First, mark the local window corresponding to the first suspected fall image W as local window B. Denote the mean of the centroid offset degrees corresponding to all the images to be judged in local window B as the average centroid offset degree. Obtain the offset difference eigenvalue corresponding to each image to be judged in local window B, and denote the mean of the offset difference eigenvalues corresponding to all the images to be judged in local window B as the centroid offset difference characterization value of the local window corresponding to the first suspected fall image W. And the offset difference eigenvalue corresponding to the b-th image to be judged in local window B is the normalized value of the absolute value of the difference between the weighted centroid offset degree of the b-th image to be judged and the average centroid offset degree. The weighted centroid offset degree of the b-th image to be judged is the result of multiplying the centroid offset degree of the b-th image to be judged by the reference weight of the b-th image to be judged, and the reference weight of the b-th image to be judged is where c2 is a preset second constant, exp() is the exponential function with the constant e as the base. And if b is 1, then the reference weight of the b-th image to be judged is And the purpose of setting the reference weight is to make the reference value of the later acquired images larger.

[0050] Immediately afterwards, obtain the normalized value of the elliptical deformation degree corresponding to each image to be judged in local window B, and denote it as the normalized elliptical deformation degree corresponding to the image to be judged. Then, denote the mean of the normalized elliptical deformation degrees corresponding to all the images to be judged in local window B as the elliptical deformation characterization value of the local window corresponding to the first suspected fall image W. Finally, perform a weighted sum on the included angle change characterization value, the centroid offset difference characterization value, and the elliptical deformation characterization value of the local window corresponding to the first suspected fall image W, and use the result of the weighted sum as the initial fall determination index value of the first suspected fall image W. And the specific calculation expression of the initial fall determination index value of this first suspected fall image is:

[0051]

[0052] where S is the initial fall determination index value of the first suspected fall image W, λ 1 is the first weight value, λ 2 is the second weight value, λ 3 is the third weight value, M is the total number of images in local window B, U b is the weighted centroid offset degree of the b-th image to be judged in local window B, is the average centroid offset degree, ΔR b is the elliptical deformation degree corresponding to the b-th image to be judged in local window B, and 1 - exp(-ΔR b ) is to perform a normalization process on the elliptical deformation degree corresponding to the b-th image to be judged.

[0053] In addition, when Δθ is larger, The larger and ΔR b The larger it is, it indicates that the value of S is larger, and the larger the value of S is, it indicates that the probability that the behavior type of the monitored object in the first suspected fall image W belongs to the fall behavior is greater. And in specific applications, the implementer needs to set the values of the first weight value, the second weight value, the third weight value, and the preset second threshold according to the actual situation. For example, in this embodiment, the first weight value, the second weight value, and the third weight value can be set to 0.4, 0.3, and 0.3 respectively, and the preset second threshold is set to the variance value of the centroid offset degrees corresponding to all images in the local window B.

[0054] In this embodiment, a method for screening the first suspected fall images according to the initial fall determination index value and obtaining the target fall determination index value of the second suspected fall images based on the screening result includes:

[0055] Obtain the first suspected fall images whose initial fall determination index values are greater than the preset first determination threshold, and denote them as the second suspected fall images; and the behavior type of the monitored object in the first suspected fall images whose initial fall determination index values are not greater than the preset first determination threshold belongs to the normal behavior, while the behavior type of the monitored object in the first suspected fall images whose initial fall determination index values are greater than the preset first determination threshold may belong to the bending behavior or may belong to the fall behavior. After obtaining the second suspected fall images, it is also necessary to obtain the target fall determination index value corresponding to the second suspected fall images according to the histograms of the respective images in the local window corresponding to the second suspected fall images; and the specific obtaining process of the target fall determination index value corresponding to the second suspected fall images is as follows:

[0056] In the to-be-analyzed monitoring video, any to-be-analyzed image whose behavior type belongs to the bending behavior is denoted as the reference image, and the reference local window corresponding to the reference image is obtained. All the to-be-analyzed images in the reference local window are denoted as the feature images. The obtaining method of the reference local window is the same as the obtaining method of the local window corresponding to the first suspected fall images;

[0057] First, obtain the LBP histogram of the minimum circumscribed ellipse of the target region on the feature image, and denote it as the local LBP histogram corresponding to the corresponding feature image. The abscissa of the LBP histogram is the LBP value, and the ordinate is the frequency of the corresponding LBP value. The average local LBP histogram of the local LBP histograms corresponding to all the feature images in the reference local window is denoted as the reference local LBP histogram.

[0058] Subsequently, for any second suspected fall image: Denote the local window corresponding to the second suspected fall image as the local window to be analyzed, denote all the images in the local window to be analyzed as the first images, obtain the LBP histogram of the minimum circumscribed ellipse of the target region on the first image, and denote it as the local LBP histogram corresponding to the first image; obtain the chi-square distance between the local LBP histogram of the first image and the reference local LBP histogram, and denote it as the characteristic distance representation value of the first image; denote the normalized value of the ratio of the characteristic distance representation value of the first image to the number of LBP value types on the LBP histogram as the behavior representation value corresponding to the first image; take the mean of the behavior representation values of all the first images in the local window to be analyzed as the target fall determination index value corresponding to the second suspected fall image; the characteristic chi-square distance between the local LBP histogram corresponding to the first image and the reference local LBP histogram is where G is the number of LBP value types on the LBP histogram, and H g is the frequency corresponding to the g-th LBP value on the local LBP histogram corresponding to the first image, and is the frequency corresponding to the g-th LBP value on the reference local LBP histogram

[0059] In addition, in specific applications, the implementer needs to set a preset first determination threshold and a preset second determination threshold according to the actual situation. For example, in this embodiment, both the preset first determination threshold and the preset second determination threshold can be set to 0.6.

[0060] So far, this embodiment has completed the monitoring of the fall behavior of the elderly at home.

[0061] In summary, this embodiment first uses an infrared thermal imaging camera to obtain the monitoring video to be analyzed of the monitored object, and obtains the target region on the image to be analyzed and the first suspected fall images in the monitoring video to be analyzed; then obtains the local window corresponding to the first suspected fall image, and denotes all the images in the local window as the images to be judged. According to the major axis angle, centroid, and axis ratio of the minimum circumscribed ellipse of the target region on the image to be judged, the included angle change representation value corresponding to the local window of the first suspected fall image, the centroid offset degree corresponding to the image to be judged, and the ellipse deformation degree corresponding to the image to be judged are obtained; finally, according to the included angle change representation value, the centroid offset degree, and the ellipse deformation degree, the fall behavior of the monitored object is monitored. And this embodiment can improve the accuracy of monitoring the fall behavior of the monitored object based on the included angle change representation value, the centroid offset degree, and the ellipse deformation degree.

[0062] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A system for monitoring abnormal behavior of the elderly based on thermal imaging, comprising a processor and a memory, characterized in that: The processor executes the computer program stored in the memory to implement the following steps: Using an infrared thermal imaging camera to obtain a monitoring video to be analyzed of the monitored object, the monitoring video to be analyzed is composed of an image to be analyzed, and a target area on the image to be analyzed and a first suspected fall image in the monitoring video to be analyzed are obtained, wherein the target area on the image to be analyzed is a boundary box of the monitored object in the image to be analyzed; Obtain a local window corresponding to the first suspected fall image, and record all images in the local window as images to be judged, and obtain an angle change characterization value of the local window corresponding to the first suspected fall image, a degree of gravity offset corresponding to the image to be judged, and a degree of ellipse deformation corresponding to the image to be judged according to the major axis angle, center of gravity, and axis ratio of the minimum circumscribed ellipse of the target area on the image to be judged, wherein the major axis angle of the minimum circumscribed ellipse is the angle between the major axis of the corresponding minimum circumscribed ellipse and the horizontal ground; The falling behavior of the monitored object is monitored according to the angle change characterization value, the center of gravity offset and the ellipse deformation degree.

2. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 1, characterized in that: The method for acquiring the first suspected fall image in the monitoring video to be analyzed includes: The target area on the image to be analyzed is input into the classification model, and the behavior type corresponding to the target area on the image to be analyzed is output, where the behavior type corresponding to the target area on the image to be analyzed is the behavior type of the monitored object in the target area on the image to be analyzed; according to the behavior type corresponding to the target area on the image to be analyzed, the image to be analyzed whose behavior type belongs to the fall behavior is obtained in the monitoring video to be analyzed, and all the images to be analyzed belonging to the fall behavior are recorded as the first suspected fall images.

3. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 1, characterized in that: The method for acquiring the local window corresponding to the first suspected fall image includes: For any first suspected fall image, in the monitoring video to be analyzed, a window formed by arranging the first suspected fall image and a preset number of images to be analyzed whose acquisition time is before the first suspected fall image in the order of acquisition time is recorded as the local window corresponding to the first suspected fall image.

4. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 1, characterized in that: The method for obtaining the angle change representation value of the local window corresponding to the first suspected fall image includes: For any first suspected fall image, a cosine value sequence of a local window corresponding to the first suspected fall image is obtained, and the mean value of the cosine value sequence is used as a characterization value of the angle change of the local window corresponding to the first suspected fall image, the tth cosine value in the cosine value sequence is the cosine value of the tth target angle in the target angle sequence, the tth target angle in the target angle sequence is the absolute value of the difference between the major axis angle of the t+1th image to be judged in the local window corresponding to the first suspected fall image and the major axis angle of the tth image to be judged, the major axis angle of the t+1th image to be judged is the major axis angle of the minimum circumscribed ellipse of the target area on the t+1th image to be judged, and the major axis angle of the tth image to be judged is the major axis angle of the minimum circumscribed ellipse of the target area on the tth image to be judged.

5. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 1, characterized in that: The method for obtaining the center of gravity offset corresponding to the image to be judged includes: For the b-th image to be judged in the local window B corresponding to any first suspected fall image: the adjacent image to be judged on the left side and the adjacent image to be judged on the right side of the b-th image to be judged in the local window B are respectively recorded as the left adjacent image and the right adjacent image of the b-th image to be judged, and the distance between the center of gravity of the minimum circumscribed ellipse of the target area on the b-th image to be judged and the center of gravity of the minimum circumscribed ellipse of the target area on the left adjacent image is recorded as the first distance, and the distance between the center of gravity of the minimum circumscribed ellipse of the target area on the b-th image to be judged and the center of gravity of the minimum circumscribed ellipse of the target area on the right adjacent image is recorded as the second distance; if b is 1, Norm(D2) is used as the center of gravity offset corresponding to the b-th image to be judged; if b is M, Norm(D1) is used as the center of gravity offset corresponding to the b-th image to be judged; if b is not equal to 1 and M, As the centroid offset corresponding to the bth image to be judged, D1 is the second distance, D1 is the first distance, c is the preset first constant, Norm() is the normalization function, and M is the total number of images in the local window B.

6. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 5, characterized in that: The method for obtaining the degree of ellipse deformation corresponding to the image to be determined comprises: For the b-th image to be determined: the axis ratio of the minimum circumscribed ellipse of the target area on the b-th image to be determined, the axis ratio of the minimum circumscribed ellipse of the target area on the left adjacent image of the b-th image to be determined, and the axis ratio of the minimum circumscribed ellipse of the target area on the right adjacent image of the b-th image to be determined are respectively recorded as the first axis ratio, the second axis ratio, and the third axis ratio, the difference between the third axis ratio and the first axis ratio is recorded as the first axis ratio difference, the difference between the first axis ratio and the second axis ratio is recorded as the second axis ratio difference, the ratio of the first axis ratio difference to the first axis ratio is recorded as the first characteristic ratio, and the ratio of the second axis ratio difference to the second axis ratio is recorded as the second characteristic ratio; If b is not equal to 1 and M, the absolute value of the difference between the first feature ratio and the second feature ratio is recorded as the degree of ellipse deformation corresponding to the b-th image to be judged. If b is equal to 1, the absolute value of the first feature ratio is recorded as the degree of ellipse deformation corresponding to the b-th image to be judged. If b is equal to M, the second feature ratio is recorded as the degree of ellipse deformation corresponding to the b-th image to be judged.

7. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 2, characterized in that: The method for monitoring the falling behavior of the monitored object according to the angle change characterization value, the center of gravity offset and the ellipse deformation degree includes: According to the angle change characterization value, the center of gravity offset and the ellipse deformation degree, an initial fall judgment index value of the first suspected fall image is obtained; according to the initial fall judgment index value, the first suspected fall image is screened, and according to the screening result, a target fall judgment index value of the second suspected fall image is obtained; it is determined whether the target fall judgment index value of the second suspected fall image is greater than a preset second judgment threshold, and if so, it is determined that the behavior type of the monitored object in the corresponding second suspected fall image belongs to a fall behavior.

8. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 7, characterized in that: The method for obtaining the initial fall determination index value of the first suspected fall image includes: For any first suspected fall image: The local window corresponding to the first suspected fall image is recorded as local window B, and the average of the center of gravity deviations corresponding to all the to-be-judged images in the local window B is recorded as the average center of gravity deviation; Obtain the offset difference characteristic values ​​corresponding to each image to be judged in the local window B, and record the average of the offset difference characteristic values ​​corresponding to all the images to be judged in the local window B as the gravity center offset difference characterization value of the local window corresponding to the first suspected fall image, the offset difference characteristic value corresponding to the b-th image to be judged in the local window B is the normalized value of the absolute value of the difference between the weighted gravity center offset of the b-th image to be judged and the average gravity center offset, the weighted gravity center offset of the b-th image to be judged is the result of multiplying the gravity center offset of the b-th image to be judged by the reference weight of the b-th image to be judged, and the reference weight of the b-th image to be judged is c2 is a preset second constant, exp() is an exponential function with the constant e as the base; the normalized value of the degree of elliptical deformation corresponding to each image to be judged in the local window B is recorded as the normalized degree of elliptical deformation corresponding to the corresponding image to be judged, and the average value of the normalized degree of elliptical deformation corresponding to all images to be judged in the local window B is recorded as the elliptical deformation representation value of the local window corresponding to the first suspected fall image; the weighted sum of the angle change representation value, the center of gravity offset difference representation value and the elliptical deformation representation value of the local window corresponding to the first suspected fall image is used as the initial fall judgment index value of the first suspected fall image.

9. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 7, characterized in that: The method of screening the first suspected fall image according to the initial fall determination index value, and obtaining a target fall determination index value of the second suspected fall image according to the screening result, comprises: The first suspected fall image whose initial fall judgment index value is greater than the preset first judgment threshold is recorded as the second suspected fall image, and the target fall judgment index value corresponding to the second suspected fall image is obtained according to the histogram of each image in the local window corresponding to the second suspected fall image.

10. The abnormal behavior monitoring system for the elderly based on thermal imaging as claimed in claim 9, characterized in that: The method of obtaining a target fall determination index value corresponding to the second suspected fall image according to a histogram of each image in a local window corresponding to the second suspected fall image includes: In the monitoring video to be analyzed, any image to be analyzed whose behavior type belongs to bending behavior is recorded as a reference image, and a reference local window corresponding to the reference image is obtained, and all images to be analyzed in the reference local window are recorded as feature images, and the method for obtaining the reference local window is the same as the method for obtaining the local window corresponding to the first suspected fall image; Obtaining an LBP histogram of the minimum circumscribed ellipse of the target area on the feature image, and recording it as a local LBP histogram corresponding to the corresponding feature image, wherein the abscissa on the LBP histogram is the LBP value and the ordinate is the frequency of the corresponding LBP value, and an average local LBP histogram of the local LBP histograms corresponding to all feature images in the reference local window is recorded as a reference local LBP histogram; For any second suspected fall image: the local window corresponding to the second suspected fall image is recorded as the local window to be analyzed, and all images in the local window to be analyzed are recorded as first images, and the LBP histogram of the minimum circumscribed ellipse of the target area on the first image is obtained, and recorded as the local LBP histogram corresponding to the first image; the chi-square distance between the local LBP histogram of the first image and the reference local LBP histogram is obtained, and recorded as the characteristic distance representation value of the first image; the normalized value of the ratio of the characteristic distance representation value of the first image to the number of LBP value types on the LBP histogram is recorded as the behavior representation value of the corresponding first image; the average of the behavior representation values ​​of all first images in the local window to be analyzed is used as the target fall judgment index value corresponding to the second suspected fall image.

Images