A person fall detection method and system based on binocular camera
Through binocular camera and image processing technology, the frame difference method and bit layering algorithm are used to obtain key information, and the denoising processing is combined with the non-local mean filtering algorithm, which solves the accuracy problem of traditional methods in dark light environments, and achieves the correctness of the two methods. High accuracy detection of personnel falls.
Patent Information
- Application Number
- CN202410261776.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-07
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-03-07
AI Technical Summary
Traditional fall detection methods are prone to false alarms or missed alarms in darker light environments, and it is difficult for a single sensor to accurately capture fall events, resulting in inaccurate monitoring.
Using a binocular camera-based personnel fall detection method, the key bit layers of the image are obtained through frame difference method and bit layering algorithm, cluster analysis, and denoising processing is performed with non-local mean filtering algorithm to obtain key points in the denoising image. , determine whether the person has fallen.
In darker light environments, the accuracy of fall detection is improved, false alarms and missed reports are reduced, and timely and accurate detection of falls for personnel.
Smart Images

Figure CN118279170B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for detecting a person falling based on a binocular camera. Background Art
[0002] In daily life, falls of the elderly or people with specific health problems are common but dangerous. Therefore, early detection and timely intervention of falls are crucial to individual health. Traditional fall detection methods are mainly based on accelerometers or sensor networks, but these methods may have problems of false positives or false negatives. In addition, the information of a single sensor is limited, making it difficult to fully and accurately capture fall events. The fall detection method based on visual information can make up for the shortcomings of sensor detection and detect falls in a timely and accurate manner. When using visual technology to monitor whether a person falls, it is mainly through obtaining key points on the person and judging whether the person falls according to the movement of the key points. However, in a dark environment, the collected images are affected by large noise, making the key points obtained inaccurate, resulting in inaccurate monitoring of falls. Therefore, it is necessary to denoise the collected images.
[0003] The non-local mean filtering algorithm is a filtering technology commonly used in image processing. Different from the traditional local mean filtering, the non-local mean filtering uses global information. By setting a sliding window to slide the entire image, several sliding windows are obtained, and the sliding windows are weighted averaged to reduce noise and maintain image details. Therefore, the filtering weight of each sliding window affects the accuracy of image denoising. If the filtering weight of the sliding window is not set accurately, it will affect the denoising effect of the image. Summary of the invention
[0004] In order to solve the above problems, the present invention provides a person fall detection method and system based on a binocular camera.
[0005] The present invention provides a method for detecting a person falling down based on a binocular camera using the following technical solutions:
[0006] An embodiment of the present invention provides a method for detecting a person falling down based on a binocular camera, the method comprising the following steps:
[0007] Collect each frame of image;
[0008] Record any frame image as the current frame image, and obtain the current frame marked image according to the current frame image and the next frame image of the current frame image; split the current frame marked image to obtain several bit layer images of the current frame marked image and the difference pixels in each bit layer image of the current frame marked image; according to the aggregation and number of the difference pixels in the bit layer image, obtain the possibility that each bit layer image of the current frame marked image is a key bit layer image; according to the possibility that each bit layer image of the current frame marked image is a key bit layer image, obtain the key bit layer of the current frame marked image. Image; cluster the difference pixels in the key bit layer image of the current frame marked image and other bit layer images respectively to obtain several clustering clusters of the key bit layer image of the current frame marked image and several clustering clusters of each other bit layer image of the current frame marked image; obtain the possibility degree of each clustering cluster in the key bit layer image of the current frame marked image being a reference cluster according to the several clustering clusters of the key bit layer image and the several clustering clusters of the other bit layer images; obtain the reference block of the current frame image according to the possibility degree of each clustering cluster in the key bit layer image of the current frame marked image being a reference cluster;
[0009] A sliding window is constructed with each pixel of the reference block of the current frame image as the central pixel to obtain each sliding window; a filtering weight of each sliding window is obtained according to the average gray value of the pixel in the reference block of the current frame image and the average gray value of the pixel in each sliding window; a filtering result of the central pixel of each sliding window is obtained according to the filtering weight of each sliding window and the average gray value of the pixel in the reference block of the current frame image to obtain a denoised image of the current frame;
[0010] Obtain each frame of denoised image; obtain key points in each frame of denoised image; obtain a judgment value of a person falling in each frame of denoised image according to the movement of key points in adjacent frame images; and judge whether a person has fallen according to the judgment value of a person falling in each frame of denoised image.
[0011] Preferably, the method of obtaining the current frame marked image according to the current frame image and the next frame image of the current frame image; splitting the current frame marked image to obtain several bit layer images of the current frame marked image and the difference pixels in each bit layer image of the current frame marked image includes the following specific steps:
[0012] Using a frame difference method, performing frame difference between the next frame image of the current frame image and the current frame image, obtaining difference pixel points between the current frame image and the next frame image of the current frame image, marking the difference pixel points between the current frame image and the next frame image of the current frame image in the current frame image, and obtaining a current frame marked image;
[0013] The current frame marked image is split using a bit layering algorithm to obtain a plurality of bit layer images of the current frame marked image and difference pixels in each bit layer image of the current frame marked image.
[0014] Preferably, the method of obtaining the possibility that each bit layer image of the current frame marked image is a key bit layer image according to the aggregation and number of difference pixel points in the bit layer image; obtaining the key bit layer image of the current frame marked image according to the possibility that each bit layer image of the current frame marked image is a key bit layer image includes the following specific steps:
[0015]
[0016] In the formula, p t Represents the possibility that the t-th bit layer image of the current frame marked image is a key bit layer image; N t represents the number of difference pixels in the tth bit layer image of the current frame mark image; M represents the number of difference pixels in the current frame mark image; d t (i,j) represents the Euclidean distance between the i-th difference pixel and the j-th difference pixel in the t-th bit layer image in the current frame marked image; exp() represents an exponential function with a natural constant as the degree.
[0017] Preferably, clustering the difference pixels in the key bit layer image of the current frame mark image and other bit layer images respectively to obtain several clustering clusters of the key bit layer image of the current frame mark image and several clustering clusters of each other bit layer image of the current frame mark image includes the following specific steps:
[0018] The number of clusters β and the cluster radius r are preset, and the DBSCAN density clustering algorithm is used based on β and r to cluster the difference pixels in the key bit layer image of the current frame marked image and the difference pixels in other bit layer images respectively, to obtain several clustering clusters of the key bit layer image of the current frame marked image and several clustering clusters of each other bit layer image of the current frame marked image.
[0019] Preferably, the method of obtaining the possibility that each cluster in the key bit layer image of the current frame marked image is a reference cluster based on the clusters of the key bit layer image and the clusters of other bit layer images includes the following specific steps:
[0020] Return each cluster in the key bit layer image of the current frame mark image to the current frame image to obtain a number of clusters of the current frame image;
[0021]
[0022] In the formula, qg Represents the probability that the gth cluster in the key bit layer image of the current frame mark image is the reference cluster; n g represents the number of difference pixels in the g-th cluster in the key bit layer image of the current frame mark image; A represents the number of difference pixels in the key bit layer image of the current frame mark image; Represents the average gray value of the difference pixel points of the g-th cluster in the current frame image; represents the average gray value of the pixels in the current frame image; || represents the absolute value symbol; d(g,hx) represents the distance between the centroid of the g-th cluster in the key bit layer image of the current frame marked image and the centroid of the x-th cluster in the other h-th bit layer image of the current frame marked image; H represents the number of other bit layer images in the current frame marked image; Z h Represents the number of clusters in the hth bit-layer image of the current frame mark image; G represents the sum of the number of clusters in all other bit-layer images of the current frame mark image; norm() represents the linear normalization function; exp() represents the exponential function with a natural constant as the base.
[0023] Preferably, the step of obtaining the reference block of the current frame image according to the possibility that each cluster in the key bit layer image of the current frame marked image is a reference cluster comprises the following specific steps:
[0024] Among the possible degrees that each cluster in the key bit layer image of the current frame marked image is a reference cluster, the cluster corresponding to the maximum possible degree is used as the reference cluster in the key bit layer image of the current frame marked image, and the corresponding reference cluster of the current frame image is obtained. The minimum enclosing rectangle of the reference cluster of the current frame image is obtained, and the area where the minimum enclosing rectangle is located is used as the reference block of the current frame image.
[0025] Preferably, the step of obtaining the filtering weight of each sliding window according to the average grayscale value of the pixels in the reference block of the current frame image and the average grayscale value of the pixels in each sliding window comprises the following specific steps:
[0026]
[0027] In the formula, α c Represents the filter weight of the cth sliding window; Represents the average gray value of the pixel in the cth sliding window; represents the average grayscale value of the pixels in the reference block; S represents the grayscale variance of all pixels in the reference block; V represents the grayscale variance of the pixels in the current frame image except the pixels in the reference block; S c represents the grayscale variance of the pixel in the cth sliding window; || represents the absolute value function; norm() represents the linear normalization function.
[0028] Preferably, the filtering result of the central pixel of each sliding window is obtained according to the filtering weight of each sliding window and the average gray value of the pixel in the reference block of the current frame image to obtain the denoised image of the current frame, which includes the following specific steps:
[0029]
[0030] In the formula, B c Represents the filtering result of the central pixel of the c-th sliding window; α c Represents the filter weight of the cth sliding window; Represents the average gray value of the pixels in the reference block of the current frame image; L c Represents the grayscale value of the central pixel of the c-th sliding window;
[0031] The current frame image after the reference block of the current frame image is denoised is recorded as the current frame denoised image.
[0032] Preferably, the step of obtaining key points in each denoised image frame; and obtaining a judgment value of a person falling in each denoised image frame according to the movement of key points in adjacent image frames comprises the following specific steps:
[0033] Use SIFT key point detection algorithm to detect key points of each denoised image frame, obtain the key points that have shifted in each denoised image frame, and record them as key points in each denoised image frame;
[0034] The displacement direction of the oth key point in the f-th frame denoised image compared to the oth key point in the f-1th frame denoised image is recorded as the displacement direction of the oth key point in the f-th frame denoised image; the displacement distance of the oth key point in the f-th frame denoised image compared to the oth key point in the f-1th frame denoised image is recorded as the displacement distance of the oth key point in the f-th frame denoised image;
[0035]
[0036] In the formula, γ f represents the judgment value of a person falling in the f-th frame of the denoised image; d fo represents the displacement distance of the oth key point in the fth frame denoised image; θ fo represents the displacement direction of the oth key point in the fth frame denoised image; d f+1,o represents the displacement distance of the oth key point in the f+1th frame denoised image; θ f+1,o represents the displacement direction of the oth key point in the f+1th frame denoised image; t f represents the acquisition time of the f-th frame image; t f+1represents the acquisition time of the f+1th frame image; || represents the absolute value symbol; U represents the number of key points in each frame of the denoised image.
[0037] The present invention also provides a binocular camera-based personnel fall detection system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements any one of the above steps when executing the computer program.
[0038] The beneficial effects of the technical solution of the present invention are as follows: the present invention first collects each frame of image. It is known that people in the surveillance video are moving. Therefore, the reference block of each frame of image is obtained according to the change of the grayscale value of the pixel points between adjacent frame images. The reference block of each frame of image represents the personnel area in each frame of image, which is convenient for subsequent denoising of the reference block of each frame of image; each sliding window is constructed with each pixel point of the reference block of the current frame image as the central pixel point; the filtering weight of each sliding window is obtained according to the grayscale variance of the pixel points in the reference block of the current frame image and the degree of noise influence of each sliding window. When the degree of noise influence of the reference block is greater and the degree of noise influence of each sliding window is greater, the filtering weight of the sliding window is greater, and the central pixel point of the sliding window should be filtered. According to the filtering weight of each sliding window, the filtering result of the central pixel point of each sliding window is obtained, and the reference block of the current frame image is filtered, so that the denoised current frame image is more accurate, and then the judgment of the fall of the person is more accurate according to each denoised frame image. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0040] Figure 1 The present invention is a flowchart of the steps of a method for detecting a person falling down based on a binocular camera. DETAILED DESCRIPTION
[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a person fall detection method based on a binocular camera proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0043] The following is a detailed description of a method for detecting a person falling down based on a binocular camera provided by the present invention in conjunction with the accompanying drawings.
[0044] See also Figure 1 , which shows a flowchart of a method for detecting a person falling down based on a binocular camera according to an embodiment of the present invention, the method comprising the following steps:
[0045] S001. Acquire each frame of image.
[0046] It should be noted that the purpose of the present invention is to perform denoising on each frame of the surveillance video, and then determine whether a person falls based on each frame of the denoising image.
[0047] In the embodiment of the present invention, a binocular camera is used to capture surveillance video within the activity range of personnel, and the surveillance video is processed into frames to obtain each frame image.
[0048] So far, each frame of the image has been collected.
[0049] S002. Record any frame image as the current frame image, obtain the current frame marked image, obtain several bit layer images of the current frame marked image, obtain the possibility degree of each bit layer image of the current frame marked image to be a key bit layer image, obtain the key bit layer image of the current frame marked image, cluster the difference pixel points in the key bit layer image of the current frame marked image and other positive bit layer images respectively; obtain the possibility degree of each clustering cluster in the key bit layer image of the current frame marked image to be a reference cluster, and then obtain the reference cluster of the current frame image; obtain the reference block of the current frame image according to the reference cluster of the current frame image.
[0050] It should be noted that when using a binocular camera to collect surveillance videos in the range of human activities, due to the dark environment in which the surveillance is collected, Gaussian white noise exists in each frame of the image, and the key surveillance object in the surveillance video is the moving person. Therefore, when denoising each frame of the image, the key denoising area is the person in the image. Therefore, it is first necessary to obtain the reference block of each frame of the image, that is, the personnel area in each frame of the image, based on the change in the grayscale value of the pixel points before the adjacent frame of the image. Since the people in the surveillance video are moving, the reference block of each frame of the image is obtained based on the difference between the adjacent frame images to filter and denoise it. In the embodiment of the present invention, any frame of the image is recorded as the current frame of the image, and the current frame of the image is used as an example for analysis.
[0051] In an embodiment of the present invention, any frame image is recorded as the current frame image, and the frame difference method is used to perform frame difference between the next frame image of the current frame image and the current frame image to obtain the difference pixel points between the current frame image and the next frame image of the current frame image, and the difference pixel points between the current frame image and the next frame image of the current frame image are marked in the current frame image to obtain the current frame marked image.
[0052] It should be noted that the difference pixels in the current frame image obtained may include pixels in the personnel area or pixels in other areas, and the grayscale values of pixels in different areas are different. Therefore, in an embodiment of the present invention, the current frame marked image is split into multiple different bit layer images, and different bit layer images reflect different grayscale value distributions. When the density of difference pixels in any bit layer image is large and the number of difference pixels in the bit layer image is large, it means that the difference pixels in the bit layer image are more likely to be pixels in the personnel area, and therefore the bit layer image is more likely to be a key bit layer image.
[0053] It should be further explained that after obtaining the key bit layer image of the current frame marking image, the difference pixel points in the key bit layer image of the current frame marking image are clustered, and each cluster cluster is analyzed to obtain the possibility of each cluster cluster in the key bit layer image of the current frame marking image being a reference cluster, and then the reference cluster of the current frame image is obtained, that is, the pixel points of the personnel area in the current frame image in the reference cluster of the current frame image, and then the reference block of the current frame image is obtained.
[0054] In the embodiment of the present invention, a bit layering algorithm is used to split the current frame marked image to obtain eight bit layer images of the current frame marked image. It should be noted that the bit layering algorithm is a well-known technology and will not be described in detail in the embodiment of the present invention.
[0055] Get the probability that each bit-layer image of the current frame marked image is a key bit-layer image:
[0056]
[0057] In the formula, p t Represents the possibility that the t-th bit layer image of the current frame marked image is a key bit layer image; N t represents the number of difference pixels in the t-th bit layer image of the current frame mark image; M represents the number of difference pixels in the current frame mark image; The ratio of the number of difference pixels in the t-th bit layer image of the current frame mark image to the number of difference pixels in the current frame mark image. The larger the ratio, the more pixels of the moving person are contained in the current frame mark image, and the greater the possibility that the t-th bit layer image of the current frame mark image is the key bit layer image; d t (i, j) represents the Euclidean distance between the i-th difference pixel and the j-th difference pixel in the t-th bit layer image in the current frame marked image; Represents the average distance between the difference pixels in the t-th bit layer image in the current frame mark image, reflecting the aggregation of the difference pixels in the bit layer image. It is known that the difference pixels in the current frame mark image include moving personnel pixels and other noise pixels, while the moving personnel pixels are more concentrated and the noise pixels are more random. Therefore The smaller the value is, the greater the density of the difference pixels is, which means that the possibility that the tth bit layer image of the current frame marked image is the key bit layer image is greater; exp() represents an exponential function with a natural constant as the degree. This embodiment uses the exp(-x) model to present the inverse proportional relationship. x is the input of the model. The implementer can set the inverse proportional function according to the actual situation.
[0058] In the possibility degree that each bit layer image of the current frame marked image is a key bit layer image, the bit layer image corresponding to the maximum value of the possibility degree is obtained as the key bit layer image of the current frame marked image.
[0059] In an embodiment of the present invention, the number of clusters β and the cluster radius r are preset, and the DBSCAN density clustering algorithm is used to cluster the difference pixels in the key bit layer image of the current frame marking image and the difference pixels in other bit layer images respectively. The distance measurement uses the Euclidean distance between the difference pixels to obtain several clustering clusters of the key bit layer image of the current frame marking image and several clustering clusters of each other bit layer image of the current frame marking image. In an embodiment of the present invention, the number of clusters β is preset to 5, and the cluster radius r is preset to 2. In other embodiments, the implementer may preset the values of β and r according to the specific implementation situation.
[0060] Each cluster in the key bit layer image of the current frame mark image is returned to the current frame image to obtain a plurality of clusters of the current frame image.
[0061] In the embodiment of the present invention, the possibility that the g-th cluster in the key bit layer image of the current frame marked image is a reference cluster is obtained:
[0062]
[0063] In the formula, q gRepresents the probability that the gth cluster in the key bit layer image of the current frame mark image is the reference cluster; n g represents the number of difference pixels in the g-th cluster in the key bit layer image of the current frame mark image; A represents the number of difference pixels in the key bit layer image of the current frame mark image; The ratio of the number of difference pixels in the g-th cluster to the number of all difference pixels in the key bit layer image representing the current frame mark image. When the number of pixels in the g-th cluster is larger, it is more likely to be a moving person area in the current frame image. Therefore, The larger the value of is, the greater the possibility that the gth cluster in the current key bit layer image is the reference cluster; Represents the average gray value of the difference pixel points of the g-th cluster in the current frame image; represents the average gray value of the pixels in the current frame image; || represents the absolute value symbol; d(g,hx) represents the distance between the centroid of the g-th cluster in the key bit layer image of the current frame marked image and the centroid of the x-th cluster in the other h-th bit layer image of the current frame marked image; H represents the number of other bit layer images in the current frame marked image, that is, the number of bit layer images in the current frame marked image except the key bit layer, Z h represents the number of clusters in the hth bit-layer image of the current frame mark image; G represents the sum of the number of clusters in all other bit-layer images of the current frame mark image; Represents the absolute value of the difference between the average grayscale value of the difference pixel points of the g-th cluster in the current frame image and the average grayscale value of the pixel points of the current frame image. If the g-th cluster in the current frame image contains multiple grayscale pixels, the smaller the grayscale difference with the current frame image, that is, The smaller the value of is, the greater the possibility that the gth cluster in the key bit layer image of the current frame mark image is the reference cluster; represents the mean distance between the centroid of the g-th cluster in the key bit layer image and the centroid of each cluster in other bit layer images. The smaller the value, the more likely it is that the g-th cluster in the key bit layer image is the reference cluster. norm() represents the normalization function, which uses a linear normalization method. The normalization object is the centroid of all clusters in the key bit layer image of the current frame. The value of; exp() represents an exponential function with a natural constant as the degree. This embodiment adopts the exp(-x) model to present the inverse proportional relationship. x is the input of the model. The implementer can set the inverse proportional function according to the actual situation.
[0064] Among the possible degrees that each cluster in the key bit layer image of the current frame marked image is a reference cluster, the cluster corresponding to the maximum possible degree is used as the reference cluster in the key bit layer image of the current frame marked image, and the corresponding reference cluster of the current frame image is obtained. The minimum enclosing rectangle of the reference cluster of the current frame image is obtained, and the area where the minimum enclosing rectangle is located is used as the reference block of the current frame image.
[0065] At this point, any frame image is recorded as the current frame image, the current frame marked image is obtained, several bit layer images of the current frame marked image are obtained, the possibility of each bit layer image of the current frame marked image being a key bit layer image is obtained, the key bit layer image of the current frame marked image is obtained, and the difference pixels in the key bit layer image of the current frame marked image and other positive bit layer images are clustered respectively; the possibility of each clustering cluster in the key bit layer image of the current frame marked image being a reference cluster is obtained, and then the reference cluster of the current frame image is obtained; the reference block of the current frame image is obtained according to the reference cluster of the current frame image.
[0066] S003. Slide the reference block of the current frame image with the preset sliding window to obtain each sliding window, obtain the filtering weight of each sliding window, filter the reference block of the current frame image, obtain the denoised image of the current frame, and obtain the denoised image of each frame.
[0067] It should be noted that, after obtaining the reference block of the current frame image, it is necessary to use a non-local mean filtering algorithm to filter the reference block of the current frame image. The non-local mean filtering algorithm is a filtering technology commonly used in image processing. It is different from the traditional local mean filtering. The non-local mean filtering algorithm uses global information and needs to combine the information of the entire image to perform denoising operations on the reference block. Therefore, the present invention presets a sliding window to slide the reference block of the current frame image to obtain each sliding window, and filters the center pixel point corresponding to each sliding window, thereby filtering the reference block of the current frame image. Therefore, it is first necessary to combine the degree of noise influence of each sliding window to adapt the filtering weight of each sliding window, so as to facilitate the subsequent filtering of the center pixel point corresponding to each sliding window according to the filtering weight of each sliding window, so that the filtering result is more accurate.
[0068] In an embodiment of the present invention, the size of the sliding window is preset to be N×N, and each pixel point of the reference block of the current frame image is taken as the center pixel point to construct a sliding window of size N×N to obtain each sliding window. In an embodiment of the present invention, N=5 is preset. In other embodiments, the implementer may set the value of N according to the specific implementation situation. It should be noted that during the construction of the sliding window, there is a situation where the pixel point is close to the boundary of the reference block of the current frame image, which makes it impossible to obtain a complete sliding window. In this embodiment, the sliding window is completed by quadratic linear interpolation.
[0069] Get the filter weights for each sliding window:
[0070]
[0071] In the formula, α c Represents the filter weight of the cth sliding window; Represents the average gray value of the pixel in the cth sliding window; represents the average grayscale value of the pixels in the reference block of the current frame image; S represents the grayscale variance of all pixels in the reference block of the current frame image; V represents the grayscale variance of the pixels in the current frame image except the pixels in the reference block; S c represents the grayscale variance of the pixels in the c-th sliding window, reflecting the degree of noise influence on the c-th sliding window; || represents the absolute value function; |SV| represents the absolute value of the difference between the grayscale variance of all pixels in the reference block and the grayscale variance of pixels other than the reference block. The larger its value is, the greater the degree of noise influence on the pixels in the reference block is, so the filtering weight of the c-th sliding window is larger; Represents the mean difference between the average grayscale value of the pixels in the c-th sliding window and the average grayscale value of the pixels in each other sliding window. The grayscale feature of the c-th sliding window is obtained according to the grayscale difference of different sliding windows. The larger the difference, the greater the influence of noise on the c-th sliding window. Combined with the value of |SV|, when the pixels in the reference block are greatly affected by noise and when the c-th sliding window is more affected by noise, the greater the filtering weight of the c-th sliding window. norm() represents the normalization function, which uses a linear normalization function. The normalization object is all sliding windows. The value of , and similarly obtain the filter weight of each sliding window.
[0072] It should be noted that the filtering weight of each sliding window is obtained, that is, when the filtering weight is larger, it is more necessary to filter the central pixel point of the sliding window, and at this time, more attention should be paid to the average grayscale value of all pixels in the reference block of the current frame image, that is, when the filtering weight is smaller, it is less necessary to filter the central pixel point of the sliding window, and at this time, more attention should be paid to the grayscale value of the central pixel point of the sliding window. Therefore, in an embodiment of the present invention, according to the filtering weight of each sliding window and the average grayscale value of the reference block of the current frame image, the central pixel point of each sliding window is filtered, thereby completing the filtering of the reference block of the current frame image.
[0073] In an embodiment of the present invention, the filtering result of the central pixel of each sliding window is obtained:
[0074]
[0075] In the formula, B c Represents the filtering result of the central pixel of the c-th sliding window; α c Represents the filter weight of the cth sliding window; Represents the average gray value of the pixels in the reference block of the current frame image; L c Represents the grayscale value of the center pixel of the c-th sliding window.
[0076] The current frame image after the reference block of the current frame image is denoised is recorded as the current frame denoised image.
[0077] Similarly, obtain the denoised image for each frame.
[0078] At this point, the preset sliding window slides the reference block of the current frame image to obtain each sliding window, obtain the filtering weight of each sliding window to filter the reference block of the current frame image, obtain the denoised image of the current frame, and obtain the denoised image of each frame.
[0079] S004. Obtain key points in each denoised image frame, obtain a judgment value of a person falling in each denoised image frame according to the movement of key points in adjacent denoised images, and judge whether a person has fallen according to the judgment value of a person falling in each denoised image frame.
[0080] It should be noted that in step S003, denoising is performed on each frame of the image. Therefore, it is possible to determine whether a person falls in the surveillance video based on each frame of the image after denoising. When using visual technology to determine whether a person falls, it is mainly through obtaining key points on the person and determining whether the person falls based on the movement of the key points. That is, the key points in each denoised image frame are obtained, and whether a person falls is determined by the movement of the key points between adjacent denoised images.
[0081] In the embodiment of the present invention, the SIFT key point detection algorithm is used to perform key point detection on each denoised image frame, and the key points that are displaced in each denoised image frame are obtained and recorded as the key points in each denoised image frame.
[0082] Obtain the displacement direction of each key point in each denoised image frame: record the displacement direction of the oth key point in the f-th denoised image frame compared to the oth key point in the f-1th denoised image frame as the displacement direction of the oth key point in the f-th denoised image frame; record the displacement distance of the oth key point in the f-th denoised image frame compared to the oth key point in the f-1th denoised image frame as the displacement distance of the oth key point in the f-th denoised image frame;
[0083] Get the judgment value of the person falling in each frame of the denoised image:
[0084]
[0085] In the formula, γ f represents the judgment value of a person falling in the f-th frame of the denoised image; d fo represents the displacement distance of the oth key point in the fth frame denoised image; θ fo represents the displacement direction of the oth key point in the fth frame denoised image; d f+1,o represents the displacement distance of the oth key point in the f+1th frame denoised image; θ f+1,o represents the displacement direction of the oth key point in the f+1th frame denoised image; t f represents the acquisition time of the f-th frame image; t f+1 represents the acquisition time of the f+1th frame image; || represents the absolute value symbol; U represents the number of key points in each frame of the denoised image; γ f The larger the value of is, the greater the movement of the key points between the f-th frame denoised image and the f+1-th frame denoised image. In this case, the judgment value of a person falling in the f-th frame denoised image is larger, and the person is more likely to fall.
[0086] A fall judgment threshold T1 is preset. When the judgment value of a person falling in any frame of image is greater than or equal to the fall judgment threshold, it is considered that a person falling has occurred in the monitoring video. When the judgment value of a person falling in each frame of image is less than the fall judgment threshold, it is considered that no person falling has occurred in the monitoring video. In an embodiment of the present invention, the fall judgment threshold T1 is preset to 0.34. In other embodiments, the implementer may set the fall judgment threshold according to the specific implementation situation.
[0087] At this point, the judgment value of a person falling in each frame of the denoised image is obtained, and the person falling is judged according to the judgment value of a person falling in each frame of the denoised image.
[0088] This embodiment provides a binocular camera-based human fall detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements steps S001 to S004 when executing the computer program.
[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for detecting a person falling down based on a binocular camera, characterized in that: The method comprises the following steps: Collect each frame of image; Record any frame image as the current frame image, and obtain the current frame marked image according to the current frame image and the next frame image of the current frame image; split the current frame marked image to obtain several bit layer images of the current frame marked image and the difference pixels in each bit layer image of the current frame marked image; according to the aggregation and number of the difference pixels in the bit layer image, obtain the possibility that each bit layer image of the current frame marked image is a key bit layer image; according to the possibility that each bit layer image of the current frame marked image is a key bit layer image, obtain the key bit layer of the current frame marked image. Image; cluster the difference pixels in the key bit layer image of the current frame marked image and other bit layer images respectively to obtain several clustering clusters of the key bit layer image of the current frame marked image and several clustering clusters of each other bit layer image of the current frame marked image; obtain the possibility degree of each clustering cluster in the key bit layer image of the current frame marked image being a reference cluster according to the several clustering clusters of the key bit layer image and the several clustering clusters of the other bit layer images; obtain the reference block of the current frame image according to the possibility degree of each clustering cluster in the key bit layer image of the current frame marked image being a reference cluster; A sliding window is constructed with each pixel of the reference block of the current frame image as the central pixel to obtain each sliding window; a filtering weight of each sliding window is obtained according to the average gray value of the pixel in the reference block of the current frame image and the average gray value of the pixel in each sliding window; a filtering result of the central pixel of each sliding window is obtained according to the filtering weight of each sliding window and the average gray value of the pixel in the reference block of the current frame image to obtain a denoised image of the current frame; Obtain each frame of denoised image; obtain key points in each frame of denoised image; obtain a judgment value of a person falling in each frame of denoised image according to the movement of key points in adjacent frames of image; and judge the person falling according to the judgment value of the person falling in each frame of denoised image; The method of obtaining the possibility that each bit layer image of the current frame marked image is a key bit layer image according to the aggregation and number of difference pixel points in the bit layer image; obtaining the key bit layer image of the current frame marked image according to the possibility that each bit layer image of the current frame marked image is a key bit layer image, includes the following specific steps: In the formula, p t Represents the possibility that the t-th bit layer image of the current frame marked image is a key bit layer image; N t represents the number of difference pixels in the tth bit layer image of the current frame mark image; M represents the number of difference pixels in the current frame mark image; d t (i,j) represents the Euclidean distance between the i-th difference pixel and the j-th difference pixel in the t-th bit layer image in the current frame mark image; exp() represents an exponential function with a natural constant as the degree; The method of obtaining the possibility that each cluster in the key bit layer image of the current frame marked image is a reference cluster according to the clusters of the key bit layer image and the clusters of other bit layer images includes the following specific steps: Return each cluster in the key bit layer image of the current frame mark image to the current frame image to obtain a number of clusters of the current frame image; In the formula, q g Represents the probability that the gth cluster in the key bit layer image of the current frame mark image is the reference cluster; n g represents the number of difference pixels in the g-th cluster in the key bit layer image of the current frame mark image; A represents the number of difference pixels in the key bit layer image of the current frame mark image; Represents the average gray value of the difference pixel points of the g-th cluster in the current frame image; represents the average gray value of the pixels in the current frame image; || represents the absolute value symbol; d(g,hx) represents the distance between the centroid of the g-th cluster in the key bit layer image of the current frame marked image and the centroid of the x-th cluster in the other h-th bit layer image of the current frame marked image; H represents the number of other bit layer images in the current frame marked image; Z h Represents the number of clusters in the hth bit-layer image of the current frame mark image; G represents the sum of the number of clusters in all other bit-layer images of the current frame mark image; norm() represents the linear normalization function; exp() represents the exponential function with a natural constant as the base.
2. The method for detecting a person falling down based on a binocular camera according to claim 1, characterized in that: The method of obtaining a current frame marked image according to the current frame image and the next frame image of the current frame image; splitting the current frame marked image to obtain a plurality of bit layer images of the current frame marked image and difference pixels in each bit layer image of the current frame marked image comprises the following specific steps: Using a frame difference method, performing frame difference between the next frame image of the current frame image and the current frame image, obtaining difference pixel points between the current frame image and the next frame image of the current frame image, marking the difference pixel points between the current frame image and the next frame image of the current frame image in the current frame image, and obtaining a current frame marked image; The current frame marked image is split using a bit layering algorithm to obtain a plurality of bit layer images of the current frame marked image and difference pixels in each bit layer image of the current frame marked image.
3. The method for detecting a person falling down based on a binocular camera according to claim 1, characterized in that: The step of clustering the difference pixels in the key bit layer image of the current frame mark image and other bit layer images respectively to obtain a plurality of clustering clusters of the key bit layer image of the current frame mark image and a plurality of clustering clusters of each other bit layer image of the current frame mark image includes the following specific steps: The number of clusters β and the cluster radius r are preset, and the DBSCAN density clustering algorithm is used based on β and r to cluster the difference pixels in the key bit layer image of the current frame marked image and the difference pixels in other bit layer images respectively, to obtain several clustering clusters of the key bit layer image of the current frame marked image and several clustering clusters of each other bit layer image of the current frame marked image.
4. The method for detecting a person falling down based on a binocular camera according to claim 1, characterized in that: The step of obtaining the reference block of the current frame image according to the possibility that each cluster in the key bit layer image of the current frame marked image is a reference cluster includes the following specific steps: Among the possible degrees that each cluster in the key bit layer image of the current frame marked image is a reference cluster, the cluster corresponding to the maximum possible degree is used as the reference cluster in the key bit layer image of the current frame marked image, and the corresponding reference cluster of the current frame image is obtained. The minimum enclosing rectangle of the reference cluster of the current frame image is obtained, and the area where the minimum enclosing rectangle is located is used as the reference block of the current frame image.
5. The method for detecting a person falling down based on a binocular camera according to claim 1, characterized in that: The specific steps of obtaining the filtering weight of each sliding window according to the average grayscale value of the pixels in the reference block of the current frame image and the average grayscale value of the pixels in each sliding window are as follows: In the formula, α c Represents the filter weight of the cth sliding window; Represents the average gray value of the pixel in the cth sliding window; Represents the average gray value of pixels in the reference block; S represents the grayscale variance of all pixels in the reference block; V represents the grayscale variance of pixels in the current frame image except the pixels in the reference block; S c represents the grayscale variance of the pixel in the cth sliding window; || represents the absolute value function; norm() represents the linear normalization function.
6. The method for detecting a person falling down based on a binocular camera according to claim 1, characterized in that: The specific steps of obtaining the filtering result of the central pixel of each sliding window according to the filtering weight of each sliding window and the average gray value of the pixel in the reference block of the current frame image to obtain the denoised image of the current frame are as follows: In the formula, B c Represents the filtering result of the central pixel of the c-th sliding window; α c Represents the filter weight of the cth sliding window; Represents the average gray value of the pixels in the reference block of the current frame image; L c Represents the grayscale value of the central pixel of the c-th sliding window; The current frame image after the reference block of the current frame image is denoised is recorded as the current frame denoised image.
7. The method for detecting a person falling down based on a binocular camera according to claim 1, characterized in that: The method of obtaining key points in each denoised image frame and obtaining a judgment value of a person falling in each denoised image frame according to the movement of key points in adjacent image frames includes the following specific steps: Use SIFT key point detection algorithm to detect key points of each denoised image frame, obtain the key points that have shifted in each denoised image frame, and record them as key points in each denoised image frame; The displacement direction of the oth key point in the f-th frame denoised image compared to the oth key point in the f-1th frame denoised image is recorded as the displacement direction of the oth key point in the f-th frame denoised image; the displacement distance of the oth key point in the f-th frame denoised image compared to the oth key point in the f-1th frame denoised image is recorded as the displacement distance of the oth key point in the f-th frame denoised image; In the formula, γ f represents the judgment value of a person falling in the f-th frame of the denoised image; d fo represents the displacement distance of the oth key point in the fth frame denoised image; θ fo represents the displacement direction of the oth key point in the fth frame denoised image; d f+1,o represents the displacement distance of the oth key point in the f+1th frame denoised image; θ f+1,o represents the displacement direction of the oth key point in the f+1th frame denoised image; t f represents the acquisition time of the f-th frame image; t f+1 represents the acquisition time of the f+1th frame image; || represents the absolute value symbol; U represents the number of key points in each frame of the denoised image.
8. A person fall detection system based on a binocular camera, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a method for detecting a person falling based on a binocular camera as described in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Color image segmentation method and device
CN107274424A
Multi-target image dynamic capturing method and related device
CN116385316A
Early warning method and system for falling in elevator
CN117284894A
Bit stream structure for compressed point cloud data
US20200021847A1
