A method for detecting the opening and closing state of a door

Abstract

The present invention relates to a method for detecting the opening and closing state of a door, and belongs to the field of computer vision technology. A grayscale image of a front view image of a door and a wall on which the door is located in a closed state is used as a reference image, and a grayscale image of a current moment image with the same size and the same viewing angle as the reference image is combined to obtain a frame difference image; the frame difference image is threshold filtered according to an image threshold adjusted by the difference between the brightness characteristic values ​​of the current moment image and the reference image and a set maximum value of the image threshold; edge feature extraction is performed on the threshold filtered frame difference image to obtain an edge feature image, and a Hough transform is performed on the edge feature image with the straight line where the left and right boundaries of the door are located or a line parallel to the straight line as the x-axis, and the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or a line parallel to the straight line is obtained according to the angle of each straight line in the edge feature image in a polar coordinate system, and the opening and closing state of the door at the current moment is judged according to the maximum value.

Description

Technical Field

[0001] The invention relates to a method for detecting the opening and closing state of a door, and belongs to the technical field of computer vision. Background Art

[0002] With the widespread deployment of data centers, real-time online monitoring of sensitive areas involving information security, such as computer rooms and offices, is becoming increasingly important. Initially, the method of arranging dedicated personnel to guard sensitive areas can ensure the security of these sensitive areas, but the labor and management costs are high; subsequently, surveillance cameras were used to monitor the opening and closing status of doors in sensitive areas. Although the labor cost was reduced, manual monitoring is still required. In addition, the transmission and processing of massive amounts of monitoring data will consume a lot of time and traffic costs.

[0003] There is an existing door opening and closing detection algorithm based on convolutional neural network that can detect the opening and closing status of the door. Compared with the manual monitoring method, it is more convenient, saves manpower and material resources, and has good robustness, but it has high requirements on the generalization of the training data set, and it takes time to collect and label the task data; in addition, the detection method based on neural network often has a large amount of calculation and high storage cost, resulting in poor real-time performance; there is also a traditional computer vision processing method, which performs relatively well on smaller data sets and has better real-time performance than the door opening and closing detection algorithm based on convolutional neural network, but this method is easily affected by environmental factors such as light, resulting in low detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting the opening and closing state of a door, so as to solve the problem that the current traditional computer vision processing method is easily affected by environmental factors such as lighting, resulting in low detection accuracy.

[0005] In order to solve the above technical problems, the present invention provides a method for detecting the opening and closing state of a door, the method comprising: taking a grayscale image of a front view image of a door and a wall on which the door is located when the door is closed as a reference image, combining a grayscale image of a current moment image with the same size and the same viewing angle as the reference image to obtain a frame difference image;

[0006] Adjusting the image threshold according to the difference between the brightness characteristic values ​​of the current image and the reference image and the set maximum value of the image threshold, and performing threshold filtering on the frame difference image according to the image threshold;

[0007] Edge feature extraction is performed on the frame difference image after threshold filtering to obtain an edge feature map, and Hough transform is performed on the edge feature map with the straight line where the left and right boundaries of the door are located or the parallel line of the straight line as the x-axis to obtain the angle of each straight line in the edge feature map in the polar coordinate system; according to the angle of each straight line in the polar coordinate system, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is obtained, and the opening and closing state of the door at the current moment is judged according to the maximum value.

[0008] Further, the manner of adjusting the image threshold by the difference and the set maximum value of the image threshold includes:

[0009]

[0010] In the formula, threshold is the image threshold, L F L is the brightness characteristic value of the picture at the current moment; oT is the brightness characteristic value of the reference image, max t is the maximum value set for the image threshold, bias is the bias parameter, and k is the scaling factor.

[0011] Furthermore, the method of judging the opening and closing state of the door at the current moment according to the maximum value of the angles between each straight line and the straight lines where the left and right boundaries of the door are located or the parallel lines of the straight lines includes:

[0012] If the maximum value of the angles between each straight line and the x-axis is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, it is determined that the door is in the open state at the current moment; otherwise, it is determined that the door is in the closed state at the current moment.

[0013] Furthermore, the method of obtaining the maximum value of the angles between each straight line and the straight lines where the left and right boundaries of the door are located or the parallel lines of the straight lines according to the angles of each straight line in the polar coordinate system includes:

[0014] theta d =max(thetas sub )-min(thetas sub )

[0015]

[0016] Where theta d Thetas is the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line. sub[i] is the angle between the ith line and the x-axis obtained by converting the angle of the ith line in the polar coordinate system into the angle between the ith line and the x-axis itself. n is the number of lines. thetas[i] is the angle of the ith line in the polar coordinate system. max and min are operators for taking the maximum and minimum values, respectively.

[0017] Furthermore, the edge feature extraction of the frame difference image after the threshold filtering to obtain the edge feature image includes:

[0018] After obtaining the gradient amplitude and gradient direction of each pixel in the frame difference image after threshold filtering, non-maximum suppression processing is performed to obtain an edge feature map.

[0019] Furthermore, the non-maximum suppression method includes:

[0020] S1: Determine whether the gradient amplitude of each pixel point is greater than the maximum value of the gradient amplitudes in the gradient direction. If so, set the pixel value of the point to zero; otherwise, set the pixel value of the pixel point to the maximum pixel value; then proceed to step S2;

[0021] S2: If the gradient amplitude of the pixel point with the maximum pixel value is greater than the maximum set gradient threshold, the pixel value of the pixel point remains unchanged; if the gradient amplitude of the pixel point with the maximum pixel value is less than the maximum set gradient threshold and greater than the minimum set gradient threshold, the pixel value of the pixel point is set to a first pixel fixed value, which is less than the maximum pixel value and greater than 0; if the gradient amplitude of the pixel point with the maximum pixel value is less than the minimum set gradient threshold, the pixel value of the point is set to zero; then proceed to step S3;

[0022] S3: If the pixel value of the pixel point at this time is the first pixel value and the pixel value of the adjacent pixel point diagonally opposite to the pixel point is the maximum pixel value, or the pixel value of the pixel point at this time is the maximum pixel value, then the pixel value of the pixel point is set to the maximum pixel value, otherwise the pixel value of the pixel point is set to 0.

[0023] Furthermore, the method of performing threshold filtering on the frame difference image according to the image threshold includes: if the pixel value of a pixel point of the frame difference image is less than the image threshold, the pixel value of the pixel point is set to zero; otherwise, the pixel value of the pixel point is set to the maximum pixel value.

[0024] Furthermore, the grayscale image of the front view image of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain the frame difference image in the following manner:

[0025] A frame difference image is obtained according to the absolute value of pixel-by-pixel difference between the reference image and the grayscale image of the current moment image.

[0026] Furthermore, the reference image is updated according to the judgment result of the door opening and closing state, and the method of updating the reference image includes: if it is determined that the door is in a closed state at the current moment, the grayscale image of the picture at the current moment is saved in the reference image set according to the time period to which the current moment belongs, as an updated reference image.

[0027] Beneficial effect: The present invention provides a new method for detecting the opening and closing state of a door. The method first uses a grayscale image of a front view image of a door and the wall on which it is located when the door is closed as a reference image, and combines the grayscale image of a current moment image with the same size and viewing angle as the reference image to obtain a frame difference image; threshold filtering is performed on the frame difference image according to the image threshold, and the pixel changes caused by the light conditions in the environment can be filtered out by threshold filtering, thereby reducing the influence of factors such as the light in the environment on the detection of the opening and closing state of the door; before threshold filtering is performed on the frame difference image, a difference between the brightness feature values ​​of the current moment image and the reference image and the image The image threshold is adjusted by setting the maximum value of the threshold, so that the frame difference image after the threshold filtering can adapt to the change of the ambient brightness; secondly, the edge feature of the frame difference image after the threshold filtering is extracted to obtain an edge feature map, and the edge feature map is subjected to Hough transformation with the straight line where the left and right boundaries of the door are located or the parallel lines of the straight lines where the left and right boundaries of the door are located as the x-axis to obtain the angle of each straight line in the edge feature map in the polar coordinate system; according to the angle of each straight line in the polar coordinate system, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel lines of the straight line is obtained, and the opening and closing state of the door at the current moment is judged according to the maximum value. Among them, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is equivalent to the size of the angle between the upper boundary of the door and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line, or the size of the angle between the lower boundary and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line. Judging the opening and closing state of the door at the current moment based on the maximum value can reduce the influence of other straight lines (such as the straight lines of the pattern on the door, the straight lines on the sides of the door, etc.) on judging the opening and closing state of the door. This method only needs a grayscale image of the front view image of the door and the wall on which it is located when the door is closed as a reference image and a current moment image with the same size and the same perspective as the reference image to perform door opening and closing detection, that is, door opening and closing state detection is realized when the data set is small. While retaining the key difference information of the door opening and closing state, the interference caused by environmental changes is reduced, so that the accuracy of the door opening and closing state detection is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the door opening and closing detection method in the embodiment of the method for detecting the door opening and closing state of the present invention. DETAILED DESCRIPTION

[0029] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0030] Door opening and closing state detection method embodiment

[0031] The present embodiment provides a technical solution for a method for detecting the opening and closing state of a door. The method performs threshold filtering on a frame difference image obtained based on a grayscale image of a reference image in a closed door state and a current image. Before the threshold filtering, the image threshold is adjusted based on the difference between the brightness feature values ​​of the current image and the reference image and a set maximum value of the image threshold. Edge features are then extracted from the frame difference image after the threshold filtering. The opening and closing state of the door at the current moment is judged by the maximum value of the angles between each straight line obtained based on the angle of each straight line in the edge feature image in a polar coordinate system and the straight lines where the left and right boundaries of the door are located or the parallel lines of the straight lines. While retaining key difference information on the opening and closing state of the door, the interference caused by environmental changes is reduced, thereby improving the accuracy of detection.

[0032] The method specifically includes:

[0033] The grayscale image of the front view of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain a frame difference image;

[0034] The image threshold is adjusted according to the difference between the brightness characteristic values ​​of the current image and the reference image and the set maximum value of the image threshold, and the frame difference image is threshold filtered according to the image threshold;

[0035] The edge feature extraction is performed on the frame difference image after threshold filtering to obtain an edge feature image, and the edge feature image is subjected to Hough transformation with the straight line where the left and right boundaries of the door are located or the parallel line where the left and right boundaries of the door are located as the x-axis to obtain the angle of each straight line in the edge feature image in the polar coordinate system; according to the angle of each straight line in the polar coordinate system, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is obtained, and the opening and closing state of the door at the current moment is judged according to the maximum value. This method only needs the grayscale image of the front view image of the door and the wall on which it is located in the closed state as the reference image and the current moment image with the same size and the same perspective as the reference image to perform door opening and closing detection, that is, the door opening and closing state detection is realized when the data set is small, while retaining the key difference information of the door opening and closing state, reducing the interference caused by environmental changes, so that the accuracy of the door opening and closing state detection is higher.

[0036] The flow chart of the door opening and closing detection method in this embodiment is as follows Figure 1 As shown below, combined Figure 1The detection method of the door opening and closing state is described in detail:

[0037] The grayscale image of the front view picture of the door and the wall on which it is located when the door is closed is used as a reference image, and the frame difference image is obtained by combining the grayscale image of the current moment picture with the same size and the same viewing angle as the reference image; in this embodiment, the method of obtaining the reference image is: according to the current moment, the reference image corresponding to the current moment is selected from the reference image set, and the method of obtaining the reference image set is: the video of the front view direction of the door and the wall on which it is located when the door is closed is collected at a set time interval to obtain the front view picture of the door and the wall on which it is located when the door is closed, and then the collected picture is converted into a grayscale image and stored in the reference image list I oT =[I o0 ,I o1 ,...,I os ](S refers to the total number of reference pictures minus 1).

[0038] In this embodiment, the grayscale image of the front view of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain the frame difference image in a specific manner: the frame difference image is obtained according to the absolute value of the pixel-by-pixel difference between the reference image and the grayscale image of the current moment image, which can avoid the situation where only the pixel-by-pixel difference leads to the loss of some door frame details and key information in the obtained frame difference image, so that the accuracy of the door opening and closing state detection is higher; the calculation formula for obtaining the pixel value of the frame difference image is:

[0039] I sub (i,j,k)=|Frame(i,j,k)-I oT (i,j,k)|,i∈[0,h],j∈[0,w],k∈[0,c]

[0040] In the formula, I sub (i, j, k) is the pixel value of the frame difference image, Frame(i, j, k) is the pixel value of the current image with the same size and the same viewing angle as the reference image, I oT (i, j, k) is the pixel value of the grayscale image of the front view of the door and the wall on which it is located when the door is closed (that is, the pixel value of the reference image corresponding to the current moment); h is the height, w is the width, c is the number of channels, and T is the sequence number of the reference image corresponding to the current moment in the reference image list.

[0041] In other embodiments, only the grayscale image of the reference image and the current image may be subtracted pixel by pixel to obtain a frame difference image, which may result in the loss of some door frame details and key information in the obtained frame difference image, thereby affecting the accuracy of the door opening and closing state detection.

[0042] In this embodiment, the size of the reference image and the current moment picture with the same size and viewing angle as the reference image is 720 in height h, 1280 in width w, and 3 in number of channels c; the image threshold is adjusted according to the difference between the brightness feature values ​​of the current moment picture with the same size and viewing angle as the reference image and the reference image and the set maximum value of the image threshold, so that the frame difference image after threshold filtering can adapt to changes in light conditions in the environment.

[0043] In this embodiment, the image threshold is adjusted by the difference between the brightness feature values ​​of the current image with the same size and the same viewing angle as the reference image and the reference image and the set maximum value of the image threshold as follows:

[0044]

[0045] In the formula, threshold is the image threshold, L F is the brightness characteristic value of the image at the current moment; L oT is the brightness characteristic value of the reference image, max t is the maximum value of the image threshold, bias is the bias parameter, and k is the scaling factor. In this embodiment, k, bias, max t Usually, they are 1.2, 40, and 210 respectively. Among them, the brightness characteristic value L of the current image is obtained F The formula is:

[0046] L F =mean[Frame()]

[0047] Where, L F is the brightness feature value of the current image, mean refers to the mean symbol, and Frame() refers to the pixel value of the current image;

[0048] Get the brightness eigenvalue L of the reference image oT The formula is:

[0049] L oT =mean[I oT ()]

[0050] Where, L oT is the brightness feature value of the reference image, mean refers to the mean symbol, I oT () refers to the pixel value of the reference image.

[0051] The frame difference image is subjected to threshold filtering according to the image threshold to perform binarization processing. The threshold filtering processing can filter out some pixel changes caused by environmental factors.

[0052] In this embodiment, the method of performing threshold filtering on the frame difference image according to the image threshold to perform binarization processing is specifically as follows: if the pixel value of a certain pixel point of the frame difference image is less than the image threshold, the pixel value of the pixel point is set to zero; otherwise, the pixel value of the pixel point is set to the maximum pixel value. The formula for performing threshold filtering on the frame difference image according to the image threshold is:

[0053]

[0054] Where Thd(i,j,k) is the pixel value of the frame difference image after threshold filtering, 255 is the maximum pixel value, and I sub (i, j, k) is the pixel value of the frame difference image, threshold is the image threshold, h is the height, w is the width, and c is the number of channels.

[0055] Perform edge feature extraction on the frame difference image after threshold filtering to obtain an edge feature image;

[0056] In this embodiment, the method of extracting edge features from the frame difference image after threshold filtering to obtain the edge feature image is specifically as follows:

[0057] After obtaining the gradient amplitude and gradient direction of each pixel in the frame difference image according to the frame difference image after threshold filtering, non-maximum suppression processing is performed to obtain an edge feature map.

[0058] The formula for calculating the gradient amplitude of each pixel in the frame difference image is:

[0059]

[0060] Where G(i,j,k) is the gradient amplitude of the (i,j,k)th pixel, G x (i,j,k) and G x All refer to the horizontal gradient amplitude of the (i, j, k)th pixel point, G y (i,j,k) and G y is the vertical gradient amplitude of the (i,j,k)th pixel, is the partial derivative of the pixel value Thd of the frame difference image after threshold filtering in the horizontal direction; is the partial derivative of the pixel value Thd of the frame difference image after threshold filtering in the vertical direction;

[0061] The formula for calculating the gradient direction of each pixel in the frame difference map is:

[0062] θ(i,j,k)=atan2(G y (i,j,k),G x (i,j,k))

[0063] θ0=θ(i0,j0,k0),i0∈[0,h],j0∈[0,w],k0∈[0,c]

[0064] Where θ(i,j,k) is the gradient direction of the (i,j,k)th pixel; G x (i,j,k) is the horizontal gradient amplitude of the (i,j,k)th pixel, G y (i,j,k) is the vertical gradient amplitude of the (i,j,k)th pixel, θ0 is the gradient direction of the (i0,j0,k0)th pixel;

[0065] In this embodiment, the non-maximum suppression method is specifically as follows:

[0066] S1: Determine whether the gradient amplitude of each pixel point is greater than the maximum value of the gradient amplitude in the gradient direction. If it is less, set the pixel value of the point to zero; otherwise, set the pixel value of the pixel point to the maximum pixel value, that is:

[0067]

[0068] Where Edge0(i0,j0) is the pixel value of the (i0,j0)th pixel after the first non-maximum suppression, G(i0,j0,k0) is the gradient amplitude of the (i0,j0,k0)th pixel, is the gradient amplitude of the (i, j, k)th pixel in the gradient direction θ0, max is the maximum value operator, and 255 is the maximum pixel value;

[0069] Then proceed to step S2;

[0070] S2: If the gradient amplitude of the pixel point with the maximum pixel value is greater than the maximum set gradient threshold, the pixel value of the pixel point remains unchanged; if the gradient amplitude of the pixel point with the maximum pixel value is less than the maximum set gradient threshold and greater than the minimum set gradient threshold, the pixel value of the pixel point is set to the first pixel fixed value, which is less than the maximum pixel value and greater than 0; if the gradient amplitude of the pixel point with the maximum pixel value is less than the minimum set gradient threshold, the pixel value of the point is set to zero, that is:

[0071]

[0072] Where Edge1(i,j) is the pixel value of the (i,j)th pixel after the second non-maximum suppression, Edge0(i,j) is the pixel value of the (i,j)th pixel after the first non-maximum suppression, G(i,j,k) is the gradient amplitude of the (i,j,k)th pixel, Thd is max Set the maximum gradient threshold, Thd minis the minimum set gradient threshold, 255 is the maximum pixel value, and 125 is the first pixel value;

[0073] Then proceed to step S3;

[0074] S3: If the pixel value of the pixel point at this time is the first pixel value and the pixel value of the adjacent pixel point diagonally opposite to the pixel point is the maximum pixel value, or the pixel value of the pixel point at this time is the maximum pixel value, then the pixel value of the pixel point is set to the maximum pixel value, otherwise the pixel value of the pixel point is set to 0, that is:

[0075]

[0076] In the formula, Edge2(i,j) is the pixel value of the (i,j)th pixel after the third non-maximum suppression, Edge1(i,j) is the pixel value of the (i,j)th pixel after the second non-maximum suppression; Edge1(i±1,j±1) is the pixel value of the diagonally adjacent pixel of the (i,j)th pixel after the second non-maximum suppression, max is the maximum value operator; 255 is the maximum pixel value, and 125 is the first pixel constant. After three non-maximum suppressions, the noise in the edge feature map can be better removed, making the straight edges in the edge feature map clearer.

[0077] Performing Hough transformation on the edge feature map with the straight line where the left and right boundaries of the door are located or the parallel line where the left and right boundaries of the door are located as the x-axis to obtain the angle of each straight line in the edge feature map in the polar coordinate system;

[0078] The edge feature map is subjected to Hough transform with the straight line where the left and right boundaries of the door are located or the parallel line where the left and right boundaries of the door are located as the x-axis. The formula for obtaining the angle of each straight line in the edge feature map in the polar coordinate system is:

[0079] Rho,theta=Houghlines(Edge2(i,j)),i∈[0,h],j∈[0,w]

[0080] Where theta is the angle of each line in the polar coordinate system, Rho is the distance from the origin to the line along the vector perpendicular to the line in the polar coordinate system; Edge2(i,j) is the pixel value of the (i,j)th pixel after the third non-maximum suppression, and Houghlines refers to Hough transform;

[0081] According to the angle of each straight line in the polar coordinate system, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is obtained, and the opening and closing state of the door at the current moment is judged according to the maximum value. Among them, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is obtained, which is the difference between the maximum value of the angle between each straight line and the x-axis and the minimum value of the angle between each straight line and the x-axis. The maximum value of the angle between each straight line and the x-axis is equivalent to the size of the angle between the upper boundary of the door and the x-axis or the size of the angle between the lower boundary of the door and the x-axis. The minimum value of the angle between each straight line and the x-axis is equivalent to the angle between the straight line where the left and right boundaries of the door are located or the parallel line of the straight line and the x-axis. Theoretically, the angle between the straight line where the left and right boundaries of the door are located or the parallel line of the straight line and the x-axis should be 0, so each straight line is divided into The maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is equivalent to the size of the angle between the upper boundary of the door and the x-axis or the size of the angle between the lower boundary of the door and the x-axis. However, when the image is distorted due to reasons such as incorrect viewing angle, the angle between the straight line where the left and right boundaries of the door are located or the parallel line of the straight line except the x-axis and the selected x-axis may not be 0; therefore, using the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line to judge the opening and closing state of the door at the current moment can avoid the influence of other interfering straight lines (such as straight lines of patterns on the door, straight lines on the sides of the door, etc.) and the distortion of the image on the judgment of the opening and closing state of the door. Errors caused by the distortion of the image

[0082] In this embodiment, the method of obtaining the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line according to the angle of each straight line in the polar coordinate system includes:

[0083] theta d =max(thetas sub )-min(thetas sub )

[0084]

[0085] Where theta d That is, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line, thetas sub [i] is the angle between the line and the x-axis obtained by converting the angle of the line where the x-axis is located in the polar coordinate system into the angle between the line where the x-axis is located and the x-axis itself (thetas sub[i] is actually equivalent to the angle between the line and the x-axis), n is the number of lines, thetas[i] is the angle of the ith line in the polar coordinate system; max and min are operators for taking the maximum and minimum values ​​respectively.

[0086] Methods for judging the opening and closing state of the door at the current moment according to the maximum value of the angles between each straight line and the x-axis include:

[0087] If the maximum value of the angle between each straight line and the x-axis is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, it is determined that the door is in the open state at the current moment; otherwise, it is determined that the door is in the closed state at the current moment:

[0088]

[0089] In the formula, Function(theta d ) is the door opening and closing state at the current moment, Thd d To set the angle range; in this embodiment, the angle range Thd is set d is [0.5, 1.571], the upper limit of the angle is set to 1.571 (i.e., π / 2), and the lower limit of the angle is set to 0.5 (i.e., a number slightly larger than 0 and smaller than π / 2).

[0090] In this embodiment, when it is determined that the door is in an open state at the current moment, the abnormal state video and picture are saved and an alarm message is issued.

[0091] In this embodiment, the reference image is updated based on the judgment result of the door opening or closing state. The specific method of updating the reference image is as follows: if it is determined that the door is in a closed state at the current moment, the grayscale image of the picture at the current moment is saved in the reference image set according to the time period to which the current moment belongs, as an updated reference image.

[0092] According to the positive alarm rate comparison table of the door opening and closing state detection method of this embodiment, the traditional visual tripwire intrusion and area intrusion method (traditional visual processing method) widely used by cameras, and the YOLOv2 neural network algorithm in multiple environments with different light and dark, the positive alarm rate comparison table of the door opening and closing state detection in the pictures with the same input picture size of (720, 1280, 3) is shown in Table 1, and the inference delay (ms) comparison table of the door opening and closing state detection is shown in Table 2;

[0093] Table 1

[0094]

[0095] Table 2

[0096]

[0097] It can be seen from Table 1 and Table 2 that the door opening and closing detection method in this embodiment is used to detect the door opening and closing state. Under different light and disturbance conditions, compared with traditional computer vision, the detection rate of positive alarm is increased by 5%-13%. Overall, the detection accuracy is higher and the number of false detections and missed detections is less. Compared with the neural network model algorithm, the detection rate of positive alarm is slightly lower by 2%-5%, but the detection speed of the door opening and closing detection method in this embodiment is faster than that of the neural network model algorithm, indicating that it has good real-time performance and low computational cost.

[0098] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, but do not constitute a limitation to the present invention.

Claims

1. A method for detecting the opening and closing state of a door, characterized in that: include: The grayscale image of the front view of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain a frame difference image; Adjusting the image threshold according to the difference between the brightness characteristic values ​​of the current image and the reference image and the set maximum value of the image threshold, and performing threshold filtering on the frame difference image according to the image threshold; Edge feature extraction is performed on the frame difference image after threshold filtering to obtain an edge feature map, and Hough transform is performed on the edge feature map with the straight line where the left and right boundaries of the door are located or the parallel line of the straight line as the x-axis to obtain the angle of each straight line in the edge feature map in the polar coordinate system; according to the angle of each straight line in the polar coordinate system, the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line is obtained, and the opening and closing state of the door at the current moment is judged according to the maximum value.

2. The method for detecting the door opening and closing state according to claim 1, characterized in that: The method of adjusting the image threshold by the difference and the set maximum value of the image threshold includes: In the formula, threshold is the image threshold, L F is the brightness characteristic value of the current moment image or the grayscale image of the current moment image; L oT is the brightness characteristic value of the reference image, max t is the maximum value set for the image threshold, bias is the bias parameter, and k is the scaling factor.

3. The method for detecting the door opening and closing state according to claim 1 or 2, characterized in that: Methods for determining the current opening and closing state of the door according to the maximum value of the angles between each straight line and the straight lines where the left and right boundaries of the door are located or the parallel lines of the straight lines include: If the maximum value of the angles between each straight line and the x-axis is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, it is determined that the door is in the open state at the current moment; otherwise, it is determined that the door is in the closed state at the current moment.

4. The method for detecting the door opening and closing state according to claim 3, characterized in that: Methods for obtaining the maximum value of the angles between each straight line and the straight lines where the left and right boundaries of the door are located or the parallel lines of the straight lines according to the angles of each straight line in the polar coordinate system include: theta d =max(theta sub )-min(thetas sub ) Where theta d Thetas is the maximum value of the angle between each straight line and the straight line where the left and right boundaries of the door are located or the parallel line of the straight line. sub [i] is the angle between the ith line and the x-axis obtained by converting the angle of the ith line in the polar coordinate system into the angle between the ith line and the x-axis itself. n is the number of lines. thetas[i] is the angle of the ith line in the polar coordinate system. max and min are operators for taking the maximum and minimum values, respectively.

5. The method for detecting the door opening and closing state according to claim 1 or 2, characterized in that: Methods for extracting edge features from the frame difference image after threshold filtering to obtain an edge feature image include: After obtaining the gradient amplitude and gradient direction of each pixel in the frame difference image after threshold filtering, non-maximum suppression processing is performed to obtain an edge feature map.

6. The method for detecting the door opening and closing state according to claim 5, characterized in that: The non-maximum suppression method includes: S1: Determine whether the gradient amplitude of each pixel point is greater than the maximum value of the gradient amplitudes in the gradient direction. If so, set the pixel value of the point to zero; otherwise, set the pixel value of the pixel point to the maximum pixel value; then proceed to step S2; S2: If the gradient amplitude of the pixel point with the maximum pixel value is greater than the maximum set gradient threshold, the pixel value of the pixel point remains unchanged; if the gradient amplitude of the pixel point with the maximum pixel value is less than the maximum set gradient threshold and greater than the minimum set gradient threshold, the pixel value of the pixel point is set to a first pixel fixed value, which is less than the maximum pixel value and greater than 0; if the gradient amplitude of the pixel point with the maximum pixel value is less than the minimum set gradient threshold, the pixel value of the point is set to zero; then proceed to step S3; S3: If the pixel value of the pixel point at this time is the first pixel value and the pixel value of the adjacent pixel point diagonally opposite to the pixel point is the maximum pixel value, or the pixel value of the pixel point at this time is the maximum pixel value, then the pixel value of the pixel point is set to the maximum pixel value, otherwise the pixel value of the pixel point is set to 0.

7. The method for detecting the door opening and closing state according to claim 5 or 6, characterized in that: The method of performing threshold filtering on the frame difference image according to the image threshold includes: if the pixel value of a pixel point of the frame difference image is less than the image threshold, the pixel value of the pixel point is set to zero; otherwise, the pixel value of the pixel point is set to the maximum pixel value.

8. The method for detecting the door opening and closing state according to claim 1 or 2, characterized in that: The method of using the grayscale image of the front view of the door and the wall on which it is located when the door is closed as a reference image and combining the grayscale image of the current moment image with the same size and the same viewing angle as the reference image to obtain the frame difference image includes: A frame difference image is obtained according to the absolute value of pixel-by-pixel difference between the reference image and the grayscale image of the current moment image.

9. The method for detecting the door opening and closing state according to claim 3, characterized in that: The reference image is updated according to the judgment result of the door opening and closing state, and the method of updating the reference image includes: if it is determined that the door is in a closed state at the current moment, the grayscale image of the current moment picture is saved in the reference image set according to the time period to which the current moment belongs, as an updated reference image.

Images