A method for detecting the opening and closing state of a door
By inputting the frame difference graph into the neural network model and combining geometric principles analysis, the problem of high data collection, high training cost and poor real-time performance of the gate opening and closing state detection algorithm based on the convolutional neural network is solved, and efficient, accurate and real-time gate opening and closing state detection is achieved.
Patent Information
- Application Number
- CN202411781890.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-04-04
AI Technical Summary
The gate opening and closing state detection algorithm based on convolutional neural network requires a large number of data sets, has high quality requirements for the data set, and the training model has a large complexity and parameters, resulting in high data set costs, high training costs and poor real-time performance.
By taking the grayscale image of the front view of the door and its wall in the closed state as the reference image, the grayscale image of the current time picture with the same size as the reference image and the same perspective as the reference image, the frame difference diagram is obtained, and the frame difference diagram is input to the trained neural network model, linear feature information is extracted, and the geometric principle is used to analyze and judge the opening and closing state of the door.
It reduces the amount of parameters of neural network models and the workload required for training, reduces the cost of data sets, improves the real-time and accuracy of detection, and ensures high scenario adaptability in different environments.
Smart Images
Figure CN119762431A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for detecting the opening and closing state of a door, and belongs to the technical field of computer vision. Background Art
[0002] With the widespread deployment of data centers, real-time online monitoring of sensitive areas involving information security, such as computer rooms and offices, is becoming increasingly important. Initially, the method of arranging dedicated personnel to guard sensitive areas can ensure the security of these sensitive areas, but the labor and management costs are high; later, surveillance cameras were used to monitor the opening and closing status of doors in sensitive areas. Although the labor cost was reduced, manual monitoring is still required. In addition, the transmission and processing of massive monitoring data will consume a lot of time and traffic costs.
[0003] With the development of visual algorithms, the method of door opening and closing status detection based on traditional computer vision processing only needs to identify the video of the monitored door to output the door opening and closing status, avoiding manual viewing of a large amount of monitoring data. However, this method is easily affected by environmental changes and the detection accuracy is poor.
[0004] In recent years, with the development of neural network image recognition technology, the door opening and closing state detection algorithm based on convolutional neural network can train the neural network model according to the image data set marked with the actual opening and closing state of the door, and then obtain the opening and closing state of the door according to the trained neural network model. Compared with the door opening and closing state detection method based on traditional computer vision processing, it has better robustness and higher accuracy; however, this door opening and closing state detection algorithm based on convolutional neural network requires a large amount of data set, has high requirements on the quality of the data set, and has a high complexity of the training model and a large number of model parameters. Therefore, the data set cost of the model of this detection algorithm is large, the training cost is high, and the real-time performance is poor. Summary of the invention
[0005] The purpose of the present invention is to provide a method for detecting the opening and closing state of a door, so as to solve the problems that the current door opening and closing state detection algorithm based on convolutional neural network requires a large data set cost, high calculation cost and poor real-time performance.
[0006] In order to solve the above technical problems, the present invention provides a method for detecting the opening and closing state of a door, the method comprising:
[0007] The grayscale image of the front view of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain a frame difference image;
[0008] The frame difference image is input into a trained neural network model to obtain a feature image, and the score of the probability of existence of each straight line is obtained according to the data of the first channel of the feature image; the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line are obtained according to the data of the second channel to the fifth channel of the feature image; the opening and closing state of the door at the current moment is judged according to the angle between the straight line with the largest slope and the straight line with the smallest slope among the straight lines selected so that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score is greater than the score threshold;
[0009] The training steps of the model include:
[0010] S1: Input the frame difference image in the training set into the model and output the feature map; obtain the score of the probability of existence of each straight line according to the data of the first channel of the feature map; obtain the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line according to the data of the second channel to the fifth channel of the feature map;
[0011] S2: Update the model based on the loss function of the model calculated according to the obtained scores of the existence probabilities of each straight line, the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line and their corresponding true labels, and iterate S1-S2 until the training is completed.
[0012] Further, the method of judging the opening and closing state of the door at the current moment according to the angle between the straight line with the largest slope and the straight line with the smallest slope among the selected straight lines satisfying the condition that the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the score is greater than the score threshold includes:
[0013] If the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the corresponding scores of each straight line are greater than the score threshold, and the angle between the straight line with the largest slope and the straight line with the smallest slope among the selected straight lines is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, then it is determined that the door is in the open state at the current moment; otherwise, it is determined that the door is in the closed state at the current moment.
[0014] Furthermore, the reference image is combined with a grayscale image of a current moment image with the same size and the same viewing angle as the reference image to obtain a frame difference image, including:
[0015] An initial frame difference map is obtained by using a grayscale map of a current image with a reference image of the same size and the same viewing angle as the reference image; the image threshold is adjusted according to the difference between the brightness feature values of the current image and the reference image and the set maximum value of the image threshold, and the initial frame difference map is threshold filtered according to the image threshold to obtain a frame difference map for input into a trained neural network model.
[0016] Further, the manner of adjusting the image threshold by the difference and the set maximum value of the image threshold includes:
[0017]
[0018] In the formula, threshold is the image threshold, L F L is the brightness characteristic value of the picture at the current moment; oT is the brightness characteristic value of the reference image, max t is the maximum value set for the image threshold, bias is the bias parameter, and k is the scaling factor.
[0019] Furthermore, the method of obtaining the score of the existence probability of each straight line according to the data of the first channel of the feature map includes:
[0020] The data output by the first channel and corresponding to the scores of the existence probabilities of the respective straight lines are respectively normalized to obtain the scores of the existence probabilities of the respective straight lines.
[0021] Further, the method of obtaining the selected straight line that satisfies the conditions that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score is greater than the score threshold includes:
[0022] The normalized scores are sorted by size, and a preset number of straight lines with the largest normalized scores are selected as selected straight lines; if the distance between the starting point coordinates and the end point coordinates of the selected straight line is greater than a distance threshold and the score is greater than the score threshold, then the straight line is determined to be a selected straight line that satisfies the conditions that the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the score is greater than the score threshold, otherwise it is determined not to be a selected straight line.
[0023] Furthermore, the method of normalizing the data output by the first channel and corresponding to the scores of the existence probabilities of each straight line includes:
[0024]
[0025] In the formula, heats(i,j) is the score of the probability of existence of the straight line corresponding to the (i,j)th feature output by the first channel, and center(i,j) is the data of the (i,j)th feature output by the first channel.
[0026] Furthermore, the initial frame difference image is obtained by using a grayscale image of a current moment image having a unified size with the reference image and the same viewing angle as the reference image, including:
[0027] An initial frame difference image is obtained according to the absolute value of pixel-by-pixel difference between the reference image and the grayscale image of the current moment image.
[0028] Furthermore, the initial frame difference image is threshold filtered according to the image threshold as follows:
[0029] If the pixel value of a certain pixel point of the initial frame difference image is less than the image threshold, the pixel value of the pixel point is set to zero; otherwise, the pixel value of the pixel point is set to the maximum pixel value.
[0030] Furthermore, the reference image is updated according to the judgment result of the door opening and closing state, and the method of updating the reference image includes: if it is determined that the door is in a closed state at the current moment, the grayscale image of the picture at the current moment is saved in the reference image set according to the time period to which the current moment belongs, as an updated reference image.
[0031] Beneficial effect: The present invention provides a new method for detecting the opening and closing state of a door. The method detects the closing state of the door based on a neural network. First, a frame difference map is obtained based on a grayscale image of a front view image of the door and the wall on which it is located when the door is closed as a reference image and a grayscale image of a current moment image with the same size and viewing angle as the reference image. The obtained frame difference map is input into a trained neural network model to obtain a feature map. Then, the straight line feature information obtained according to the feature map is analyzed. The opening and closing state of the door is judged based on the angle between the straight line with the largest slope and the straight line with the smallest slope among the straight lines selected such that the distance between the starting point coordinates and the end point coordinates is greater than a distance threshold and the corresponding score is greater than a score threshold. Compared with the existing method of inputting a picture of the door into a trained neural network model to directly obtain the opening and closing state of the door to detect the opening and closing state of the door. The method for detecting the opening and closing state of a door only needs to train a neural network model that can extract straight line feature information to ensure high precision, high stability and high scene adaptability of the straight line feature information. The part for judging the opening and closing state of the door is realized through geometric principle analysis on the basis of the straight line feature information extracted by the model; that is, the neural network and the geometric principle are combined, which can not only take advantage of the high precision of the neural network and the small influence of external factors, but also share the workload of the judgment part in the door opening and closing state detection task through geometric principle analysis. Therefore, under the premise of ensuring accuracy, stability and scene adaptability, the required parameter amount of the neural network model can be effectively reduced, the workload required for training is reduced, and the cost of the data set required for training is lower, thereby making the training cost lower and the real-time performance higher in actual applications after training. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a flow chart of a method for detecting a door opening and closing state in an embodiment of the method for detecting a door opening and closing state of the present invention. DETAILED DESCRIPTION
[0033] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.
[0034] Door opening and closing state detection method embodiment
[0035] The present embodiment provides a technical solution for a method for detecting the opening and closing state of a door, wherein the method obtains a feature map by inputting a frame difference map obtained according to a grayscale map of a reference image and a current image into a trained neural network model, extracts straight line feature information in the feature map, and then performs geometric principle analysis on the obtained straight line feature information to determine the opening and closing state of the door; compared with the existing method for detecting the opening and closing state of a door by inputting a picture of the door into a trained neural network model to directly obtain the opening and closing state of the door, the present invention combines neural network model processing with geometric principle analysis to determine the opening and closing state of the door, which can not only utilize the advantages of high accuracy of neural networks and less influence by external factors, but also share the workload of the judgment part in the door opening and closing state detection task through geometric principle analysis. Therefore, under the premise of ensuring accuracy stability and scene adaptability, the required amount of parameters of the neural network model can be effectively reduced, the workload required for training can be reduced, and the cost of the data set required for training can be lowered, thereby reducing the training cost and achieving higher real-time performance in actual applications after training.
[0036] The flow chart of the method for detecting the door opening and closing state is as follows Figure 1 As shown, the method specifically includes:
[0037] The grayscale image of the front view of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain a frame difference image;
[0038] The frame difference image is input into the trained neural network model to obtain a feature map, and the score of the existence probability of each straight line is obtained according to the data of the first channel of the feature map; the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line are obtained according to the data of the second channel to the fifth channel of the feature map; the opening and closing state of the door at the current moment is judged according to the angle between the straight line with the largest slope and the straight line with the smallest slope among the straight lines selected so that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score of the existence probability of each straight line is greater than the score threshold;
[0039] The training steps of the neural network model include:
[0040] S1: Input the frame difference image in the training set into the neural network model and output the feature map; obtain the score of the probability of existence of each straight line according to the data of the first channel of the feature map; obtain the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line according to the data of the second to fifth channels of the feature map;
[0041] S2: Update the neural network model based on the loss function of the neural network model calculated based on the scores of the existence probabilities of each straight line, the horizontal and vertical coordinates of the starting point and the end point of each straight line, and their corresponding true labels, and iterate S1-S2 until the training is completed.
[0042] This method only needs to train a neural network model that can extract straight line feature information to ensure high precision, high stability and high scene adaptability of the straight line feature information. The part of judging the door opening and closing state is realized through geometric principle analysis based on the straight line feature information extracted by the model. That is, the neural network and geometric principle are combined, which can not only take advantage of the high precision of the neural network and the small influence of external factors, but also share the workload of the judgment part in the door opening and closing state detection task through geometric principle analysis. Therefore, under the premise of ensuring accuracy, stability and scene adaptability, it can effectively reduce the number of parameters of the required neural network model, reduce the workload required for training, and make the cost of the data set required for training lower, thereby making the training cost lower and having higher real-time performance in practical applications.
[0043] The following is a detailed description of this method:
[0044] The grayscale image of the front view picture of the door and the wall on which it is located when the door is closed is used as a reference image, and the frame difference image is obtained by combining the grayscale image of the current moment picture with the same size and the same viewing angle as the reference image. In this embodiment, the method of obtaining the reference image is: according to the current moment, the reference image corresponding to the current moment is selected from the reference image set, and the method of obtaining the reference image set is: the video of the front view direction of the door and the wall on which it is located when the door is closed is collected at a certain time interval to obtain the front view picture of the door and the wall on which it is located when the door is closed, and then the collected picture is converted into a grayscale image and stored in the reference image list I. oT =[I o0 ,I o1 ,...,I os ](S refers to the total number of reference images minus 1).
[0045] In this embodiment, the reference image and the current moment picture with the same size and the same viewing angle as the reference image have a height h of 720, a width w of 1280, and a number of channels c of 3 (in other embodiments, it can be flexibly set according to needs. It should be noted that the number of channels here is independent of the first channel to the fifth channel of the feature map); the reference image is combined with the grayscale image of the current moment picture with the same size and the same viewing angle as the reference image to obtain the frame difference map in the following manner:
[0046] An initial frame difference image is obtained by comparing a reference image with a grayscale image of a current image of the same size and the same viewing angle as the reference image; the image threshold is adjusted according to the difference between the brightness characteristic values of the current image and the reference image and the set maximum value of the image threshold. In this embodiment, the method of adjusting the image threshold according to the difference between the brightness characteristic values of the current image and the reference image and the set maximum value of the image threshold includes:
[0047]
[0048] In the formula, threshold is the image threshold, L F is the brightness characteristic value of the image at the current moment; L oT is the brightness characteristic value of the reference image, max t is the maximum value of the image threshold, bias is the bias parameter, and k is the scaling factor. In this embodiment, k, bias, max t Usually, they are 1.2, 40, and 210 respectively. Among them, the brightness characteristic value L of the current image is obtained F The formula is:
[0049] L F =mean[Frame()]
[0050] Where, L F is the brightness feature value of the current image, mean refers to the mean symbol, and Frame() refers to the pixel value of the current image;
[0051] Get the brightness eigenvalue L of the reference image oi The formula is:
[0052] L oT =mean[I oT ()]
[0053] Where, L oT is the brightness feature value of the reference image, mean refers to the mean symbol, I oT () refers to the pixel value of the reference image.
[0054] The initial frame difference image is threshold filtered according to the image threshold to obtain a frame difference image for input into the trained neural network model. The threshold filtering process can filter out some pixel changes in the initial frame difference image caused by environmental factors, thereby reducing the influence of environmental factors. In this embodiment, the method of threshold filtering the initial frame difference image according to the image threshold is specifically as follows:
[0055] If the pixel value of a certain pixel of the initial frame difference image is less than the image threshold, the pixel value of the pixel is set to zero, otherwise, the pixel value of the pixel is set to the maximum pixel value. The formula for threshold filtering the frame difference image according to the image threshold is:
[0056]
[0057] Where Thd(i,j,k) is the pixel value of the frame difference image after threshold filtering, 255 is the maximum pixel value, and I sub (i, j, k) is the pixel value of the frame difference image, threshold is the image threshold, h is the height, w is the width, and c is the number of channels.
[0058] In this embodiment, the method of obtaining the initial frame difference map by using the reference image and the grayscale map of the current moment picture with the same size and the same viewing angle as the reference image is as follows: the initial frame difference map is obtained according to the absolute value of the pixel-by-pixel difference between the reference image and the grayscale map of the current moment picture. This method of obtaining the initial frame difference map can avoid the situation where the details of the door frame of the obtained frame difference map are lost and key information is lost due to only pixel-by-pixel difference, thereby making the door opening and closing state detection more accurate; the calculation formula for obtaining the pixel value of the frame difference map is:
[0059] I sub (i,j,k)=|Frame(i,j,k)-I oT (i,j,k)|,i∈[0,h],j∈[0,w],k∈[0,c]
[0060] In the formula, I sub (i, j, k) is the pixel value of the frame difference image, Frame(i, j, k) is the pixel value of the current image with the same size and the same viewing angle as the reference image, I oT (i, j, k) is the pixel value of the grayscale image of the front view of the door and the wall on which it is located when the door is closed (i.e., the pixel value of the reference image corresponding to the current moment); h is the height, w is the width, c is the number of channels, and T is the sequence number of the reference image corresponding to the current moment in the reference image list. In other embodiments, the grayscale image of the reference image and the image at the current moment may be subtracted pixel by pixel to obtain a frame difference image, which may result in the loss of some door frame details and key information in the obtained frame difference image, thereby affecting the accuracy of the door opening and closing state detection.
[0061] The frame difference map used to be input into the trained neural network model is input into the trained model to obtain a feature map features(i,j,k) with a shape of [5, 256, 256], i∈[0,4], j∈[0,255], k∈[0,255], wherein the training steps of the neural network model include:
[0062] S1: Input the frame difference image in the training set into the neural network model and output the feature map; obtain the score of the probability of existence of each straight line according to the data of the first channel of the feature map; obtain the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line according to the data of the second to fifth channels of the feature map;
[0063] S2: The loss function of the neural network model is calculated based on the scores of the probability of existence of each straight line, the horizontal and vertical coordinates of the starting point and the end point of each straight line, and their corresponding true labels, wherein the true label corresponding to the data extracted from the first channel of the feature map is the probability of the existence of the straight line; the true labels corresponding to the data extracted from the second to fifth channels of the feature map are the horizontal and vertical coordinates of each straight line and the horizontal and vertical coordinates of the end point, respectively; the neural network model is updated, and S1-S2 is iterated until the training is completed.
[0064] The scores of the existence probability of each straight line are obtained according to the data of the first channel of the feature map; the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line are obtained according to the data of the second to fifth channels of the feature map; wherein the data of the first channel of the feature map is:
[0065] center(j,k)=features(0,j,k)j∈[0,255],k∈[0,255]
[0066] In the formula, features(0,j,k) and center(j,k) are the data of the first channel of the feature map;
[0067] The data of the second to fifth channels of the feature map are:
[0068] displacements(i,j,k)=features(i,j,k),i∈[1,4],j∈[0,255],k∈[0,255]
[0069]
[0070] Where displacements(i,j,k) and features(i,j,k)(i∈[1,4]) are the data of the second to fifth channels (i.e., i is 1-4) in the feature map; x1 and y1 represent the horizontal and vertical coordinates of the starting point of each straight line, and x2 and y2 represent the horizontal and vertical coordinates of the end point of each straight line, respectively.
[0071] The door opening and closing state is determined based on the angle between the straight line with the largest slope and the straight line with the smallest slope selected from the straight lines that satisfy the conditions that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score of the existence probability of each straight line is greater than the score threshold. In this embodiment, the method of obtaining the score of the existence probability of each straight line based on the data of the first channel of the feature map is specifically as follows:
[0072] The data outputted from the first channel of the feature graph and corresponding to the scores of the probability of existence of each straight line are respectively normalized to obtain the scores of the probability of existence of each straight line. The method of normalizing the data outputted from the first channel of the feature graph and corresponding to the scores of the probability of existence of each straight line includes:
[0073]
[0074] In the formula, heats(i,j) is the score of the probability of existence of the straight line corresponding to the (i,j)th feature output by the first channel, and center(i,j) is the data of the (i,j)th feature output by the first channel.
[0075] Normalization processing can remove the interference of noise and make the data output by the first channel smoother. In other embodiments, normalization processing may not be performed;
[0076] In this embodiment, the method of obtaining the selected straight line that satisfies the conditions that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score of the straight line existence probability is greater than the score threshold is specifically as follows:
[0077] The scores of the existence probabilities of the straight lines after normalization are sorted by size, and a preset number of straight lines with the largest scores after normalization are selected as the selected straight lines (in other embodiments, normalization may not be performed. If normalization is not performed, the scores of the existence probabilities of the straight lines that have not been normalized are directly sorted).
[0078] In this embodiment, the number is set to 200. If the distance between the starting point coordinates and the end point coordinates of the selected straight line is greater than the distance threshold Thdd (when the distance is greater than the distance threshold, the possibility that the straight line is a real straight line on the door is greater, and the possibility that it is a noise straight line is reduced) and the score of the straight line existence probability is greater than the score threshold Thds (when the score of the straight line existence probability is greater than the score threshold, the possibility that the straight line exists is greater), then the straight line is determined to be a selected straight line that satisfies the conditions that the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the score of the straight line existence probability is greater than the score threshold, otherwise it is determined not to be. In this embodiment, the distance threshold Thdd is set to 30 pixels; the score threshold Thds is set to 0.20.
[0079] In this embodiment, the method of judging the opening and closing state of the door at the current moment according to the angle between the straight line with the largest slope and the straight line with the smallest slope among the straight lines selected so that the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the score is greater than the score threshold is as follows:
[0080] If the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the corresponding scores of each straight line are greater than the score threshold, and the angle between the straight line with the largest slope and the straight line with the smallest slope among the selected straight lines is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, then it is determined that the door is in the open state at the current moment; otherwise, it is determined that the door is in the closed state at the current moment; wherein the straight line with the largest slope is equivalent to the straight line where the upper and lower boundaries of the door are located or the parallel line of the straight line where the upper and lower boundaries of the door are located, and the straight line with the smallest slope is equivalent to the straight line where the left and right boundaries of the door are located or the parallel line of the straight line where the left and right boundaries of the door are located; in this embodiment, the door opening and closing state is determined by judging whether the tangent value of the angle between the straight line with the largest slope and the straight line with the smallest slope among the selected straight lines satisfies the condition that it is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, that is:
[0081]
[0082] k1=max{k i}
[0083] k2=min{k i}
[0084] In the formula, Function(theta d ) is the door opening and closing state at the current moment, Thd d Theta is the set angle range (the set angle range is reflected by the tangent value of the set angle, the upper limit of the range is the upper limit of the set angle, and the lower limit is the lower limit of the set angle); d is the tangent value of the angle between the line with the largest slope and the line with the smallest slope among the selected lines, x1 and y1 are the horizontal and vertical coordinates of the starting point of the ith line, x2 and y2 are the horizontal and vertical coordinates of the end point of the ith line, k1 is the slope of the line with the largest slope among the selected lines, and k2 is the slope of the line with the smallest slope among the selected lines; in this embodiment, the range of the tangent value of the angle is set to Thd d To [1.7,10], the upper limit of the tangent value of the angle is set to 10, and the lower limit of the tangent value of the angle is set to 1.7.
[0085] In this embodiment, when it is determined that the door is in an open state at the current moment, the abnormal state video and picture are saved and an alarm message is issued.
[0086] In this embodiment, the reference image is updated based on the judgment result of the door opening or closing state. The specific method of updating the reference image is as follows: if it is determined that the door is in a closed state at the current moment, the grayscale image of the picture at the current moment is saved in the reference image set according to the time period to which the current moment belongs, as an updated reference image.
[0087] According to the positive alarm rate comparison table of the door opening and closing detection method of this embodiment, the traditional visual edge extraction Hough transform straight line detection widely used by cameras for door opening and closing state detection (edge extraction Hough transform), and the YOLOv2 neural network algorithm for detecting the door opening and closing state in pictures with the same input picture size of (720, 1280, 3) in multiple environments with different light and dark, as shown in Table 1, the inference delay (ms) comparison table of door opening and closing state detection is shown in Table 2;
[0088] Table 1
[0089]
[0090] Table 2
[0091]
[0092]
[0093] It can be seen from Table 1 and Table 2 that the door opening and closing detection method in this embodiment is used to detect the opening and closing status of the access control. Under different light and disturbance conditions, compared with traditional computer vision, the detection rate is increased by 3%-7%; the detection speed is increased by more than 100ms. Overall, while maintaining the detection accuracy, the detection speed is improved, the false detection and missed detection are reduced, and the real-time requirements are met. Compared with the traditional neural network model algorithm, while ensuring that the detection rate is basically the same, the door opening and closing detection method in this embodiment has a faster detection speed, which shows that it has good real-time performance and requires low computing costs.
[0094] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, but do not constitute a limitation to the present invention.
Claims
1. A method for detecting the opening and closing state of a door, characterized in that: include: The grayscale image of the front view of the door and the wall on which it is located when the door is closed is used as a reference image, and the grayscale image of the current moment image with the same size and the same viewing angle as the reference image is combined to obtain a frame difference image; The frame difference image is input into a trained neural network model to obtain a feature map, and the score of the probability of existence of each straight line is obtained according to the data of the first channel of the feature map; The horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line are obtained according to the data of the second channel to the fifth channel of the characteristic graph; the opening and closing state of the door at the current moment is determined according to the angle between the straight line with the largest slope and the straight line with the smallest slope among the straight lines selected so that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score is greater than the score threshold; The training steps of the model include: S1: Input the frame difference image in the training set into the model and output the feature map; obtain the score of the probability of existence of each straight line according to the data of the first channel of the feature map; obtain the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line according to the data of the second channel to the fifth channel of the feature map; S2: Update the model based on the loss function of the model calculated according to the obtained scores of the existence probabilities of each straight line, the horizontal and vertical coordinates of the starting point and the horizontal and vertical coordinates of the end point of each straight line and their corresponding true labels, and iterate S1-S2 until the training is completed.
2. The method for detecting the door opening and closing state according to claim 1, characterized in that: The method of judging the opening and closing state of the door at the current moment according to the angle between the straight line with the largest slope and the straight line with the smallest slope among the straight lines selected so as to satisfy the condition that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score is greater than the score threshold includes: If the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the corresponding scores of each straight line are greater than the score threshold, and the angle between the straight line with the largest slope and the straight line with the smallest slope among the selected straight lines is less than or equal to the set angle upper limit and greater than or equal to the set angle lower limit, then it is determined that the door is in the open state at the current moment; otherwise, it is determined that the door is in the closed state at the current moment.
3. The method for detecting the door opening and closing state according to claim 1 or 2, characterized in that: Methods of obtaining a frame difference image by combining the reference image with a grayscale image of the current moment image with the same size and the same viewing angle as the reference image include: An initial frame difference map is obtained by using a grayscale map of a current image with a reference image of the same size and the same viewing angle as the reference image; the image threshold is adjusted according to the difference between the brightness feature values of the current image and the reference image and the set maximum value of the image threshold, and the initial frame difference map is threshold filtered according to the image threshold to obtain a frame difference map for input into a trained neural network model.
4. The method for detecting the door opening and closing state according to claim 3, characterized in that: The method of adjusting the image threshold by the difference and the set maximum value of the image threshold includes: In the formula, threshold is the image threshold, L F L is the brightness characteristic value of the picture at the current moment; oT is the brightness characteristic value of the reference image, max t is the maximum value set for the image threshold, bias is the bias parameter, and k is the scaling factor.
5. The method for detecting the door opening and closing state according to claim 1 or 2, characterized in that: Methods for obtaining the scores of the existence probabilities of each straight line according to the data of the first channel of the feature graph include: The data output by the first channel and corresponding to the scores of the existence probabilities of the respective straight lines are respectively normalized to obtain the scores of the existence probabilities of the respective straight lines.
6. The method for detecting the door opening and closing state according to claim 5, characterized in that: The method of obtaining the selected straight line that satisfies the requirement that the distance between the starting point coordinate and the end point coordinate is greater than the distance threshold and the score is greater than the score threshold includes: The normalized scores are sorted by size, and a preset number of straight lines with the largest normalized scores are selected as selected straight lines; if the distance between the starting point coordinates and the end point coordinates of the selected straight line is greater than a distance threshold and the score is greater than the score threshold, then the straight line is determined to be a selected straight line that satisfies the conditions that the distance between the starting point coordinates and the end point coordinates is greater than the distance threshold and the score is greater than the score threshold, otherwise it is determined not to be a selected straight line.
7. The method for detecting the door opening and closing state according to claim 5, characterized in that: The method of normalizing the data output by the first channel and corresponding to the scores of the existence probabilities of each straight line includes: In the formula, heats(i,j) is the score of the probability of existence of the straight line corresponding to the (i,j)th feature output by the first channel, and center(i,j) is the data of the (i,j)th feature output by the first channel.
8. The method for detecting the door opening and closing state according to claim 3, characterized in that: Methods for obtaining an initial frame difference image by using a grayscale image of a current moment image with a uniform size and the same viewing angle as the reference image include: An initial frame difference image is obtained according to the absolute value of pixel-by-pixel difference between the reference image and the grayscale image of the current moment image.
9. The method for detecting the door opening and closing state according to claim 8, characterized in that: The method of threshold filtering the initial frame difference image according to the image threshold is: If the pixel value of a certain pixel point of the initial frame difference image is less than the image threshold, the pixel value of the pixel point is set to zero; otherwise, the pixel value of the pixel point is set to the maximum pixel value.
10. The method for detecting the door opening and closing state according to claim 1 or 2, characterized in that: The reference image is updated according to the judgment result of the door opening and closing state, and the method of updating the reference image includes: if it is determined that the door is in a closed state at the current moment, the grayscale image of the current moment picture is saved in the reference image set according to the time period to which the current moment belongs, as an updated reference image.