Control room cabinet door opening and closing state identification method and system

Abstract

The invention relates to the technical field of intelligent monitoring, and discloses a control room cabinet door opening and closing state identification method and system, and the method comprises the steps: carrying out the instance segmentation and anchor point positioning of a cabinet door region in a control room monitoring image, so as to obtain the scanning point coordinates of each cabinet door in the cabinet door region; determining a cabinet door state identification area of each cabinet door by using a FloodFill algorithm according to the anchor point coordinates of each cabinet door; performing single-cabinet-door mask processing on the cabinet door state identification area of each cabinet door by using a Canny algorithm to obtain a single-cabinet-door state identification area of each cabinet door; judging whether a cabinet door in an open state exists or not according to the cabinet door area of the single cabinet door state identification area of each cabinet door; and when the cabinet door in the open state exists, alarm processing is performed on the cabinet door in the open state, so that the accuracy of identifying the opening and closing state of the cabinet door of the control room is improved, the phenomenon of mistaken identification of the opening and closing state of the cabinet door is avoided, and the potential safety hazard of a monitoring scene is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent monitoring, and in particular relates to a method and system for identifying the opening and closing status of a cabinet door in a control room. Background Art

[0002] The control room of a hydropower station can control various process indicators of the production system and adjust the valve position to control the site so that the hydropower station system can operate normally. Therefore, it is very important to monitor the behavior of the control room, analyze the behavior, and prevent abnormal behaviors such as misoperation. Intelligent video surveillance based on control room behavior analysis has more powerful intelligent features and more efficient management efficiency than traditional video surveillance. Therefore, the use of intelligent monitoring based on control room behavior analysis is an important development direction of the hydropower station system and has great significance for the intelligent construction of the hydropower station system.

[0003] However, the accuracy of the recognition of the opening and closing state of the cabinet door in the intelligent video monitoring based on the control room behavior analysis in the prior art needs to be improved, and there will still be the phenomenon of misidentification of the cabinet door opening and closing state, which brings considerable safety hazards to the monitoring scene and is not conducive to the security management of the monitoring scene. Because there are high-voltage power supplies, cables and other components inside the electrical control cabinet, if the cabinet door is in an open state for a long time or is not closed tightly, it will lead to safety hazards such as exposure of components and poor contact of power lines. If personnel accidentally touch the electrical equipment, it is also easy to cause electric shock accidents, which seriously endangers personal safety. On the other hand, the equipment in the control cabinet has strict requirements on the working environment. If the cabinet door is in an open state for a long time or is not closed tightly, it is easy to be affected by the external environment such as rats, insects, dust, moisture, pollution, etc., which will cause damage to components and equipment failure, thereby affecting work efficiency. Therefore, how to improve the accuracy of the recognition of the opening and closing state of the cabinet door in the control room and avoid the occurrence of misidentification of the opening and closing state of the cabinet door has become a problem to be solved urgently. Summary of the invention

[0004] In order to solve the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a method and system for identifying the opening and closing status of cabinet doors in a control room, aiming to solve the technical problem that the prior art cannot improve the accuracy of identifying the opening and closing status of cabinet doors in the control room, so as to avoid the occurrence of misidentification of the opening and closing status of cabinet doors.

[0005] The technical solution adopted by the present invention is:

[0006] A method for identifying the opening and closing state of a control room cabinet door comprises the following steps:

[0007] Perform instance segmentation and anchor point positioning on the cabinet door area in the control room monitoring screen to obtain the coordinates of the points of each cabinet door in the cabinet door area;

[0008] Determine the door state recognition area of ​​each door using the FloodFill algorithm according to the anchor point coordinates of each door;

[0009] The Canny algorithm is used to perform single door mask processing on the cabinet state recognition area of ​​each cabinet door to obtain the single door state recognition area of ​​each cabinet door;

[0010] Determine whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door;

[0011] When there is a cabinet door in an open state, an alarm process is performed on the cabinet door in an open state.

[0012] Optionally, before the step of judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door, the method further includes:

[0013] Performing corner point detection on the single door state recognition area of ​​each door using the Corner Harris corner point detection algorithm to obtain corner point information of the single door state recognition area of ​​each door;

[0014] Accordingly, the step of judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door includes:

[0015] The door area of ​​the single door status identification area of ​​each door is calculated based on the single door status identification area and corner point information of each door, and it is determined whether there is a door in an open state based on the door area of ​​the single door status identification area of ​​each door.

[0016] Optionally, the step of determining the door state recognition area of ​​each door by using a FloodFill algorithm according to the anchor point coordinates of each door includes:

[0017] The four-neighborhood filling method or the eight-neighborhood filling method in the FloodFill algorithm is used to start from the anchor point coordinates of each cabinet door, and expand and fill the adjacent unfilled areas in a breadth-first search or depth-first search traversal manner to determine the cabinet state recognition area of ​​each cabinet door.

[0018] Optionally, the step of performing single door mask processing on the cabinet door state recognition area of ​​each cabinet door by using the Canny algorithm to obtain the single door state recognition area of ​​each cabinet door includes:

[0019] Performing grayscale conversion processing and Gaussian filtering processing on the cabinet door state recognition area of ​​each cabinet door to obtain a filtered grayscale image of the cabinet door state recognition area of ​​each cabinet door;

[0020] The Sobel operator is used to calculate the gradient intensity and gradient direction of each pixel in the filtered grayscale image, and non-maximum suppression processing is performed according to the gradient intensity and gradient direction of each pixel to obtain the local maximum pixel;

[0021] The single door edge of the door state recognition area of ​​each door is identified by using the Canny algorithm according to the local maximum pixel point using the double threshold method;

[0022] The door status recognition area of ​​each door is subjected to single door masking processing based on the single door edge of the door status recognition area of ​​each door to obtain the single door status recognition area of ​​each door.

[0023] Optionally, the step of judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door includes:

[0024] Determining whether there is a cabinet door in an open state according to whether the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door is less than or equal to a preset cabinet door area threshold;

[0025] When there is a cabinet door in an open state, the step of performing an alarm process on the cabinet door in an open state includes:

[0026] When there is a cabinet door in an open state, the opening time of the cabinet door in an open state is calculated;

[0027] When the opening time is greater than or equal to the preset opening time, an alarm is processed for the cabinet door whose opening time is greater than or equal to the preset opening time.

[0028] A control room cabinet door opening and closing state recognition system, comprising:

[0029] Anchor point positioning module, used to perform instance segmentation and anchor point positioning on the cabinet door area in the control room monitoring screen to obtain the coordinates of the points of each cabinet door in the cabinet door area;

[0030] An area determination module, used to determine the door state recognition area of ​​each door using a FloodFill algorithm according to the anchor point coordinates of each door;

[0031] A mask processing module, used for performing single-door mask processing on the cabinet state recognition area of ​​each cabinet door by using the Canny algorithm to obtain the single-door state recognition area of ​​each cabinet door;

[0032] A state judgment module, used to judge whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door;

[0033] The cabinet door alarm module is used to perform alarm processing on the cabinet door in the open state when there is a cabinet door in the open state.

[0034] Optionally, the state judgment module is further used to perform corner point detection on the single door state recognition area of ​​each door by using the CornerHarris corner point detection algorithm to obtain corner point information of the single door state recognition area of ​​each door;

[0035] The status judgment module is also used to calculate the door area of ​​the single door status identification area of ​​each door based on the single door status identification area and corner point information of each door, and to judge whether there is a door in an open state based on the door area of ​​the single door status identification area of ​​each door.

[0036] Optionally, the area determination module is also used to use the four-neighborhood filling method or the eight-neighborhood filling method in the FloodFill algorithm to start from the anchor point coordinates of each cabinet door, and to expand and fill in adjacent unfilled areas in a breadth-first search or depth-first search traversal manner to determine the cabinet status identification area of ​​each cabinet door.

[0037] Optionally, the mask processing module is further used to perform grayscale conversion processing and Gaussian filtering processing on the cabinet door state recognition area of ​​each cabinet door to obtain a filtered grayscale image of the cabinet door state recognition area of ​​each cabinet door;

[0038] The mask processing module is also used to calculate the gradient intensity and gradient direction of each pixel in the filtered grayscale image using the Sobel operator, and perform non-maximum suppression processing according to the gradient intensity and gradient direction of each pixel to obtain local maximum pixel points;

[0039] The mask processing module is also used to identify the single cabinet door edge of the cabinet door state recognition area of ​​each cabinet door by using the Canny algorithm according to the local maximum pixel point using the double threshold method;

[0040] The mask processing module is also used to perform single door mask processing on the door status identification area of ​​each cabinet door based on the single door edge of the door status identification area of ​​each cabinet door to obtain the single door status identification area of ​​each cabinet door.

[0041] Optionally, the state judgment module is further used to judge whether there is a cabinet door in an open state according to whether the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door is less than or equal to a preset cabinet door area threshold;

[0042] The cabinet door alarm module is further used to calculate the opening time of the cabinet door in the open state when there is a cabinet door in the open state;

[0043] The cabinet door alarm module is also used to perform alarm processing on the cabinet door whose opening time is greater than or equal to the preset opening time when the opening time is greater than or equal to the preset opening time.

[0044] The beneficial effects of the present invention are:

[0045] In the present invention, the cabinet door area in the control room monitoring screen is segmented and anchored to obtain the coordinates of the points of each cabinet door in the cabinet door area; the cabinet door state identification area of ​​each cabinet door is determined by the FloodFill algorithm according to the anchor point coordinates of each cabinet door; the cabinet door state identification area of ​​each cabinet door is masked by the Canny algorithm to obtain the single cabinet door state identification area of ​​each cabinet door; it is determined whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door; when there is a cabinet door in an open state, an alarm is processed for the cabinet door in an open state. By using algorithms such as FloodFill and Canny to identify the opening and closing state of the cabinet door area in the control room monitoring screen, and combining the area calculation of the current cabinet door opening and closing state, the accuracy of the control room cabinet door opening and closing state identification is further improved, the occurrence of the phenomenon of misidentification of the cabinet door opening and closing state is avoided, the safety hazards of the monitoring scene are reduced, and it is also beneficial to the security management of the monitoring scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of a first embodiment of a method for identifying the opening and closing status of a cabinet door in a control room of the present invention;

[0047] Figure 2 It is a diagram of the main components of the SAM model involved in the first embodiment of the method for identifying the opening and closing state of a cabinet door in a control room of the present invention;

[0048] Figure 3 The structure diagram of the mask decoder involved in the first embodiment of the method for identifying the opening and closing state of a cabinet door in a control room of the present invention;

[0049] Figure 4 Schematic diagram of the position of the cabinet door anchor point involved in the first embodiment of the control room cabinet door opening and closing state identification method of the present invention;

[0050] Figure 5 It is a schematic diagram of the filling method involved in the first embodiment of the method for identifying the opening and closing state of a cabinet door in a control room of the present invention;

[0051] Figure 6 This is a cabinet door state area recognition effect diagram involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention;

[0052] Figure 7 This is a diagram of the edge connection process involved in the first embodiment of the method for identifying the opening and closing state of a cabinet door in a control room of the present invention;

[0053] Figure 8 A comparison diagram of the door mask processing effects involved in the first embodiment of the control room door opening and closing state recognition method of the present invention;

[0054] Fig. 9 Schematic diagram of Corner Harris corner point detection involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention;

[0055] Fig.10 A comparison diagram of the Corner Harris corner point detection effect involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention;

[0056] Fig.11 This is a comparison diagram of the cabinet door opening and closing state recognition effect involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention;

[0057] Fig.12 A technical roadmap for the first embodiment of the method for identifying the opening and closing state of a cabinet door in a control room of the present invention;

[0058] Fig.13 This is a structural block diagram of the first embodiment of the control room cabinet door opening and closing status recognition system of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0061] The embodiment of the present invention provides a method for identifying the opening and closing state of a cabinet door in a control room, referring to Figure 1 , Figure 1 It is a flow chart of the first embodiment of the method for identifying the opening and closing status of a cabinet door in a control room of the present invention.

[0062] In this embodiment, the control room door opening and closing state recognition method comprises the following steps:

[0063] Step S10: performing instance segmentation and anchor point positioning on the cabinet door area in the control room monitoring screen to obtain the coordinates of the points of each cabinet door in the cabinet door area;

[0064] It should be noted that in this embodiment, instance segmentation of the cabinet door area in the control room monitoring screen is realized based on the Segment Anything Model (SAM). The Meta team released the promptable segmentation model SAM in 2023 and trained it on a dataset SA-1B with more than 1 billion masks (Mask, use selected images, graphics or objects to block the processed image to extract the area of ​​interest or structural features). Combined with segmentation prompts, it can be transferred to new images for segmentation and output predicted masks with zero samples, and has strong generalization ability on common images.

[0065] Reference Figure 2 , Figure 2 This is a diagram of the main components of the SAM model involved in the first embodiment of the control room cabinet door opening and closing status identification method of the present invention. Figure 2 In the paper, the image encoder uses a pre-trained vision transformer (ViT) to process high-resolution image input; the prompt encoder is used to input prompts such as points, boxes, masks, etc.; the mask decoder can effectively map image embedding and prompt embedding to output mask.

[0066] Reference Figure 3 , Figure 3 This is a structural diagram of a mask decoder involved in the first embodiment of the method for identifying the opening and closing status of a cabinet door in a control room of the present invention. Figure 3 In , the mask decoder uses a 2-layer decoder, and each decoder layer performs 4 steps, including self-attention mechanism, multilayer perceptron (MLP), and 2 cross attention mechanisms.

[0067] In the pre-training stage, the SAM model simulates a series of prompts for each training sample and compares the predicted mask with the true situation, estimating the count metric score I between each predicted mask and the object it covers. oU , so as to sort the output masks, the formula is:

[0068]

[0069] Where, T P is the number of pixels correctly predicted as foreground, F p is the number of background pixels that are incorrectly predicted as foreground pixels, F N is the number of pixels in the foreground that are mistakenly predicted as background, I oU It reflects the intersection-over-union ratio between the predicted mask and the true mask, and can effectively evaluate the results of image segmentation.

[0070] The SAM model mainly uses the focal loss function LF Supervised mask prediction, which is a dynamically scaled cross entropy loss function that modulates the t ) γ The weight of samples that are easy to distinguish during training can be dynamically reduced. The formula is:

[0071] L F (p t )=-(1-p t ) γ lgp t ;

[0072] in,

[0073] Where, L F (p t ) is the focal loss function, p is the probability of the model predicting that it belongs to the foreground, ranging from 0 to 1, p t The probability of the model predicting a certain category, y is the representative value of the foreground or background, and takes values ​​of 1 and -1, representing the foreground and background respectively. γ ≥ 0 is an adjustable focus parameter. It can be understood that the larger the γ value, the more the model focuses on samples that are difficult to classify.

[0074] Therefore, the above SAM model can be used to achieve door instance segmentation. By dividing each door instance, the area to which each door belongs can be obtained. Next, anchor points are located for each door. The selection of anchor points should avoid the special control area of ​​the door and the area with occlusion. Figure 4 , Figure 4 Schematic diagram of the position of the cabinet door anchor point involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention, Figure 4 In the example, the cabinet doors are arranged in rows, and the cabinet door anchor point can be selected in the middle position above the cabinet door to avoid obstruction, which is also conducive to improving the accuracy of subsequent area recognition.

[0075] Step S20: Determine the door state recognition area of ​​each door using the FloodFill algorithm according to the anchor point coordinates of each door;

[0076] In this embodiment, the four-neighborhood filling method or the eight-neighborhood filling method in the FloodFill algorithm can be used to start from the anchor point coordinates of each cabinet door, and expand and fill in the adjacent unfilled areas in a breadth-first search or depth-first search traversal manner to determine the cabinet status identification area of ​​each cabinet door.

[0077] The FloodFill algorithm, also known as the flood filling algorithm, can be used to fill areas with similar colors, similar to the paint bucket function. The algorithm starts from a seed point and searches for pixels with similar colors around it and fills them. The search method is similar to the traversal of a graph, and can be a depth traversal (Depth First Search, DFS) or a breadth traversal (Broad First Search, BFS). Through continuous iterations, the connected area where the seed point is located will be filled with a specified color to distinguish it from the surrounding area. The FloodFill algorithm has the function of finding connected areas and segmenting images. In order to prevent filling to the boundary, a mask can be made before filling the target area. The mask will speed up the filling process because the FloodFill algorithm can be set to only process the area covered by the mask.

[0078] It should be noted that the FloodFill algorithm can choose to fill in four or eight neighborhoods. Figure 5 , Figure 5 Schematic diagram of the filling method involved in the first embodiment of the method for identifying the opening and closing status of a cabinet door in a control room of the present invention, Figure 5 In the example, let f(i, j) be the pixel currently being traversed. The four-neighborhood filling algorithm will search for four adjacent pixels f(i, j-1), f(i, j+1), f(i-1, j), and f(i+1, j) in turn. If these four pixels are not filled and are close in color to the seed point, they will be filled. The eight-neighborhood filling algorithm differs from the four-neighborhood filling algorithm in that it adds four adjacent pixels f(i-1, j-1), f(i-1, j+1), f(i+1, j-1), and f(i+1, j+1) to the four-neighborhood filling algorithm.

[0079] In a specific implementation, the door opening and closing state can be identified by utilizing the size change of the door area when the door is open and closed. That is, the door state area can be obtained based on the FloodFill algorithm. Since there is a significant difference between the door opening and closing states, the door state area has different sizes. Figure 6 , Figure 6 This is a cabinet door state area recognition effect diagram involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention, Figure 6 In the FloodFill algorithm, the state area of ​​each door is obtained by using the anchor point. Figure 6 (a) is the state area recognition effect diagram of the cabinet door 2, Figure 6 (b) is the state area recognition effect diagram of the cabinet door 4, Figure 6 (a) is a state area recognition effect diagram for correctly identifying the cabinet door 2, but Figure 6(b) It can be seen that because the camera is tilted, the edges between the cabinet doors are not obvious enough, resulting in the cabinet door state area covering the cabinet doors next to it. Further adjustment is needed. Figure 6 (b) The cabinet door is processed.

[0080] Step S30: using the Canny algorithm to perform single door mask processing on the cabinet door state recognition area of ​​each cabinet door to obtain the single door state recognition area of ​​each cabinet door;

[0081] It should be noted that the Canny operator detects edges in a very robust way. It can extract image edges without interfering with image features and separate noise from the input image. It is currently the most practical and functional detection operator. The basic idea is to first use a two-dimensional Gaussian filter template to smooth the image, then use the derivative operator to calculate the gradient of the pixel grayscale in the x and y directions, and calculate the amplitude and direction of the gradient. Then, perform non-maximum suppression to find the maximum value of the pixel in the gradient direction to determine whether to provisionally define the central pixel as an edge point. Finally, perform double threshold selection and make a final judgment on the provisional edge point to select the final edge.

[0082] In this embodiment, the door state recognition region of each door may be first subjected to grayscale conversion processing and Gaussian filtering processing to obtain a filtered grayscale image of the door state recognition region of each door.

[0083] In the Canny algorithm, a filter is first used to remove the noise in the image after the grayscale conversion process, that is, the filter is used to convolve with the image. The purpose of filtering is to eliminate noise. The noise in the image corresponds to the high-frequency part of the image, so the noise can be eliminated in the frequency domain. Usually, edge detection and filtering are in conflict, because after filtering, the image is pixel-weighted averaged according to the template, which reduces the noise but blurs the edge of the image at the same time, resulting in increased uncertainty in locating the edge. Therefore, it is necessary to produce a compromise between the two. The kernel determined by the Gaussian function can resist noise interference and will not have much impact on the sensitivity of edge detection. The two-dimensional Gaussian kernel used in this embodiment is defined as follows:

[0084]

[0085] Where K2 represents a two-dimensional Gaussian function, σ represents the standard deviation of the filter, and (x, y) represents a pixel.

[0086] It is easy to understand that if the template size is too large, the edge details will be filtered out and the positioning accuracy will be low. If the template size is too small, there will be large noise. The standard deviation σ represents the degree of discreteness of the data. If σ is small, the central coefficient of the generated template is large, and the surrounding coefficients are small, so the smoothing effect on the image is not very obvious; on the contrary, if σ is large, the coefficients of the generated template are not very different, which is more similar to the mean template, and the smoothing effect on the image is more obvious. This embodiment uses a Gaussian convolution kernel H with a size of 3*3, a 3*3 window in the image is A, and the filtered pixel point is r. The Gaussian filtering calculation process is as follows:

[0087]

[0088] Where r represents the pixel for Gaussian filtering, H represents the 3*3 Gaussian convolution kernel, A represents the 9 pixels in the 3*3 window in the image, and the central pixel is set to e. There are eight pixels in its neighborhood, which are set to a, b, c, d, f, g, h, and i.

[0089] Next, the Sobel operator is used to calculate the gradient intensity and gradient direction of each pixel in the filtered grayscale image, and non-maximum suppression processing is performed according to the gradient intensity and gradient direction of each pixel to obtain local maximum pixel points.

[0090] It is easy to understand that the edges in the image can be in any direction. The gradient size G and gradient direction θ of each pixel can be calculated by calculating the first-order derivatives in the horizontal and vertical directions of each pixel, and then using the following two formulas.

[0091]

[0092] In the formula, G represents the gradient intensity of the pixel point, θ represents the gradient direction of the pixel point, and G x Represents the gradient value in the horizontal direction, G y Represents the gradient value in the vertical direction, the positive direction of x is from left to right (i.e. horizontal direction), and the positive direction of y is from top to bottom (i.e. vertical direction).

[0093] It should be noted that the first-order derivatives of the horizontal gradient and the vertical gradient can be obtained by filtering with operators such as Roberts, Prewitt, and Sobel. In this embodiment, the Sobel operator is preferably used to replace the traditional 2*2 template to reduce the generation of isolated edge points and pseudo edge points.

[0094] Among them, Roberts operator is one of the oldest edge detectors in digital image processing, using the mask pair g in the following formula: x and g y The first-order derivative of is numerically approximated.

[0095]

[0096] In the formula, g x is the gradient operator corresponding to the Roberts operator in the x direction, g y is the gradient operator corresponding to the Roberts operator in the y direction.

[0097] From this template, we can see that the Roberts operator can detect steeper edges at 45° and 135°. However, it has no smoothing effect and is sensitive to noise. Although the Roberts mask of size 2x2 is relatively simple to implement, it is not symmetrical relative to the center point. It is more reasonable to detect edges symmetrically, which requires an odd number of templates, the minimum of which is 3*3. Such templates provide more information on the direction of finding the edge because they take into account the influence of the pixels around the center point.

[0098] The 3*3 template used by the Prewitt operator is as follows:

[0099]

[0100] In the formula, g x is the gradient operator corresponding to the Prewitt operator in the x direction, g y is the gradient operator corresponding to the Prewitt operator in the y direction. x The mask can be constructed as the convolution of two filters, i.e. a vertical vector [1 0 1] T and a horizontal vector [1 0 1]. The same reasoning applies to g y Therefore, these operators can be interpreted as the successive use of a horizontal low-pass filter and a vertical derivative filter. As a result, they exhibit smoothing properties in the direction perpendicular to the gradient and are noise suppressive, but have poor pixel-averaged edge localization.

[0101] The Sobel operator is a slight change from the Prewitt operator, using a larger weight on the center coefficient. It can provide better image smoothing, but the detected edges are prone to be multiple pixels wide. As shown in the following formula:

[0102]

[0103] In the formula, g x is the gradient operator corresponding to the Sobel operator in the x direction, g y The Sobel operator corresponds to the gradient operator in the y direction.

[0104] Assume that the image window A is the central pixel e, and convolve A with the horizontal and vertical templates of Sobel respectively, then the horizontal gradient G of the central pixel e can be obtained. x and the vertical gradient G y , the calculation process is as follows:

[0105]

[0106] In the formula, the Sobel operator is two 3×3 matrices, namely S x and S y , S x Used to calculate the image x-direction gradient matrix G x , that is, G x Represents the gradient value in the horizontal direction, S y Used to calculate the image y-direction gradient matrix G y , that is, G y Represents the gradient value in the vertical direction, A represents the 9 pixels in a 3*3 window in the image (let the central pixel be e, and there are eight pixels in its neighborhood, let them be a, b, c, d, f, g, h, i), and * represents the cross-correlation operation.

[0107] Then, the single door edge of the door state recognition area of ​​each door is identified by using the Canny algorithm according to the local maximum pixel point using a double threshold method.

[0108] It should be noted that the double threshold selection is the last step in the Canny algorithm. It sets two thresholds TH1 and TH2 (TH1>TH2) to filter the local maximum pixel x' generated in the non-maximum suppression to achieve the purpose of noise reduction and edge continuity. In this process, the pixels are filtered into three sets: the strong edge set S s With weak edge set S w and discard edge set S d , respectively defined as follows:

[0109] {S w |TH1>x'>TH2};

[0110] {S s |x'>TH1};

[0111] {S d |TH2>x'};

[0112] In the formula, x' is the local maximum pixel, S s is a strong edge set, S w is a weak edge set, S d To discard the edge set, TH1 and TH2 are double thresholds set in the Canny edge detection algorithm, and TH1>TH2.

[0113] Reference Figure 7 , Figure 7 This is a diagram of the edge connection process involved in the first embodiment of the method for identifying the opening and closing status of a control room door of the present invention. Figure 7 When {S s} split point, then search in the 3*3 search window to see if there is {S w}, if so, connect it to {S s} collection, Figure 7 The final set of strong edge points shown is the single door edge of the door state recognition area of ​​each door that has been identified.

[0114] Therefore, the door status recognition area of ​​each door can be masked based on the single door edge of the door status recognition area of ​​each door to obtain the single door status recognition area of ​​each door. Figure 6 (b) shows the recognition result of the area of ​​the adjacent cabinet door. Therefore, the interference items in the image can be preprocessed first to obtain a single-color state effect image; then the mask of the single cabinet door is obtained based on the Canny algorithm, and other interference items are further eliminated to obtain the state area of ​​the single cabinet door. Figure 8 , Figure 8 This is a comparison diagram of the door mask processing effects involved in the first embodiment of the control room door opening and closing status identification method of the present invention. Figure 8 middle, Figure 8 (a) is the recognition effect diagram before the cabinet door mask treatment. Figure 8 (b) The recognition effect of the cabinet door after mask processing. Figure 8 (a) It can be seen that because there are many buttons and handles on the cabinet door, it is not conducive to the calculation of the cabinet door state area, and the recognition results of the adjacent cabinet door area will be included during recognition. Figure 8 (b) The state area of ​​a single cabinet door can be directly obtained.

[0115] In a specific implementation, in order to further improve the accuracy of identifying the opening and closing status of cabinet doors, the single cabinet door status identification area of ​​each cabinet door can also be subjected to corner point detection by using the Corner Harris corner point detection algorithm to obtain the corner point information of the single cabinet door status identification area of ​​each cabinet door. Then, the cabinet door area of ​​the single cabinet door status identification area of ​​each cabinet door is calculated based on the single cabinet door status identification area of ​​each cabinet door and the corner point information. Based on the cabinet door area of ​​the single cabinet door status identification area of ​​each cabinet door, it is determined whether there is a cabinet door in an open state.

[0116] Reference Fig. 9 , Fig. 9It is a schematic diagram of CornerHarris corner point detection involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention. Fig. 9 In the image, Corner Harris corner point detection has the property of rotation invariance. When the red frame is moved, no matter in which direction it is offset, the pixel values ​​in the frame will change greatly. Then the corner points in the area inside the red frame are the detected corner points. When the black frame is moved, the pixel values ​​in the horizontal direction do not change much, but they will change greatly in the vertical direction, or vice versa. This is generally called the edge area. When the blue frame is moved, no matter it moves horizontally or vertically, the pixel values ​​in the frame do not change much. This is the internal area.

[0117] Corner Harris corner detection moves the window in all directions (u, v) and then calculates the sum of all differences. The formula is as follows:

[0118]

[0119] Where E(u, v) is the sum of the differences after the window is moved, and the window function is w(x, y), which can be a normal rectangular window or a Gaussian window that gives different weights to each pixel.

[0120] To maximize the value of E(u, v) in corner point detection, the value of the second term on the right side of the equation must be maximized. Taylor expansion of the above equation and mathematical conversion yield the following formula:

[0121]

[0122] In the formula, I x and I y are the derivatives of the image in the x and y directions.

[0123] Finally, it is necessary to determine whether the window contains corner points for scoring. The formula is as follows:

[0124] R = det(M) - k(trace(M)) 2 ;

[0125] det(M)=λ1λ2;

[0126] trace(M = λ1 + λ2;

[0127] Where det(M) is the determinant of the structure matrix (M), which represents the product of its eigenvalues; trace(M) is the trace of the structure matrix, which represents the sum of its eigenvalues; k is an empirical parameter, usually ranging from 0.04 to 0.06, which is used to adjust the corner response function; λ1 and λ2 are the eigenvalues ​​of the matrix M, which can be used to determine whether a region is a corner, boundary or plane. For example, when λ1 and λ2 are both small, |R| is also small, and this region is a flat region; when λ1>>λ2 or λ1<<λ2, R is less than 0, and this region is an edge; when λ1 and λ2 are both large, and λ1~λ2 (λ1 and λ2 are approximately equal), R is also large, indicating that this region is a corner.

[0128] Reference Fig.10 , Fig.10 This is a comparison diagram of the CornerHarris corner point detection effects involved in the first embodiment of the control room cabinet door opening and closing state recognition method of the present invention. Fig.10 middle, Fig.10 (a) is the Corner Harris corner point effect diagram when the cabinet door is open. Fig.10 (b) is the Corner Harris corner point effect diagram when the door is closed. Corner Harris is used to detect corner points of the door mask image. There are different corner point information when the door is open or closed. Fig.10 (b) It can be seen that when the cabinet door is closed, there is more corner point information than when it is open, and the cabinet door area is larger in the closed state.

[0129] Step S40: judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door;

[0130] It is easy to understand that the area of ​​the cabinet door in different opening and closing states is different. We select pictures of cabinet doors opened and closed at different angles for testing. Refer to Table 1, which is a table of cabinet door areas in different cabinet door opening and closing states involved in the first embodiment of the control room cabinet door opening and closing state identification method of the present invention. It can be seen from Table 1 that there is a significant difference in the size of the cabinet door when it is opened and closed.

[0131] Table 1 Cabinet door area under different cabinet door opening and closing conditions

[0132]

[0133] In this embodiment, the open and closed state of the cabinet door is identified using algorithms such as FloodFill and Canny, and the area of ​​the current cabinet door state is calculated. In a specific implementation, it can be determined whether there is a cabinet door in an open state based on whether the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door is less than or equal to the preset cabinet door area threshold. Among them, the preset cabinet door area threshold can be set according to actual needs, and this embodiment does not limit this. For example, it can be set to 2500000, that is, when the cabinet door state area is less than or equal to 2500000, the cabinet door is determined to be in an open state, otherwise it is in a closed state. Refer to Fig.11 , Fig.11 This is a comparison diagram of the cabinet door opening and closing status recognition effects involved in the first embodiment of the control room cabinet door opening and closing status recognition method of the present invention. Fig.11 middle, Fig.11 (a) The cabinet door is recognized as open. Fig.11 (b) The cabinet door is identified as being in a closed state.

[0134] Step S50: When there is a cabinet door in an open state, an alarm process is performed on the cabinet door in an open state.

[0135] Reference Fig.12 , Fig.12 This is a technical roadmap involved in the first embodiment of the method for identifying the opening and closing status of a control room door of the present invention. Fig.12 In the example, each cabinet door is first divided into instances to obtain the area to which each cabinet door belongs; then, each cabinet door is anchored, and the selection of anchor points avoids the special control area of ​​the cabinet door and the area where there is occlusion; then, based on the anchor point coordinates of each cabinet door, the FloodFill algorithm is used to obtain the state area of ​​each cabinet door; then the Canny algorithm is used to perform cabinet door masking to eliminate other areas that do not belong to the current cabinet door; further, the corner points of the cabinet door can be detected to determine the corner point information of the current cabinet door; since the cabinet door area is different under different cabinet door states, the area threshold can be used to determine whether the cabinet door is in an open state; for the cabinet door that is currently in an open state, the opening time can be calculated, and when the opening time is greater than or equal to the preset opening time, the cabinet door with an opening time greater than or equal to the preset opening time is alarmed. Among them, the preset opening time can be set according to actual needs, and this embodiment does not limit this. For example, it can be set to 10 minutes, that is, when the time exceeds 10 minutes, an alarm is processed.

[0136] In the specific implementation, alarm processing can be performed on the cabinet doors whose opening time is greater than or equal to the preset opening time based on the obtained area to which each cabinet door belongs. For example, alarm information can be generated based on the area to which the cabinet doors whose opening time exceeds 10 minutes belong, so as to remind the monitoring personnel of the monitoring screen to conduct comprehensive emergency dispatch and command in time when the alarm is sounded, so that the monitoring personnel do not need to keep an eye on the monitoring screen all day for dynamic monitoring of the monitoring control room, but only need to conduct comprehensive emergency dispatch and command in time when the system alarms, which effectively reduces the labor intensity and improves the monitoring effectiveness. On the other hand, it also effectively makes up for the weakness of human eye monitoring and improves the monitoring accuracy. In addition, the monitoring end adopts the "alarm trigger" video acquisition mode, which can only record and collect the target behavior specified by the program (such as the cabinet door is in the open state), effectively improving the efficiency of post-review of the video, reducing the storage redundancy of irrelevant videos, saving hard disk storage space, and further improving the management efficiency of the monitoring end.

[0137] In this embodiment, the cabinet door area in the control room monitoring screen is segmented and anchored to obtain the coordinates of the points of each cabinet door in the cabinet door area; the cabinet door state identification area of ​​each cabinet door is determined by the FloodFill algorithm according to the anchor point coordinates of each cabinet door; the cabinet door state identification area of ​​each cabinet door is masked by the Canny algorithm to obtain the single cabinet door state identification area of ​​each cabinet door; it is determined whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door; when there is a cabinet door in an open state, an alarm is processed for the cabinet door in an open state. By using algorithms such as FloodFill and Canny to identify the opening and closing state of the cabinet door area in the control room monitoring screen, and combining the area calculation of the current cabinet door opening and closing state, the accuracy of the control room cabinet door opening and closing state identification is further improved, the occurrence of misidentification of the cabinet door opening and closing state is avoided, the safety hazards of the monitoring scene are reduced, and it is also beneficial to the security management of the monitoring scene.

[0138] Reference Fig.13 , Fig.13 This is a structural block diagram of the first embodiment of the control room cabinet door opening and closing status recognition system of the present invention.

[0139] like Fig.13 As shown, the control room cabinet door opening and closing state recognition system proposed in the embodiment of the present invention includes:

[0140] Anchor point positioning module 100, used to perform instance segmentation and anchor point positioning on the cabinet door area in the control room monitoring screen to obtain the coordinates of the points of each cabinet door in the cabinet door area;

[0141] The area determination module 200 is used to determine the door state recognition area of ​​each door using the FloodFill algorithm according to the anchor point coordinates of each door;

[0142] The mask processing module 300 is used to perform single-door mask processing on the cabinet door state recognition area of ​​each cabinet door by using the Canny algorithm to obtain the single-door state recognition area of ​​each cabinet door;

[0143] A state determination module 400, for determining whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door;

[0144] The cabinet door alarm module 500 is used to perform alarm processing on the cabinet door in the open state when there is a cabinet door in the open state.

[0145] Optionally, the state judgment module 400 is further used to perform corner point detection on the single door state recognition area of ​​each door by using a Corner Harris corner point detection algorithm to obtain corner point information of the single door state recognition area of ​​each door;

[0146] The state judgment module 400 is also used to calculate the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door according to the single cabinet door state identification area of ​​each cabinet door and the corner point information, and to judge whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door.

[0147] Optionally, the area determination module 200 is also used to use the four-neighborhood filling method or the eight-neighborhood filling method in the FloodFill algorithm to start from the anchor point coordinates of each cabinet door, and to expand and fill in adjacent unfilled areas in a breadth-first search or depth-first search traversal manner to determine the cabinet status identification area of ​​each cabinet door.

[0148] Optionally, the mask processing module 300 is further used to perform grayscale conversion processing and Gaussian filtering processing on the cabinet door state recognition area of ​​each cabinet door to obtain a filtered grayscale image of the cabinet door state recognition area of ​​each cabinet door;

[0149] The mask processing module 300 is further used to calculate the gradient intensity and gradient direction of each pixel in the filtered grayscale image using a Sobel operator, and perform non-maximum suppression processing according to the gradient intensity and gradient direction of each pixel to obtain a local maximum pixel;

[0150] The mask processing module 300 is further used to identify the single door edge of the door state recognition area of ​​each door by using the Canny algorithm according to the local maximum pixel point using a double threshold method;

[0151] The mask processing module 300 is further used to perform single door mask processing on the door status identification area of ​​each door based on the single door edge of the door status identification area of ​​each door to obtain the single door status identification area of ​​each door.

[0152] Optionally, the state judgment module 400 is further used to judge whether there is a cabinet door in an open state according to whether the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door is less than or equal to a preset cabinet door area threshold;

[0153] The cabinet door alarm module 500 is further used to calculate the opening time of the cabinet door in the open state when there is a cabinet door in the open state;

[0154] The cabinet door alarm module 500 is also used to perform alarm processing on the cabinet door whose opening time is greater than or equal to the preset opening time when the opening time is greater than or equal to the preset opening time.

[0155] In this embodiment, the cabinet door area in the control room monitoring screen is segmented and anchored to obtain the coordinates of the points of each cabinet door in the cabinet door area; the cabinet door state identification area of ​​each cabinet door is determined by the FloodFill algorithm according to the anchor point coordinates of each cabinet door; the cabinet door state identification area of ​​each cabinet door is masked by the Canny algorithm to obtain the single cabinet door state identification area of ​​each cabinet door; it is determined whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state identification area of ​​each cabinet door; when there is a cabinet door in an open state, an alarm is processed for the cabinet door in an open state. By using algorithms such as FloodFill and Canny to identify the opening and closing state of the cabinet door area in the control room monitoring screen, and combining the area calculation of the current cabinet door opening and closing state, the accuracy of the control room cabinet door opening and closing state identification is further improved, the occurrence of misidentification of the cabinet door opening and closing state is avoided, the safety hazards of the monitoring scene are reduced, and it is also beneficial to the security management of the monitoring scene.

[0156] Other embodiments or specific implementation methods of the control room cabinet door opening and closing state recognition system of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0157] The present invention is not limited to the above-mentioned optional implementation modes. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that fall within the scope defined by the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A method for identifying the opening and closing status of a control room door, characterized in that: The following steps are involved: Perform instance segmentation and anchor point positioning on the cabinet door area in the control room monitoring screen to obtain the coordinates of the points of each cabinet door in the cabinet door area; Determine the door state recognition area of ​​each door using the FloodFill algorithm according to the anchor point coordinates of each door; The Canny algorithm is used to perform single door mask processing on the cabinet state recognition area of ​​each cabinet door to obtain the single door state recognition area of ​​each cabinet door; Determine whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door; When there is a cabinet door in an open state, an alarm process is performed on the cabinet door in an open state.

2. A control room door opening and closing state recognition method according to claim 1, characterized in that: Before the step of judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door, the following steps are also included: Performing corner point detection on the single door state recognition area of ​​each door using the Corner Harris corner point detection algorithm to obtain corner point information of the single door state recognition area of ​​each door; Accordingly, the step of judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door includes: The door area of ​​the single door status identification area of ​​each door is calculated based on the single door status identification area and corner point information of each door, and it is determined whether there is a door in an open state based on the door area of ​​the single door status identification area of ​​each door.

3. The method for identifying the opening and closing status of a control room door according to claim 1, characterized in that: The step of determining the door state recognition area of ​​each door by using the FloodFill algorithm according to the anchor point coordinates of each door includes: The four-neighborhood filling method or the eight-neighborhood filling method in the FloodFill algorithm is used to start from the anchor point coordinates of each cabinet door, and expand and fill the adjacent unfilled areas in a breadth-first search or depth-first search traversal manner to determine the cabinet state recognition area of ​​each cabinet door.

4. The method for identifying the opening and closing status of a control room door according to claim 1, characterized in that: The step of performing single door mask processing on the cabinet door state recognition area of ​​each cabinet door by using the Canny algorithm to obtain the single door state recognition area of ​​each cabinet door includes: Performing grayscale conversion processing and Gaussian filtering processing on the cabinet door state recognition area of ​​each cabinet door to obtain a filtered grayscale image of the cabinet door state recognition area of ​​each cabinet door; The Sobel operator is used to calculate the gradient intensity and gradient direction of each pixel in the filtered grayscale image, and non-maximum suppression processing is performed according to the gradient intensity and gradient direction of each pixel to obtain the local maximum pixel; The single door edge of the door state recognition area of ​​each door is identified by using the Canny algorithm according to the local maximum pixel point using the double threshold method; The door status recognition area of ​​each door is subjected to single door masking processing based on the single door edge of the door status recognition area of ​​each door to obtain the single door status recognition area of ​​each door.

5. The method for identifying the opening and closing status of a control room door according to claim 1, characterized in that: The step of judging whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door comprises: Determining whether there is a cabinet door in an open state according to whether the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door is less than or equal to a preset cabinet door area threshold; When there is a cabinet door in an open state, the step of performing an alarm process on the cabinet door in an open state includes: When there is a cabinet door in an open state, the opening time of the cabinet door in an open state is calculated; When the opening time is greater than or equal to the preset opening time, an alarm is processed for the cabinet door whose opening time is greater than or equal to the preset opening time.

6. A control room door opening and closing state recognition system, used in the control room door opening and closing state recognition method according to claim 1, characterized in that: include: Anchor point positioning module, used to perform instance segmentation and anchor point positioning on the cabinet door area in the control room monitoring screen to obtain the coordinates of the points of each cabinet door in the cabinet door area; An area determination module, used to determine the door state recognition area of ​​each door using a FloodFill algorithm according to the anchor point coordinates of each door; A mask processing module, used for performing single-door mask processing on the cabinet state recognition area of ​​each cabinet door by using the Canny algorithm to obtain the single-door state recognition area of ​​each cabinet door; A state judgment module, used to judge whether there is a cabinet door in an open state according to the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door; The cabinet door alarm module is used to perform alarm processing on the cabinet door in the open state when there is a cabinet door in the open state.

7. The control room door opening and closing state recognition system according to claim 6 is characterized in that: The state judgment module is further used to perform corner point detection on the single door state recognition area of ​​each door by using the Corner Harris corner point detection algorithm to obtain corner point information of the single door state recognition area of ​​each door; The status judgment module is also used to calculate the door area of ​​the single door status identification area of ​​each door based on the single door status identification area and corner point information of each door, and to judge whether there is a door in an open state based on the door area of ​​the single door status identification area of ​​each door.

8. The control room door opening and closing state recognition system according to claim 6 is characterized by: The area determination module is also used to use the four-neighborhood filling method or the eight-neighborhood filling method in the FloodFill algorithm to start from the anchor point coordinates of each cabinet door, and to expand and fill the adjacent unfilled areas in a breadth-first search or depth-first search traversal manner to determine the cabinet state identification area of ​​each cabinet door.

9. The control room door opening and closing state recognition system according to claim 6, characterized in that: The mask processing module is further used to perform grayscale conversion processing and Gaussian filtering processing on the cabinet door state recognition area of ​​each cabinet door to obtain a filtered grayscale image of the cabinet door state recognition area of ​​each cabinet door; The mask processing module is also used to calculate the gradient intensity and gradient direction of each pixel in the filtered grayscale image using the Sobel operator, and perform non-maximum suppression processing according to the gradient intensity and gradient direction of each pixel to obtain local maximum pixel points; The mask processing module is also used to identify the single cabinet door edge of the cabinet door state recognition area of ​​each cabinet door by using the Canny algorithm according to the local maximum pixel point using the double threshold method; The mask processing module is also used to perform single door mask processing on the door status identification area of ​​each cabinet door based on the single door edge of the door status identification area of ​​each cabinet door to obtain the single door status identification area of ​​each cabinet door.

10. The control room door opening and closing state recognition system according to claim 6, characterized in that: The state judgment module is further used to judge whether there is a cabinet door in an open state according to whether the cabinet door area of ​​the single cabinet door state recognition area of ​​each cabinet door is less than or equal to a preset cabinet door area threshold; The cabinet door alarm module is further used to calculate the opening time of the cabinet door in the open state when there is a cabinet door in the open state; The cabinet door alarm module is also used to perform alarm processing on the cabinet door whose opening time is greater than or equal to the preset opening time when the opening time is greater than or equal to the preset opening time.

Images