A dual-factor verification method and system for circuit breaker status
By constructing a two-factor verification model for circuit breaker status based on multi-scale feature fusion, and utilizing convolutional neural networks to extract and fuse circuit breaker image features, the problem of difficulty in identifying the status of pump station circuit breaker trolleys in existing technologies is solved, achieving rapid and accurate status monitoring and reducing safety risks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-21
- Publication Date
- 2026-04-07
AI Technical Summary
Existing technologies are insufficient for effectively monitoring the operating status of circuit breakers in pumping stations, especially for identifying changes in the trolley in the longitudinal direction. This results in the inability to conduct continuous and long-term monitoring, leading to safety risks and high costs.
A two-factor verification model for circuit breaker status is constructed by fusing multi-scale features. By extracting image features of indicator lights and truck windows through convolutional neural networks and combining text recognition and status classification, the accurate determination of circuit breaker status can be achieved.
It enables rapid and accurate monitoring of circuit breaker status, reduces the safety risks and costs of manual inspections, and improves the continuity and accuracy of monitoring.
Smart Images
Figure CN116486388B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of circuit breaker status detection and pattern recognition, and particularly relates to a two-factor verification method and system for circuit breaker status. Background Technology
[0002] Circuit breakers, with their overload and short-circuit protection functions, serve as switches and protection devices in pump station circuit systems. Regular monitoring of their operational status is crucial for ensuring the normal operation of pump stations. Currently, in northern my country, the operational status monitoring of circuit breakers in pump stations primarily relies on periodic manual inspections, simultaneously verifying whether the circuit breaker indicator lights match the status of the circuit breaker trolley displayed in the cabinet window. Traditional methods generally suffer from high costs and discontinuous monitoring, and the frequent entry and exit of inspectors in high-voltage energized rooms poses significant safety risks. These factors all contribute to the inability to continuously and sustainably monitor the operational status of circuit breakers, making it difficult to guarantee the safe operation of pump station circuit breakers.
[0003] In recent years, with the development of deep learning technology, machine vision technology has made groundbreaking progress in various fields. Utilizing machine vision technology to observe the status of circuit breaker trolleys is an effective way to monitor their operational status. Convolutional Neural Networks (CNNs), as one of the most popular methods in deep learning, have far surpassed traditional image recognition methods in terms of recognition performance when dealing with large amounts of complex data. Therefore, they have been widely used in the field of machine vision and have achieved remarkable results. The powerful image feature extraction and analysis capabilities of CNNs have brought opportunities for machine vision-based two-factor verification of circuit breaker status.
[0004] The difference between the pushed-in and pushed-out states of a circuit breaker during operation lies in the change of the circuit breaker trolley in the depth direction, and the change in depth is not significant. However, existing deep learning-based image classification algorithms have difficulty capturing changes in the depth direction at an angle perpendicular to the shooting plane, making it difficult to directly use object detection deep learning algorithms to determine the trolley state. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention proposes a two-factor verification method for circuit breaker status, thereby resolving these issues.
[0006] The first aspect of this invention discloses a two-factor verification method for circuit breaker status, the method comprising:
[0007] Step S1: Construct a two-factor verification model for circuit breaker status by fusing multi-scale features to identify the circuit breaker's operating status through input images of indicator lights and truck windows of the circuit breaker at the pumping station.
[0008] Step S2: Acquire the indicator light image and the vehicle window image, and preprocess them;
[0009] Step S3: Input the preprocessed indicator light image and the vehicle window image into the circuit breaker state two-factor verification model fused with multi-scale features, and train it.
[0010] Step S4: Input the images of the handcart window and indicator lights of the circuit breaker to be identified into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification results of the pump station.
[0011] According to the method of the first aspect of the present invention, in step S1, the circuit breaker state two-factor verification model of multi-scale feature fusion includes: a feature extraction network, a feature fusion network and a prediction and discrimination network;
[0012] The feature extraction network extracts features from indicator light images and vehicle window images through convolutional layers and multi-layer convolutional attention modules.
[0013] The feature fusion network fuses the features output by the attention module of each convolutional layer through an up-convolutional module and an upsampling process to obtain fused features.
[0014] The prediction and discrimination network includes: a text recognition module for vehicle window images and a status classification module for indicator light images;
[0015] The fused features are input into the text recognition module of the handcart window image to obtain the coordinates of the position of the text detection target box, and then the position coordinates are used to determine whether the circuit breaker handcart is in the pushed-out state or the pushed-in state.
[0016] The fused features are input into the state classification module of the indicator light image to determine whether the circuit breaker truck is in the pushed-out or pushed-in state.
[0017] The predictive discrimination network compares the recognition results of the text recognition module of the vehicle window image with the recognition results of the status classification module of the indicator light image to determine whether the circuit breaker is operating normally.
[0018] According to a first aspect of the present invention, in step S1, the method by which the feature extraction network extracts features from the indicator light image and the vehicle window image through convolutional layers and multi-layer convolutional attention modules includes:
[0019] The indicator light image and the vehicle window image are first input into a convolutional layer. The output of the convolutional layer is then input into a five-segment cascaded convolutional attention module to obtain the features of the indicator light image and the vehicle window image.
[0020] According to a method of a first aspect of the present invention, in step S1, the feature fusion network fuses the features output by the convolutional attention modules of each layer through an up-convolutional module and an upsampling process to obtain fused features. The method includes:
[0021] First, the output feature map of the fifth stage convolutional attention module is concatenated dimensionally with the output feature map of the fourth stage convolutional attention module to obtain a first concatenated feature map. This first concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a first fused feature map. Next, the first fused feature map is concatenated dimensionally with the output feature map of the third stage convolutional attention module to obtain a second concatenated feature map. This second concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a second fused feature map. Finally, the second fused feature map is concatenated dimensionally with the output feature map of the second stage convolutional attention module to obtain a third concatenated feature map. This third concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a third fused feature map. Finally, the third fused feature map is concatenated dimensionally with the output feature map of the first stage convolutional attention module to obtain a fourth concatenated feature map. This fourth concatenated feature map undergoes an up-convolution module and an upsampling process to obtain the final fused feature map.
[0022] According to a first aspect of the present invention, in step S1, the method of inputting the fused features into the text recognition module of the vehicle window image to obtain the coordinates of the position of the text detection target box includes:
[0023] The fused features are compressed through a convolutional layer to obtain a region score representing the probability of the center region of a single character and an affinity score representing the center probability of the space between adjacent characters.
[0024] The region score and affinity score are used to filter the pixel regions, and the pixel regions with region scores and affinity scores greater than a preset threshold are selected as candidate regions for text. Then, word components are connected within the candidate regions for text. Finally, a quadrilateral is established by finding four vertices to enclose the connection components corresponding to each label with the smallest area, and the coordinates of the text detection target box are obtained.
[0025] According to the method of the first aspect of the present invention, in step S1, the method of determining whether the circuit breaker trolley is in an extended or extended state based on the coordinates of the position includes:
[0026] Based on the coordinates of the positions, the maximum area of the text detection target box in the pushed-in state and the minimum area of the text detection target box in the pushed-out state of each circuit breaker truck are statistically analyzed, and the average value of the two is taken. The area of the text detection target box in the truck window image is compared with the average value. If the area of the text detection target box in the current truck window image is greater than the average value, it is classified as the pushed-out state; otherwise, it is classified as the pushed-in state.
[0027] According to a first aspect of the present invention, the method of inputting the fused features into the state classification module of the indicator light image to obtain whether the circuit breaker truck is in an outgoing or incoming state includes:
[0028] The fused features are flattened to expand the two-dimensional feature map into a one-dimensional vector, and then passed through three fully connected layers and the ReLU nonlinear activation function to obtain two-dimensional classification features. Finally, the two-dimensional classification features are processed by softmax to obtain the discrimination results of the circuit breaker truck being pushed in and pushed out.
[0029] A second aspect of the present invention discloses a two-factor verification system for circuit breaker status, the system comprising:
[0030] The first processing module is configured to construct a circuit breaker status two-factor verification model that integrates multi-scale features to identify the circuit breaker's operating status by inputting the indicator light image and the truck window image of the pump station circuit breaker.
[0031] The second processing module is configured to acquire the indicator light image and the vehicle window image, and preprocess them;
[0032] The third processing module is configured to input the preprocessed indicator light image and the vehicle window image into the circuit breaker state two-factor verification model fused with multi-scale features, and train it.
[0033] The fourth processing module is configured to input the images of the handcart windows and indicator lights of the circuit breaker to be identified into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification results of the pump station.
[0034] According to a system of a second aspect of the present invention, the first processing module is configured such that the multi-scale feature fusion circuit breaker state two-factor verification model includes: a feature extraction network, a feature fusion network, and a prediction and discrimination network;
[0035] The feature extraction network extracts features from indicator light images and vehicle window images through convolutional layers and multi-layer convolutional attention modules.
[0036] The feature fusion network fuses the features output by the attention module of each convolutional layer through an up-convolutional module and an upsampling process to obtain fused features.
[0037] The prediction and discrimination network includes: a text recognition module for vehicle window images and a status classification module for indicator light images;
[0038] The fused features are input into the text recognition module of the handcart window image to obtain the coordinates of the position of the text detection target box, and then the position coordinates are used to determine whether the circuit breaker handcart is in the pushed-out state or the pushed-in state.
[0039] The fused features are input into the state classification module of the indicator light image to determine whether the circuit breaker truck is in the pushed-out or pushed-in state.
[0040] The predictive discrimination network compares the recognition results of the text recognition module of the vehicle window image with the recognition results of the status classification module of the indicator light image to determine whether the circuit breaker is operating normally.
[0041] According to a system of a second aspect of the present invention, the first processing module is configured such that the feature extraction network extracts features from the indicator light image and the vehicle window image through convolutional layers and multi-layer convolutional attention modules, including:
[0042] The indicator light image and the vehicle window image are first input into a convolutional layer. The output of the convolutional layer is then input into a five-segment cascaded convolutional attention module to obtain the features of the indicator light image and the vehicle window image.
[0043] According to a system of a second aspect of the present invention, the first processing module is configured such that the feature fusion network fuses the features output by the convolutional attention modules of each layer through an up-convolutional module and an upsampling process to obtain fused features including:
[0044] First, the output feature map of the fifth stage convolutional attention module is concatenated dimensionally with the output feature map of the fourth stage convolutional attention module to obtain a first concatenated feature map. This first concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a first fused feature map. Next, the first fused feature map is concatenated dimensionally with the output feature map of the third stage convolutional attention module to obtain a second concatenated feature map. This second concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a second fused feature map. Finally, the second fused feature map is concatenated dimensionally with the output feature map of the second stage convolutional attention module to obtain a third concatenated feature map. This third concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a third fused feature map. Finally, the third fused feature map is concatenated dimensionally with the output feature map of the first stage convolutional attention module to obtain a fourth concatenated feature map. This fourth concatenated feature map undergoes an up-convolution module and an upsampling process to obtain the final fused feature map.
[0045] According to a system of a second aspect of the present invention, the first processing module is configured to input the fused features into a text recognition module of the vehicle window image to obtain the coordinates of the position of the text detection target box, including:
[0046] The fused features are compressed through a convolutional layer to obtain a region score representing the probability of the center region of a single character and an affinity score representing the center probability of the space between adjacent characters.
[0047] The region score and affinity score are used to filter the pixel regions, and the pixel regions with region scores and affinity scores greater than a preset threshold are selected as candidate regions for text. Then, word components are connected within the candidate regions for text. Finally, a quadrilateral is established by finding four vertices to enclose the connection components corresponding to each label with the smallest area, and the coordinates of the text detection target box are obtained.
[0048] According to a system of a second aspect of the present invention, the first processing module is configured to, further determining whether the circuit breaker trolley is in an extended or extended state based on the coordinates of the position, includes:
[0049] Based on the coordinates of the positions, the maximum area of the text detection target box in the pushed-in state and the minimum area of the text detection target box in the pushed-out state of each circuit breaker truck are statistically analyzed, and the average value of the two is taken. The area of the text detection target box in the truck window image is compared with the average value. If the area of the text detection target box in the current truck window image is greater than the average value, it is classified as the pushed-out state; otherwise, it is classified as the pushed-in state.
[0050] According to a system of a second aspect of the present invention, the first processing module is configured such that the step of inputting the fused features into the state classification module of the indicator light image to determine whether the circuit breaker truck is in an outgoing or ingoing state includes:
[0051] The fused features are flattened to expand the two-dimensional feature map into a one-dimensional vector, and then passed through three fully connected layers and the ReLU nonlinear activation function to obtain two-dimensional classification features. Finally, the two-dimensional classification features are processed by softmax to obtain the discrimination results of the circuit breaker truck being pushed in and pushed out.
[0052] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a two-factor verification method for circuit breaker status according to any one of the first aspects of this disclosure.
[0053] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a two-factor verification method for circuit breaker status according to any one of the first aspects of this disclosure.
[0054] The solution proposed in this invention has the following beneficial effects:
[0055] 1. By making full use of the multi-scale feature fusion structure and attention mechanism, a two-factor verification model for circuit breaker status based on multi-scale feature fusion is proposed. A multi-input multi-output neural network is constructed, which has the characteristics of fast detection speed and high accuracy, and is used to realize two-factor verification of circuit breaker status.
[0056] 2. This paper applies text detection algorithms to image state classification tasks, transforming changes in the depth dimension of an image into changes in the horizontal dimension to determine the image state. The main focus is on providing a solution to the problem that two-dimensional image classification algorithms are difficult to apply to recognizing changes in the depth dimension of an image.
[0057] 3. The text detection part of the circuit breaker state two-factor verification model based on multi-scale feature fusion is more accurate and fits the text target box more closely, effectively solving the problems that manual inspection methods are easily affected by subjective factors and the accuracy and fit of existing text detection networks are insufficient. Attached Figure Description
[0058] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0059] Figure 1 A flowchart of a two-factor verification method for circuit breaker status according to an embodiment of the present invention;
[0060] Figure 2 A flowchart of a two-factor verification method for circuit breaker status according to an embodiment of the present invention;
[0061] Figure 3 This is a schematic diagram of the network structure of a circuit breaker state two-factor verification model based on multi-feature fusion according to an embodiment of the present invention;
[0062] Figure 4a This is a structural diagram of the attention mechanism module according to an embodiment of the present invention;
[0063] Figure 4b This is a structural diagram of the channel attention mechanism module according to an embodiment of the present invention;
[0064] Figure 4c This is a structural diagram of the spatial attention mechanism module according to an embodiment of the present invention;
[0065] Figure 5 This is a schematic diagram of the upper convolution module according to an embodiment of the present invention;
[0066] Figure 6 This is a structural diagram of a two-factor verification system for circuit breaker status according to an embodiment of the present invention;
[0067] Figure 7 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0069] The first aspect of this invention discloses a two-factor verification method for circuit breaker status. Figure 1This is a flowchart of a two-factor verification method for circuit breaker status according to an embodiment of the present invention, as follows: Figure 1 and Figure 2 As shown, the method includes:
[0070] Step S1: Construct a two-factor verification model for circuit breaker status by fusing multi-scale features to identify the circuit breaker's operating status through input images of indicator lights and truck windows of the circuit breaker at the pumping station.
[0071] Step S2: Acquire the indicator light image and the vehicle window image, and preprocess them;
[0072] Step S3: Input the preprocessed indicator light image and the vehicle window image into the circuit breaker state two-factor verification model fused with multi-scale features, and train it.
[0073] Step S4: Input the images of the handcart window and indicator lights of the circuit breaker to be identified into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification results of the pump station.
[0074] In step S1, a two-factor verification model for circuit breaker status is constructed by fusing multi-scale features to identify the circuit breaker's operating status through input images of the indicator lights and the truck windows of the circuit breaker at the pumping station.
[0075] In some embodiments, in step S1, the multi-scale feature fusion circuit breaker state two-factor verification model includes: a feature extraction network, a feature fusion network, and a prediction and discrimination network;
[0076] The feature extraction network extracts features from indicator light images and vehicle window images through convolutional layers and multi-layer convolutional attention modules.
[0077] The feature fusion network fuses the features output by the attention module of each convolutional layer through an up-convolutional module and an upsampling process to obtain fused features.
[0078] The prediction and discrimination network includes: a text recognition module for vehicle window images and a status classification module for indicator light images;
[0079] The fused features are input into the text recognition module of the handcart window image to obtain the coordinates of the position of the text detection target box, and then the position coordinates are used to determine whether the circuit breaker handcart is in the pushed-out state or the pushed-in state.
[0080] The fused features are input into the state classification module of the indicator light image to determine whether the circuit breaker truck is in the pushed-out or pushed-in state.
[0081] The predictive discrimination network compares the recognition results of the text recognition module of the vehicle window image with the recognition results of the status classification module of the indicator light image to determine whether the circuit breaker is operating normally.
[0082] The method by which the feature extraction network extracts features from indicator light images and vehicle window images through convolutional layers and multi-layer convolutional attention modules includes:
[0083] The indicator light image and the vehicle window image are first input into a convolutional layer. The output of the convolutional layer is then input into a five-segment cascaded convolutional attention module to obtain the features of the indicator light image and the vehicle window image.
[0084] The feature fusion network fuses the features output from each convolutional attention module through an up-convolutional module and an upsampling process to obtain fused features. The methods include:
[0085] First, the output feature map of the fifth stage convolutional attention module is concatenated dimensionally with the output feature map of the fourth stage convolutional attention module to obtain a first concatenated feature map. This first concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a first fused feature map. Next, the first fused feature map is concatenated dimensionally with the output feature map of the third stage convolutional attention module to obtain a second concatenated feature map. This second concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a second fused feature map. Finally, the second fused feature map is concatenated dimensionally with the output feature map of the second stage convolutional attention module to obtain a third concatenated feature map. This third concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a third fused feature map. Finally, the third fused feature map is concatenated dimensionally with the output feature map of the first stage convolutional attention module to obtain a fourth concatenated feature map. This fourth concatenated feature map undergoes an up-convolution module and an upsampling process to obtain the final fused feature map.
[0086] The method for inputting the fused features into the text recognition module of the vehicle window image to obtain the coordinates of the text detection target box includes:
[0087] The fused features are compressed through a convolutional layer to obtain a region score representing the probability of the center region of a single character and an affinity score representing the center probability of the space between adjacent characters.
[0088] The region score and affinity score are used to filter the pixel regions, and the pixel regions with region scores and affinity scores greater than a preset threshold are selected as candidate regions for text. Then, word components are connected within the candidate regions for text. Finally, a quadrilateral is established by finding four vertices to enclose the connection components corresponding to each label with the smallest area, and the coordinates of the text detection target box are obtained.
[0089] The method for determining whether the circuit breaker trolley is in an extended or extended state based on the coordinates of the position includes:
[0090] Based on the coordinates of the positions, the maximum area of the text detection target box in the pushed-in state and the minimum area of the text detection target box in the pushed-out state of each circuit breaker truck are statistically analyzed, and the average value of the two is taken. The area of the text detection target box in the truck window image is compared with the average value. If the area of the text detection target box in the current truck window image is greater than the average value, it is classified as the pushed-out state; otherwise, it is classified as the pushed-in state.
[0091] The method of inputting the fused features into the state classification module of the indicator light image to determine whether the circuit breaker truck is in an outgoing or ingoing state includes:
[0092] The fused features are flattened to expand the two-dimensional feature map into a one-dimensional vector, and then passed through three fully connected layers and the ReLU nonlinear activation function to obtain two-dimensional classification features. Finally, the two-dimensional classification features are processed by softmax to obtain the discrimination results of the circuit breaker truck being pushed in and pushed out.
[0093] Specifically, such as Figure 3 As shown. The multi-scale feature fusion circuit breaker state two-factor verification model includes: a feature extraction network, a feature fusion network, and a prediction and discrimination network.
[0094] The feature extraction network is responsible for extracting features from the vehicle window and indicator light images fed into the network. The specific steps are as follows: First, a convolutional layer compresses the scale of the original 512×512×32 features to half its original size, increasing the dimension to twice its original size, resulting in a preliminary extracted feature map. Then, five stages are performed, each consisting of a convolutional layer and a CBAM attention mechanism module for refined feature extraction. In the first three stages, the feature dimension is increased to twice its original size, and the scale is reduced to twice its original size; in the last two stages, both the dimension and scale remain unchanged. Finally, the feature maps from these five stages are fed into the feature fusion part to combine spatial and semantic information to obtain the final fused features. The structure of the CBAM attention mechanism module is shown in Figure 4(a), and the specific process is as follows: First, it passes through the channel attention module, the structure of which is shown in Figure 4(b). For the input feature F, it is first pooled by average and max pooling layers respectively. The output one-dimensional vector is then input into the MLP neural network to obtain two intermediate variables. Subsequently, the output feature is convolved and summed, and the weight Mc is obtained by activating the Sigmoid function. The weights are multiplied by the feature F to obtain the output F' of the channel attention module. Then, the feature F' is fed into the spatial attention module. The structure of the spatial attention mechanism module is shown in Figure 4(c). The feature F' is then processed by average and max pooling to obtain two feature maps, which are then concatenated by channel and fed into the convolutional layer. The obtained intermediate variables are then activated by the activation function to obtain the weight coefficients Ms. Ms is multiplied by the feature F to obtain the final feature.
[0095] After feature extraction is complete, the feature maps from the five stages are fed into the feature fusion section. In the feature fusion section, feature maps at different scales are combined through continuous convolution and upsampling operations to obtain the final feature map. The specific process is as follows:
[0096] First, the feature map from the fifth stage is concatenated with the feature map from the fourth stage to obtain a new feature map. This new feature map then undergoes an upconvolution module and an upsampling process, increasing the feature scale by a factor of 2 and reducing the dimension by a factor of 2. This new feature map is then concatenated with the feature map from the third stage. Similar dimensional concatenation is repeated until the final feature map size is uniformly adjusted to (256×256×32) to obtain feature information containing both accurate semantic features and rich spatial features. The upconvolution module is as follows: Figure 5 As shown, the feature map dimension is reduced to twice its original size to facilitate combination with the features from the previous stage. Specifically, the input features first pass through a 1x1 convolutional layer, then through a Batch_norm layer for standardization, at which point the feature dimension remains unchanged; then through a 3x3 convolutional layer and a Batch_norm layer, the feature dimension is reduced to twice its original size.
[0097] The prediction and discrimination network includes: a text recognition module for vehicle window images and a status classification module for indicator light images;
[0098] The text recognition module of the car window image compresses the fused features through a convolutional layer to obtain a feature map with a size of 256×256 and a dimension of 2, which yields the region score representing the probability of the center region of a single character and the affinity score representing the center probability of the space between adjacent characters.
[0099] The region score and affinity score are used for filtering. Pixel regions with both scores greater than a preset threshold are selected as candidate text regions. Then, word components are connected within these candidate text regions. Finally, a quadrilateral is established by finding four vertices to enclose the connection components corresponding to each label with the smallest area, thus obtaining the coordinates of the text detection target box. Based on these coordinates, the maximum area of the text detection target box in the pushed-in state and the minimum area in the pushed-out state of each circuit breaker truck are calculated, and their average value is taken. The area of the text detection target box in the truck window image is compared with this average value. If the area of the text detection target box in the current truck window image is greater than the average value, it is classified as a pushed-out state; otherwise, it is classified as a pushed-in state.
[0100] The indicator light image state classification module flattens the fused feature (256x256x32) to expand the two-dimensional feature map into a one-dimensional vector, and then passes it through three fully connected layers of 1x1x4096, 1x1x1000, and 1x1x2 and the ReLU nonlinear activation function to obtain two-dimensional classification features. Finally, the two-dimensional classification features are processed by softmax to obtain the discrimination results of the circuit breaker truck being pushed in and pushed out.
[0101] The predictive discriminant network compares the recognition results of the text recognition module of the vehicle window image with the recognition results of the status classification module of the indicator light image to determine whether the circuit breaker is operating normally.
[0102] In step S2, the indicator light image and the vehicle window image are acquired and preprocessed.
[0103] Specifically, preprocessing the data in the dataset involves image normalization and image enhancement. Normalization ensures that the image resolution is uniformly 512×512 pixels. Image enhancement includes operations such as random gamma transformation, rotation, blurring, and noise addition. Its purpose is to increase the amount of data while preserving image features, thereby achieving better network training results and avoiding overfitting.
[0104] In step S3, the preprocessed indicator light image and the vehicle window image are input into the circuit breaker state two-factor verification model fused with multi-scale features for training.
[0105] Specifically, the cross-entropy function was selected as the loss function, the optimizer was Adam, the number of iterations was set to 100,000, the batch size was 32, and the initial learning rate was 0.01. An adaptive learning rate was used to find a balance between training speed and accuracy. An early stopping mechanism was introduced, which stopped training when the loss did not decrease after 100 consecutive iterations.
[0106] In step S4, the images of the handcart window and the indicator lights of the circuit breaker to be identified in the pump station are input into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification results of the pump station.
[0107] In summary, the solution proposed in this invention can,
[0108] 1. By making full use of the multi-scale feature fusion structure and attention mechanism, a two-factor verification model for circuit breaker status based on multi-scale feature fusion is proposed. A multi-input multi-output neural network is constructed, which has the characteristics of fast detection speed and high accuracy, and is used to realize two-factor verification of circuit breaker status.
[0109] 2. This paper applies text detection algorithms to image state classification tasks, transforming changes in the depth dimension of an image into changes in the horizontal dimension to determine the image state. The main focus is on providing a solution to the problem that two-dimensional image classification algorithms are difficult to apply to recognizing changes in the depth dimension of an image.
[0110] 3. The text detection part of the circuit breaker state two-factor verification model based on multi-scale feature fusion is more accurate and fits the text target box more closely, effectively solving the problems that manual inspection methods are easily affected by subjective factors and the accuracy and fit of existing text detection networks are insufficient.
[0111] The second aspect of the present invention discloses a two-factor verification system for circuit breaker status. Figure 6 This is a structural diagram of a two-factor verification system for circuit breaker status according to an embodiment of the present invention; as shown. Figure 6 As shown, the system 100 includes:
[0112] The first processing module 101 is configured to construct a circuit breaker status dual-factor verification model that integrates multi-scale features to identify the circuit breaker's operating status by inputting the indicator light image and the truck window image of the pump station circuit breaker.
[0113] The second processing module 102 is configured to acquire the indicator light image and the vehicle window image, and preprocess them;
[0114] The third processing module 103 is configured to input the preprocessed indicator light image and the vehicle window image into the circuit breaker state two-factor verification model fused with multi-scale features, and train it.
[0115] The fourth processing module 104 is configured to input the images of the handcart windows and indicator lights of the circuit breaker to be identified into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification result of the pump station.
[0116] According to a system of a second aspect of the present invention, the first processing module 101 is configured such that the multi-scale feature fusion circuit breaker state two-factor verification model includes: a feature extraction network, a feature fusion network, and a prediction and discrimination network;
[0117] The feature extraction network extracts features from indicator light images and vehicle window images through convolutional layers and multi-layer convolutional attention modules.
[0118] The feature fusion network fuses the features output by the attention module of each convolutional layer through an up-convolutional module and an upsampling process to obtain fused features.
[0119] The prediction and discrimination network includes: a text recognition module for vehicle window images and a status classification module for indicator light images;
[0120] The fused features are input into the text recognition module of the handcart window image to obtain the coordinates of the position of the text detection target box, and then the position coordinates are used to determine whether the circuit breaker handcart is in the pushed-out state or the pushed-in state.
[0121] The fused features are input into the state classification module of the indicator light image to determine whether the circuit breaker truck is in the pushed-out or pushed-in state.
[0122] The predictive discrimination network compares the recognition results of the text recognition module of the vehicle window image with the recognition results of the status classification module of the indicator light image to determine whether the circuit breaker is operating normally.
[0123] According to a system of a second aspect of the present invention, the first processing module 101 is configured such that the feature extraction network extracts features from the indicator light image and the vehicle window image through convolutional layers and multi-layer convolutional attention modules, including:
[0124] The indicator light image and the vehicle window image are first input into a convolutional layer. The output of the convolutional layer is then input into a five-segment cascaded convolutional attention module to obtain the features of the indicator light image and the vehicle window image.
[0125] According to a system of a second aspect of the present invention, the first processing module 101 is configured such that the feature fusion network fuses the features output by the convolutional attention modules of each layer through an up-convolutional module and an upsampling process to obtain fused features including:
[0126] First, the output feature map of the fifth stage convolutional attention module is concatenated dimensionally with the output feature map of the fourth stage convolutional attention module to obtain a first concatenated feature map. This first concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a first fused feature map. Next, the first fused feature map is concatenated dimensionally with the output feature map of the third stage convolutional attention module to obtain a second concatenated feature map. This second concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a second fused feature map. Finally, the second fused feature map is concatenated dimensionally with the output feature map of the second stage convolutional attention module to obtain a third concatenated feature map. This third concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a third fused feature map. Finally, the third fused feature map is concatenated dimensionally with the output feature map of the first stage convolutional attention module to obtain a fourth concatenated feature map. This fourth concatenated feature map undergoes an up-convolution module and an upsampling process to obtain the final fused feature map.
[0127] According to a system of a second aspect of the present invention, the first processing module 101 is configured to input the fused features into a text recognition module of the vehicle window image to obtain the coordinates of the position of the text detection target box, including:
[0128] The fused features are compressed through a convolutional layer to obtain a region score representing the probability of the center region of a single character and an affinity score representing the center probability of the space between adjacent characters.
[0129] The region score and affinity score are used to filter the pixel regions, and the pixel regions with region scores and affinity scores greater than a preset threshold are selected as candidate regions for text. Then, word components are connected within the candidate regions for text. Finally, a quadrilateral is established by finding four vertices to enclose the connection components corresponding to each label with the smallest area, and the coordinates of the text detection target box are obtained.
[0130] According to a system of a second aspect of the present invention, the first processing module 101 is configured to, further determining whether the circuit breaker trolley is in an extended or extended state based on the coordinates of the position, includes:
[0131] Based on the coordinates of the positions, the maximum area of the text detection target box in the pushed-in state and the minimum area of the text detection target box in the pushed-out state of each circuit breaker truck are statistically analyzed, and the average value of the two is taken. The area of the text detection target box in the truck window image is compared with the average value. If the area of the text detection target box in the current truck window image is greater than the average value, it is classified as the pushed-out state; otherwise, it is classified as the pushed-in state.
[0132] According to a system of a second aspect of the present invention, the first processing module 101 is configured such that the step of inputting the fused features into the state classification module of the indicator light image to determine whether the circuit breaker truck is in an outgoing or ingoing state includes:
[0133] The fused features are flattened to expand the two-dimensional feature map into a one-dimensional vector, and then passed through three fully connected layers and the ReLU nonlinear activation function to obtain two-dimensional classification features. Finally, the two-dimensional classification features are processed by softmax to obtain the discrimination results of the circuit breaker truck being pushed in and pushed out.
[0134] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a two-factor verification method for circuit breaker status according to any one of the first aspects of this invention.
[0135] Figure 7 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 7 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.
[0136] Those skilled in the art will understand that Figure 7The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0137] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a two-factor verification method for circuit breaker status according to any one of the first aspects of this invention.
[0138] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A two-factor verification method for circuit breaker status, characterized in that, The method includes: Step S1: Construct a two-factor verification model for circuit breaker status by fusing multi-scale features to identify the circuit breaker's operating status through input images of indicator lights and truck windows of the circuit breaker at the pumping station. Step S2: Acquire the indicator light image and the vehicle window image, and preprocess them; Step S3: Input the preprocessed indicator light image and the vehicle window image into the circuit breaker state two-factor verification model fused with multi-scale features, and train it. Step S4: Input the images of the handcart window and the indicator lights of the circuit breaker to be identified into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification results of the pump station. In step S1, the circuit breaker state two-factor verification model with multi-scale feature fusion includes: a feature extraction network, a feature fusion network, and a prediction and discrimination network; The feature extraction network extracts features from indicator light images and vehicle window images through convolutional layers and multi-layer convolutional attention modules. The feature fusion network fuses the features output by the attention module of each convolutional layer through an up-convolutional module and an upsampling process to obtain fused features. The prediction and discrimination network includes: a text recognition module for vehicle window images and a status classification module for indicator light images; The fused features are input into the text recognition module of the handcart window image to obtain the coordinates of the position of the text detection target box, and then the position coordinates are used to determine whether the circuit breaker handcart is in the pushed-out state or the pushed-in state. The fused features are input into the state classification module of the indicator light image to determine whether the circuit breaker truck is in the pushed-out or pushed-in state. The predictive discriminant network compares the recognition results of the text recognition module of the vehicle window image with the recognition results of the status classification module of the indicator light image to determine whether the circuit breaker is operating normally. In step S1, the method by which the feature extraction network extracts features from the indicator light image and the vehicle window image through convolutional layers and multi-layer convolutional attention modules includes: The indicator light image and the vehicle window image are first input into a convolutional layer, and the output of the convolutional layer is input into a five-segment convolutional attention module to obtain the features of the indicator light image and the vehicle window image. In step S1, the method for inputting the fused features into the text recognition module of the vehicle window image to obtain the coordinates of the text detection target box includes: The fused features are compressed through a convolutional layer to obtain a region score representing the probability of the center region of a single character and an affinity score representing the center probability of the space between adjacent characters. The region score and affinity score are filtered, and pixel regions with region score and affinity score values greater than a preset threshold are selected as candidate regions for text. Then, word components are connected within the candidate regions for text. Finally, a quadrilateral is established by finding four vertices to enclose the connection components corresponding to each label with the smallest area, and the coordinates of the position of the text detection target box are obtained. In step S1, the method for determining whether the circuit breaker trolley is in an extended or extended state based on the coordinates of the position includes: Based on the coordinates of the positions, the maximum area of the text detection target box in the pushed-in state and the minimum area of the text detection target box in the pushed-out state of each circuit breaker truck are statistically analyzed, and the average value of the two is taken. The area of the text detection target box in the truck window image is compared with the average value. If the area of the text detection target box in the current truck window image is greater than the average value, it is classified as the pushed-out state; otherwise, it is classified as the pushed-in state.
2. The two-factor verification method for circuit breaker status according to claim 1, characterized in that, In step S1, the feature fusion network fuses the features output by each convolutional attention module through an up-convolutional module and an upsampling process to obtain fused features. The method includes: First, the output feature map of the fifth stage convolutional attention module is concatenated dimensionally with the output feature map of the fourth stage convolutional attention module to obtain a first concatenated feature map. This first concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a first fused feature map. Next, the first fused feature map is concatenated dimensionally with the output feature map of the third stage convolutional attention module to obtain a second concatenated feature map. This second concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a second fused feature map. Finally, the second fused feature map is concatenated dimensionally with the output feature map of the second stage convolutional attention module to obtain a third concatenated feature map. This third concatenated feature map undergoes an up-convolution module and an upsampling process to obtain a third fused feature map. Finally, the third fused feature map is concatenated dimensionally with the output feature map of the first stage convolutional attention module to obtain a fourth concatenated feature map. This fourth concatenated feature map undergoes an up-convolution module and an upsampling process to obtain the final fused feature map.
3. The two-factor verification method for circuit breaker status according to claim 1, characterized in that, The method of inputting the fused features into the state classification module of the indicator light image to determine whether the circuit breaker truck is in an outgoing or ingoing state includes: The fused features are flattened to expand the two-dimensional feature map into a one-dimensional vector, and then passed through three fully connected layers and the ReLU nonlinear activation function to obtain two-dimensional classification features. Finally, the two-dimensional classification features are processed by softmax to obtain the discrimination results of the circuit breaker truck being pushed in and pushed out.
4. A two-factor verification system for circuit breaker status, said system employing the method described in any one of claims 1-3, characterized in that, The system includes: The first processing module is configured to construct a circuit breaker status two-factor verification model that integrates multi-scale features to identify the circuit breaker's operating status by inputting the indicator light image and the truck window image of the pump station circuit breaker. The second processing module is configured to acquire the indicator light image and the vehicle window image, and preprocess them; The third processing module is configured to input the preprocessed indicator light image and the vehicle window image into the circuit breaker state two-factor verification model fused with multi-scale features, and train it. The fourth processing module is configured to input the images of the handcart windows and indicator lights of the circuit breaker to be identified into the trained multi-scale feature fusion circuit breaker status two-factor verification model to obtain the circuit breaker handcart status two-factor verification results of the pump station.
5. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the two-factor verification method for circuit breaker status according to any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the two-factor verification method for circuit breaker status according to any one of claims 1 to 3.
Citation Information
Patent Citations
RCNN-based cubicle switch and indication lamp localization and state identification method
CN108010025A