A detection method, device, computer equipment and readable storage medium for an electrician's test pen based on deep learning
Through deep learning-based detection methods, the single-stage object detection model and key point detection model are used to automatically identify the position and key points of the test pen, which solves the problem of relying on experience in traditional test pen operation, and realizes efficient and accurate test pen detection , improves the safety and efficiency of power operations.
Patent Information
- Application Number
- CN202410931669.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-12
AI Technical Summary
The operation of traditional electric tester pens relies on professional training and experience, which increases uncertainty and risks of electrical operations and makes it difficult to achieve automated inspection.
Using deep learning-based detection methods, the pre-trained single-stage object detection model and key point detection model are used to identify the rectangular area, pen tip and pen tail end points of the pen test pen through image processing, and determine the pen body of the pen test pen to achieve automation. Test.
It improves the accuracy and efficiency of the test pen inspection, effectively improves the safety and efficiency of power operations, and reduces human error.
Smart Images

Figure CN118470310B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to a detection method, device, computer equipment and readable storage medium for an electrician's test pen based on deep learning. Background Art
[0002] In the power system, the electrician's electrician pen is an important tool for detecting whether electrical equipment and lines are energized. Before performing electrical operations, electricians must ensure that the equipment and lines are in a safe state, and the correct use of the electrician pen is a key link in this safety guarantee. However, the traditional operation of the electrician pen depends on the professional training and experience of the electrician, which increases the uncertainty and risk of the operation to a certain extent. Summary of the invention
[0003] The object of the present invention is to provide a detection method, device, computer equipment and readable storage medium for an electrician's test pen based on deep learning.
[0004] In a first aspect, an embodiment of the present invention provides a detection method of an electrician's test pen based on deep learning, comprising:
[0005] Acquire an image to be processed including an electrician tester to be tested;
[0006] Input the image to be processed into a pre-trained single-stage target detection model for preliminary detection to obtain a preliminary detection result of the electrician test pen, wherein the preliminary detection result of the electrician test pen is a rectangular area where the electrician test pen to be detected is located;
[0007] Calling a pre-trained key point detection model to perform a detailed detection on the preliminary detection result of the electrician test pen, and obtaining the tip end point and the tail end point of the electrician test pen to be detected;
[0008] The body of the electrician test pen is determined according to the pen tip point and the pen tail point, and the pen tip point, the pen tail point and the electrician test pen body are determined as the target detection result of the electrician test pen to be detected.
[0009] In an embodiment of the present invention, the single-stage target detection model is constructed based on the SSD model, and the single-stage target detection model includes a backbone network, an additional feature layer and a head structure;
[0010] The backbone network is a MobileNet V2 structure, including a Conv12 convolutional layer and a Conv16 convolutional layer, which are used to generate two feature maps of corresponding sizes;
[0011] The additional feature layer includes a Conv18 convolution layer and a Conv20 convolution layer, which are used to generate two feature maps of corresponding sizes;
[0012] The head structure includes default box generation, a classification subnetwork and a regression subnetwork, which are used to combine the four feature maps generated by the backbone network and the additional feature layer, perform multi-layer prediction fusion and non-maximum suppression, and obtain the type of the electrician test pen to be detected and the rectangular area.
[0013] In an embodiment of the present invention, the single-stage target detection model is trained in the following manner, including:
[0014] The basic single-stage target detection model is trained and optimized based on the Adam optimizer;
[0015] After each iteration of the training data for the basic single-stage target detection model is completed, the model average accuracy of the current cycle is obtained, and the model weight corresponding to the current highest model average accuracy is saved;
[0016] When the preset training stop condition is reached, the single-stage target detection model that has completed training is obtained.
[0017] In an embodiment of the present invention, the key point detection model is constructed based on PFLD, and the key point detection model includes a backbone network and a head structure;
[0018] The backbone network is built based on MobileNet V2, including four stacked MobileNet V2 bottleneck modules;
[0019] The bottleneck module includes an improved channel attention module;
[0020] The channel attention module includes a global pooling layer, a fully connected layer, a normalization layer and an attention mechanism; the global pooling layer is used to perform global average pooling or global maximum pooling operations on each channel of the input feature map; the fully connected layer is used to extract the channel attention weights related to the task from the global statistical information; the normalization layer is used to standardize the attention weights; the attention mechanism is used to multiply the obtained channel attention coefficient vector by the original feature map element by channel to weight the features of each channel;
[0021] The head structure includes a fully connected layer, the number of output channels of the fully connected layer is 4, and the fully connected layer is used to output the coordinate information of the tip point of the pen and the tail point of the pen.
[0022] In an embodiment of the present invention, the key point detection model is trained in the following manner, including:
[0023] The basic key point detection model is trained and optimized based on the Adam optimizer, and SmoothL1Loss is used as the loss function;
[0024] After each iteration of the training data for the basic key point detection model is completed, the model mean square error of the current cycle is obtained, and the model weight corresponding to the current minimum mean square error is saved;
[0025] When the preset training stop condition is reached, the key point detection model that has completed training is obtained.
[0026] In an embodiment of the present invention, the method further includes:
[0027] Obtain a pre-annotated electrician test pen rectangular frame and construct a training data set for training the single-stage target detection model;
[0028] Expand the pre-marked test pen rectangular frame by a preset multiple to obtain a test pen area;
[0029] A training data set for training the key point detection model is constructed based on the electrician test pen area.
[0030] In an embodiment of the present invention, determining the body of the electrician test pen according to the pen tip end point and the pen tail end point includes:
[0031] Obtaining a line connecting the pen tip end point and the pen tail end point;
[0032] According to the connection, the body of the electrician's pen is determined.
[0033] In a second aspect, an embodiment of the present invention provides a detection device for an electrician's test pen based on deep learning, comprising:
[0034] An acquisition module, used for acquiring an image to be processed including an electrician test pen to be detected;
[0035] A detection module is used to input the image to be processed into a pre-trained single-stage target detection model for preliminary detection to obtain a preliminary detection result of the test pen, wherein the preliminary detection result of the test pen is a rectangular area where the test pen to be detected is located; a pre-trained key point detection model is called to perform a refined detection on the preliminary detection result of the test pen to obtain the tip point and tail point of the test pen to be detected; the body of the test pen is determined according to the tip point and the tail point, and the tip point, the tail point and the body of the test pen are determined as the target detection result of the test pen to be detected.
[0036] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device executes the method described in at least one possible implementation of the first aspect.
[0037] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method described in at least one possible implementation manner of the first aspect.
[0038] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a deep learning-based detection method, device, computer equipment and readable storage medium for an electrician's test pen disclosed in the present invention, by acquiring the image to be detected, and preliminarily locating the rectangular area where the electrician's test pen is located through a pre-trained single-stage target detection model. Subsequently, the key point detection model is called to refine the preliminary detection results, and the tip point and tail point of the electrician's test pen are accurately identified. Finally, the body of the electrician's test pen is determined based on these two key points, thereby completing the comprehensive detection of the electrician's test pen. This method realizes efficient and accurate automatic detection of electrician's test pens, effectively improving the safety and efficiency of power operations. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.
[0040] Figure 1 A schematic flow chart of the steps of a detection method for an electrician's test pen based on deep learning provided in an embodiment of the present invention;
[0041] Figure 2 A schematic diagram of the structure of the improved SSD MobileNet V2 model provided in an embodiment of the present invention;
[0042] Figure 3 A schematic structural diagram of an improved bottleneck module provided in an embodiment of the present invention;
[0043] Figure 4 A schematic diagram of detecting key points of the tip and tail of an electrician's pen provided in an embodiment of the present invention;
[0044] Figure 5 A schematic diagram of the detection of the body of an electrician's test pen is provided for an embodiment of the present invention;
[0045] Figure 6 A schematic block diagram of the structure of a detection device for an electrician's test pen based on deep learning provided in an embodiment of the present invention;
[0046] Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0048] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.
[0049] In order to solve the technical problems in the aforementioned background technology, Figure 1 The present invention provides a flow chart of a method for detecting an electrician's test pen based on deep learning in an embodiment of the present invention. The method for detecting an electrician's test pen based on deep learning is introduced in detail below.
[0050] Step S201, obtaining an image to be processed including an electrician test pen to be tested;
[0051] Step S202, inputting the image to be processed into a pre-trained single-stage target detection model for preliminary detection, and obtaining a preliminary detection result of the electrician test pen, wherein the preliminary detection result of the electrician test pen is a rectangular area where the electrician test pen to be detected is located;
[0052] Step S203, calling a pre-trained key point detection model to perform a detailed detection on the preliminary detection result of the electrician test pen, and obtaining the tip end point and the tail end point of the electrician test pen to be detected;
[0053] Step S204, determining the body of the electrician test pen according to the pen tip point and the pen tail point, and determining the pen tip point, the pen tail point and the electrician test pen body as the target detection result of the electrician test pen to be detected.
[0054] In an embodiment of the present invention, for example, at the power operation site, a worker uses a high-definition camera or a smart phone to take a photo containing a test pen. This photo is then uploaded to the server. The server receives the photo and uses it as an image to be processed, preparing for the subsequent test pen detection process. The server inputs the received photo into a single-stage target detection model that has been trained with a large amount of test pen image data, such as the YOLO (You Only Look Once) model. This model can quickly identify the test pen in the image and give a rectangular area as the preliminary detection result of the test pen. For example, in this photo, the model successfully identifies the test pen and marks its position with a rectangular box. Next, the server calls another pre-trained key point detection model, which is specially trained for the test pen and is used to accurately detect the key points of the test pen, namely the tip point and the end point of the pen. In this example, the model accurately identifies the tip point and the end point of the test pen and marks them on the image. Finally, the server determines the body of the test pen based on the detected tip point and the end point of the pen by calculating the straight line or curve connecting the two points. During this process, the server will also consider features such as the shape and size of the test pen to ensure the accuracy of the detection. Finally, the server outputs the tip end point, tail end point and body of the test pen as the target detection result of the test pen to be tested. This result can be displayed intuitively on the original image, or it can be provided to other systems or applications through a data interface. For example, in this scenario, the server displays the test results on a graphical interface, and the staff can clearly see the exact position and posture of the test pen in the image. With this design, it is possible to accurately evaluate whether the test pen is operated correctly in training, monitoring and other test pen usage scenarios.
[0055] In an embodiment of the present invention, the single-stage target detection model is constructed based on the SSD model, and the single-stage target detection model includes a backbone network, an additional feature layer and a head structure;
[0056] The backbone network is a MobileNet V2 structure, including a Conv12 convolutional layer and a Conv16 convolutional layer, which are used to generate two feature maps of corresponding sizes;
[0057] The additional feature layer includes a Conv18 convolution layer and a Conv20 convolution layer, which are used to generate two feature maps of corresponding sizes;
[0058] The head structure includes default box generation, a classification subnetwork and a regression subnetwork, which are used to combine the four feature maps generated by the backbone network and the additional feature layer, perform multi-layer prediction fusion and non-maximum suppression, and obtain the type of the electrician test pen to be detected and the rectangular area.
[0059] In an embodiment of the present invention, exemplarily, in the deep learning framework of the server, the backbone network of the single-stage target detection model adopts the MobileNet V2 structure. This structure is known for its high efficiency and lightweight, and is particularly suitable for deployment in resource-limited environments. The backbone network is mainly composed of Conv12 convolutional layers and Conv16 convolutional layers, which are responsible for extracting feature information in the image and generating two feature maps of corresponding sizes. These feature maps capture details of different scales in the image and provide rich information for subsequent object detection. For example, when the server receives an image containing an electrician pen, the Conv12 and Conv16 convolutional layers of MobileNet V2 will start working, extracting features from the image layer by layer, and generating feature maps of two scales. These feature maps not only contain the overall shape information of the electrician pen, but also capture detailed features such as the pen body and the pen tip point. In addition to the backbone network, the model also includes additional feature layers, which are composed of Conv18 convolutional layers and Conv20 convolutional layers. These additional feature layers further extract deep features in the image and generate two other feature maps of corresponding sizes. These feature maps complement the feature maps generated by the backbone network, providing more details and contextual information for subsequent object detection and classification. In the scenario of electrician pen detection, the Conv18 and Conv20 convolutional layers further process the feature maps output by the backbone network to extract more detailed feature information. This information helps the model to more accurately identify the shape, size, and position of the electrician pen. The head structure is a key part of the single-stage target detection model, which includes default box generation, classification subnetwork, and regression subnetwork. This structure is responsible for combining the four feature maps generated by the backbone network and the additional feature layer, performing multi-layer prediction fusion and non-maximum suppression, and finally obtaining the type of electrician pen to be detected and the rectangular area. Specifically, in the process of electrician pen detection, the head structure first generates a series of default boxes based on the four feature maps. These boxes cover the areas in the image that may contain electrician pen. Then, the classification subnetwork classifies these default boxes to determine whether they contain electrician pen and gives the type of electrician pen (such as ordinary electrician pen, high-voltage electrician pen, etc.). At the same time, the regression subnetwork adjusts the position and size of these default boxes to more accurately select the actual position of the test pen. Finally, the non-maximum suppression (NMS) algorithm is used to remove the box selection results with high overlap, retaining the most accurate and representative test pen detection box. In this way, the server can output an image with the position and type of the test pen clearly marked as the detection result.
[0060] In an embodiment of the present invention, the single-stage target detection model is trained in the following manner.
[0061] The basic single-stage target detection model is trained and optimized based on the Adam optimizer;
[0062] After each iteration of the training data for the basic single-stage target detection model is completed, the model average accuracy of the current cycle is obtained, and the model weight corresponding to the current highest model average accuracy is saved;
[0063] When the preset training stop condition is reached, the single-stage target detection model that has completed training is obtained.
[0064] In an embodiment of the present invention, exemplarily, in the deep learning training environment of the server, we selected the Adam optimizer to train and optimize the basic single-stage target detection model. The Adam optimizer combines the ideas of momentum gradient descent and RMSprop, and can adaptively adjust the learning rate so that the model can converge faster during the training process. For example, we set an initial learning rate at the beginning and input the training data set into the model in batches for training. In each iteration, the Adam optimizer adjusts the current parameter update step size according to the previous gradient information and learning rate to ensure that the model can learn the features in the data more effectively. Whenever the model completes an iteration of the training data, the server calculates the mean Average Precision (mAP) of the model in the current cycle. This indicator can comprehensively reflect the detection performance of the model in each category and is a commonly used evaluation indicator in target detection tasks. The server compares the mAP of the current cycle with the highest mAP value saved previously. If the mAP of the current cycle is higher, the server will save the weights of the current model because the model performance corresponding to these weights is better. In this way, the server can ensure that the model weights with the best performance are always retained during the training process.
[0065] For example, at the end of the 10th epoch of training, the server calculates that the mAP of the current epoch is 0.85, which is higher than the previously saved highest mAP value of 0.83. The server then saves the weights of the current model as the new best model weights.
[0066] Before training begins, we set some training stop conditions, such as reaching the preset number of training cycles, model performance no longer significantly improving, etc. When the training process meets these conditions, the server will stop training and output the trained single-stage object detection model.
[0067] For example, we set the training cycle to 50 times. When the model completes the 50th iteration, the server will stop training. At this time, the server will load the previously saved best model weights into the model and output the trained single-stage object detection model. This model has learned how to accurately detect the test pen in the image and can automatically identify the location and type of the test pen in the new image.
[0068] In an embodiment of the present invention, the key point detection model is constructed based on PFLD, and the key point detection model includes a backbone network and a head structure;
[0069] The backbone network is built based on MobileNet V2, including four stacked MobileNet V2 bottleneck modules;
[0070] The bottleneck module includes an improved channel attention module;
[0071] The channel attention module includes a global pooling layer, a fully connected layer, a normalization layer and an attention mechanism; the global pooling layer is used to perform global average pooling or global maximum pooling operations on each channel of the input feature map; the fully connected layer is used to extract the channel attention weights related to the task from the global statistical information; the normalization layer is used to standardize the attention weights; the attention mechanism is used to multiply the obtained channel attention coefficient vector by the original feature map element by channel to weight the features of each channel;
[0072] The head structure includes a fully connected layer, the number of output channels of the fully connected layer is 4, and the fully connected layer is used to output the coordinate information of the tip point of the pen and the tail point of the pen.
[0073] In an embodiment of the present invention, exemplarily, in the deep learning framework of the server, the backbone network of the key point detection model is built based on MobileNet V2. This network structure is known for its high efficiency and light weight, and is particularly suitable for deployment in resource-constrained environments. The backbone network consists of four stacked bottleneck modules of MobileNet V2, which can effectively extract feature information from the image and provide rich feature representation for subsequent key point detection.
[0074] For example, when the server receives an image containing an electrician's pen, the backbone network starts working and extracts features from the image layer by layer through stacked bottleneck modules. These features not only contain the overall shape information of the electrician's pen, but also capture the detailed features of key parts such as the pen tip and pen tail.
[0075] Each bottleneck module contains an improved channel attention module, which can enhance the model's attention to key features and thus improve the accuracy of key point detection.
[0076] Specifically, the channel attention module first performs global average pooling or global maximum pooling operations on each channel of the input feature map through the global pooling layer to obtain global statistical information for each channel. Then, the fully connected layer extracts the task-related channel attention weights from these global statistical information. These weights reflect the importance of different channel features for the key point detection task. Then, the normalization layer standardizes the attention weights to ensure that their sum is 1 for subsequent weighted operations. Finally, the attention mechanism multiplies the obtained channel attention coefficient vector by the original feature map element by channel, thereby achieving weighted features for each channel.
[0077] In the scenario of electrician test pen detection, this channel attention mechanism can help the model focus more on the key parts of the electrician test pen, such as the tip and tail of the pen, thereby improving the detection accuracy of these key points.
[0078] The head structure of the key point detection model includes a fully connected layer with 4 output channels. These 4 channels correspond to the x and y coordinate information of the tip and end points of the electrician's pen. After the image features are processed by the backbone network and the channel attention module, they are input into the head structure. The fully connected layer predicts the key point coordinates of the tip and end of the pen based on these feature information and uses them as the final output of the model.
[0079] For example, after the model training is completed, when we input a new image of an electrician's pen to the server, the key point detection model will first extract image features through the backbone network and use the channel attention module to enhance the expression of key features. Then, the fully connected layer of the head structure will predict the specific coordinate positions of the pen tip and pen tail based on these features. The server will eventually return these coordinate information to us as the key point detection results.
[0080] In an embodiment of the present invention, the key point detection model is trained in the following manner.
[0081] The basic key point detection model is trained and optimized based on the Adam optimizer, and SmoothL1Loss is used as the loss function;
[0082] After each iteration of the training data for the basic key point detection model is completed, the model mean square error of the current cycle is obtained, and the model weight corresponding to the current minimum mean square error is saved;
[0083] When the preset training stop condition is reached, the key point detection model that has completed training is obtained.
[0084] In the embodiment of the present invention, for example, in the deep learning training environment of the server, the training of the key point detection model is a refined process, which aims to ensure that the model can accurately identify the key point positions of the electrician's pen in the image. The following is a detailed scenario description of the training process:
[0085] (1) Training optimization based on Adam optimizer
[0086] The server first initializes the key point detection model and uses the Adam optimizer for training optimization. The Adam optimizer combines the characteristics of momentum gradient descent and RMSprop, and can dynamically adjust the learning rate of each parameter, making the training process more stable and efficient. At the same time, SmoothL1Loss is selected as the loss function, which is more robust to outliers and suitable for processing regression problems such as key point detection.
[0087] For example, at the beginning of training, the server sets an initial learning rate and inputs the labeled training data set into the model in batches for training. In each iteration, the Adam optimizer updates the model parameters based on the gradient of the loss function and historical gradient information to minimize the SmoothL1Loss loss function, thereby continuously improving the key point detection accuracy of the model.
[0088] (2) Save the best model weights
[0089] After each iteration of training data, the server will calculate the mean square error (MSE) of the model in the current cycle. This indicator can measure the difference between the key point coordinates predicted by the model and the actual coordinates, and is an important indicator for evaluating the performance of the regression model. The server will compare the MSE of the current cycle with the previously saved minimum MSE value. If the MSE of the current cycle is smaller, it means that the prediction performance of the model has improved, so the server will save the weight of the current model as the optimal model weight.
[0090] For example, at the end of the 20th epoch of training, the server calculates that the MSE of the current epoch is 0.01, which is smaller than the previously saved minimum MSE value of 0.015. Therefore, the server saves the weights of the current model as the new best model weights.
[0091] (3) Obtain the trained model when the preset training stop condition is reached
[0092] Before training begins, the server sets some training stop conditions, such as reaching the preset number of training cycles, or the model performance no longer significantly improving. When the training process meets these conditions, the server stops training and outputs the trained key point detection model. This model has learned how to accurately detect the key point positions of the electrician's pen in the image, and can automatically identify the coordinates of the pen tip and pen tail in new images.
[0093] In the embodiments of the present invention, the following implementation modes are also provided.
[0094] Obtain a pre-annotated electrician test pen rectangular frame and construct a training data set for training the single-stage target detection model;
[0095] Expand the pre-marked test pen rectangular frame by a preset multiple to obtain a test pen area;
[0096] A training data set for training the key point detection model is constructed based on the electrician test pen area.
[0097] In the embodiment of the present invention, for example, in order to ensure that the single-stage target detection model and the key point detection model can accurately identify the electrician's pen and its key point positions, the server needs to perform a refined training data set construction. The following is a detailed scenario description of the process:
[0098] (1) Obtain the pre-marked test pen rectangular frame
[0099] First, the server will obtain a large number of images containing test pens from the existing image database, and use professional annotation tools to annotate the test pens in these images with rectangular boxes. These annotation information includes information such as the position and size of the test pens in the image, providing an accurate data basis for the subsequent construction of training data sets.
[0100] (2) Constructing a training dataset for a single-stage object detection model
[0101] Using the pre-annotated rectangular frame information of the electrician pen, the server can construct a training data set for training a single-stage target detection model. Specifically, the server will organize the annotated images and the corresponding rectangular frame information into a specific data format (such as VOC or COCO format) to facilitate model training and learning. These data sets not only contain the overall information of the electrician pen, but also cover images of the electrician pen at different angles and under different lighting conditions to improve the generalization ability of the model.
[0102] (3) Constructing a training dataset for the key point detection model based on the electrician area
[0103] In order to train the key point detection model, the server needs to further expand the rectangular frame of the test pen to obtain an image block containing the test pen and its surrounding area. This ensures that the model can take into account the relationship between the test pen and the surrounding environment when detecting key points, thereby improving the detection accuracy of key points. Specifically, the server will expand the rectangular frame of the test pen according to a preset multiple, intercept the image block containing the test pen and its surrounding area, and use it as a training sample for the key point detection model. At the same time, the server will also accurately mark the positions of the pen tip and pen tail in these image blocks to facilitate model learning and training. Through these refined training data set construction work, the server can provide rich and accurate training samples for the key point detection model, thereby ensuring that the model can accurately identify the key point positions of the test pen in the image.
[0104] In the embodiment of the present invention, the aforementioned step S204 may be implemented through the following example execution.
[0105] Obtaining a line connecting the pen tip end point and the pen tail end point;
[0106] According to the connection, the body of the electrician's pen is determined.
[0107] In the embodiment of the present invention, for example, in the image processing and analysis process of the server, determining the body of the electrician's pen is a key step, which depends on the previously detected pen tip point and pen tail point. The following is a detailed scenario description of this step:
[0108] (1) Get the line connecting the pen tip and pen tail
[0109] After receiving the coordinates of the pen tip and pen tail output by the key point detection model, the server will first calculate the straight line between the two points. This line not only represents the direct connection between the pen tip and the pen tail, but also provides a reference for the subsequent determination of the pen body.
[0110] For example, suppose the key point detection model outputs the pen tip point coordinates as (100, 200) and the pen tail end point coordinates as (300, 400). The server will calculate a straight line based on these two points, which passes through the two points and represents the approximate direction of the test pen.
[0111] (2) Determine the body of the test pen according to the connection line
[0112] After obtaining the line connecting the pen tip and the pen tail, the server will further use this line to determine the pen body of the test pen. Specifically, the server may set a certain width range on both sides of this line to frame the pen body area of the test pen. This width range can be adjusted according to actual conditions to ensure that the entire pen body of the test pen can be accurately covered.
[0113] Continuing with the above example, after the server determines the line between the pen tip (100, 200) and the pen tail (300, 400), it may set a 10-pixel width range on both sides of the line. In this way, the server obtains a rectangular area with a width of 20 pixels along the line, which is considered to be the body of the test pen.
[0114] In this way, the server can accurately identify and extract the body of the electrician's test pen from the image, providing important basic data for subsequent processing and analysis.
[0115] In order to more clearly describe the solution provided by the embodiment of the present application, a more detailed description is given below.
[0116] The first stage is to build a data set, collect data from electrician testers and manually annotate them for model training and testing;
[0117] The second stage is the preliminary detection of the electrician pen. The deep learning SSD target detection framework is lightweight and improved, and a single target detection model is trained to detect and locate the area position of the electrician pen in the image.
[0118] The third stage is the detailed detection of the electrician test pen, which improves the deep learning PFLD key point regression model to predict the two ends (tip and tail) of the electrician test pen.
[0119] The first stage is to build a data set: collect the image data set of the test pen, and use the annotation tool to manually annotate the three targets in the image: the rectangular frame of the test pen, the pen tip, and the center point of the pen tail. Divide the collected data into a training set and a test set in a ratio of 7:3, where the training set is used to train the model and the test set is used to evaluate the model performance. These data are only used in the model training process. After the model training is completed, these data are no longer needed in the subsequent detection and testing process.
[0120] The second stage is the preliminary detection of the electrician pen: The preliminary detection of the electrician pen is used to check whether there is an electrician pen in the image, and roughly locate the area position of the electrician pen in the form of a rectangular frame. The method of the present invention is based on a deep learning single-stage target detection model (SSD, Single Shot MultiBox Detector) to train a single target detection model for detecting and locating the area position of the electrician pen in the image. The method and steps are as follows:
[0121] Constructing SSD model: The present invention uses SSD as the basic framework for training the target detection model, and improves and optimizes the model on this basis. The network structure of SSD is divided into three parts: the backbone network Backbone, the extra feature layers ExtraFeature Layers and the head structure Head layer.
[0122] Backbone network: The original SSD backbone network uses the VGG-16 network, which is responsible for processing the input image and gradually extracting the high-level semantic features and low-level detail features of the image through a series of convolutional layers and pooling layers. However, the VGG-16 model has a large number of parameters and is not suitable for mobile deployment. This solution changes the backbone network to MobileNet V2. Please refer to Figure 2 , making the model lighter and requiring less computation. The image input is 448×448×3, and features are extracted through the backbone network MobileNet V2. The target detection feature map uses convolutional layers Conv12 and Conv16, whose feature map sizes are 56×56×16 and 28×28×32 respectively. The feature scale is relatively large and is mainly used to detect small targets.
[0123] Extra Feature Layers: SSD introduces Extra Feature Layers to generate feature maps of 4 different scales, thus covering a variety of object scales from fine-grained to coarse-grained. This solution simplifies the extra feature layers and only generates feature maps of 2 different scales, namely Conv18 and Conv20, with feature map sizes of 14×14×64 and 7×7×64 respectively. The feature map scale is small and is mainly used for large object detection.
[0124] The head structure Head layer, including the default box generation, classification sub-network, regression sub-network, and multi-layer prediction fusion and non-maximum suppression process, jointly completes the positioning and classification prediction of the target in the input image. Combined with the backbone network Backbone convolutional layers Conv12 and Conv16, and the additional feature layers Conv18 and Conv20, a total of 4 multi-scale feature maps, the model can simultaneously process features as small as pixel-level details and as large as the global context of the image, improving the detection accuracy of targets of different sizes. The SSD model makes independent predictions on feature maps of different scales and then fuses these prediction results together. The default box on each feature map will produce a set of classification probabilities and regression coordinates. Finally, non-maximum suppression (NMS) is performed on the prediction results at all scales to remove duplicate detections and obtain the final detection box and its category.
[0125] Training SSD model: Use the training set described in the first stage to train and optimize the model; use the test set described to test the model and evaluate the performance of the model indicators. The model training optimization algorithm uses the Adam optimizer, the initial learning rate is set to lr=0.001, the entire data set is iterated epoch=300 times, and the batch size batch_size=64.
[0126] During the training process, the program tests the average precision (mAP) of the model every time it completes a complete data iteration, and saves the model weight with the highest mAP.
[0127] In subsequent actual use, you only need to keep the model weight file and the corresponding inference program code to perform target detection.
[0128] Deploy the SSD MobileNet V2 model: After training, we get the target detection model for the electrician tester based on SSD MobileNet V2. We only need to deploy the model on the deep learning machine platform to perform model inference and testing to achieve target detection for the electrician tester.
[0129] The third stage is the detailed detection of the test pen: In the second stage, the preliminary detection of the test pen can only locate the rectangular area of the test pen, but cannot accurately locate the tip, tail and body of the test pen. The third stage of the detailed detection of the test pen aims to accurately locate the tip and tail of the test pen at the pixel level, and the body area can be detected by connecting the two endpoints of the tip and tail. The specific method and implementation steps are as follows:
[0130] Data processing: According to the manually labeled test pen rectangular frame, the area is expanded 1.20 times and cropped as the test pen area. These data are used as the training and test data sets for key point detection of the test pen tip and tail.
[0131] Constructing PFLD model: The method of the present invention uses the PFLD (Practical Facial Landmark Detector) model as the basic framework of the key point detection model. The original PFLD model is only applicable to the key point detection of the face, and cannot be used for the key point detection of the tip and tail of the electrician's pen. Therefore, the scheme of the present invention needs to improve and optimize PFLD. The network structure of PFLD is divided into three parts: Backbone network, auxiliary network and Head structure.
[0132] Backbone network: The backbone network is mainly used to extract image features, and uses MobileNet V2 as its basic network structure. The PFLD backbone network stacks 4 bottleneck modules of MobileNet V2; the bottleneck module uses the Conv-Relu6-Dwise-Relu6-Conv-Linear combination, and effectively reduces the model complexity and parameter quantity through depthwise separable convolution (Dwise for short), keeping the model lightweight. However, this also reduces the representation ability and efficiency of the model, and lacks the design of model attention. The solution of the present invention improves and optimizes the bottleneck structure, please refer to Figure 3 , add a channel attention module, and assign different weights to each channel by learning an attention mechanism to highlight the feature channels that are most important to the current task. This can make the model more focused on the features that contribute most to the current task, thereby improving the overall feature representation capability of the model. The channel attention module contains network layers such as Global Pooling-FC-sigmoid, and is specifically implemented as follows:
[0133] Global Pooling performs global average pooling or global maximum pooling on each channel of the input feature map to obtain the overall statistical information of the channel. In this way, each channel is reduced from the original spatial dimension (H × W) to a single scalar, forming a new vector with the same number of channels.
[0134] The fully connected layer (FC) extracts task-related channel attention weights from global statistical information, allowing the model to pay more attention to the characteristics of certain channels.
[0135] Normalization layer: In order to keep the learned attention weights within a reasonable range and facilitate subsequent multiplication with the original feature map, the converted vector needs to be normalized (such as sigmoid function) and scaled to the interval [0, 1]. This gives the attention coefficient of each channel.
[0136] Attention is applied by multiplying the obtained channel attention coefficient vector with the original feature map by channel element-wise (element-wise multiplication), thereby weighting the features of each channel. The weighted feature map retains the original spatial structure, but the influence of each channel has been adaptively adjusted.
[0137] Auxiliary network: The original PFLD model contains an auxiliary network for predicting the Euler angles of the face posture, making the predicted key point positions more stable and robust. However, the key point detection of the electrician pen implemented by the present invention only has two key points, the pen tip and the pen tail, and no posture prediction is required. Therefore, the solution of the present invention deletes the auxiliary network and simplifies the model structure.
[0138] Head structure: The output layer of the PFLD model consists of a fully connected layer. The solution of the present invention directly regresses the coordinates (x, y) of the two key points of the pen tip and the pen tail. Therefore, the number of output channels of the final fully connected layer is 4.
[0139] Training PFLD model: Use the training set to train and optimize the model; use the test set to test the model and evaluate the performance of the model indicators. During model training, the input data is uniformly scaled to 112×112, the batch size batch_size=128, the optimization algorithm uses the Adam optimizer, the initial learning rate is set to lr=0.001, and the entire data set is iterated epoch=200 times;
[0140] Improved loss function: Due to the existence of auxiliary network, the original PFLD model needs to predict the Euler angles of the face posture in addition to the key points of the face. The loss function of the training model adopts L2Loss, which is expressed as follows: Where M represents the number of samples, N represents the number of key points, and γn represents the weights of different sample face postures. is the L2 norm, which measures the distance between key points. The solution of the present invention deletes the auxiliary network, so there is no need to consider γn. Therefore, the loss function does not use L2Loss, but uses the smooth L1 loss function SmoothL1Loss, and the formula is:
[0141] ;
[0142] The SmoothL1Loss loss function combines the advantages of L1 Loss and L2 Loss to a certain extent. When the difference between the predicted value and the target value is small, SmoothL1Loss behaves like L2 Loss, making the loss function smoother; when the difference is large, it behaves like L1 Loss and is less sensitive to outliers or outliers.
[0143] During the training process, the program tests the mean square error (MSE) of the model every time it completes a complete data iteration, and saves the model weight with the minimum current MSE as the optimal model;
[0144] In subsequent actual use, you only need to keep the model weight file and the corresponding inference program code to perform key point detection.
[0145] Deploy PFLD model: After training, you will get the key point detection model for the pen tip and pen tail of the electrician. You only need to deploy the model on the deep learning machine platform to perform model inference and testing to realize the key point detection of the pen tip and pen tail of the electrician. Please refer to Figure 4 .
[0146] Pen body detection: The present invention does not directly detect the pen body of the electrician test pen. The pen body can be detected by connecting the two ends of the pen tip and the pen tail. Please refer to Figure 5 .
[0147] Please refer to Figure 6 , Figure 6 A detection device 110 for an electrician's test pen based on deep learning provided in an embodiment of the present invention includes:
[0148] An acquisition module 1101 is used to acquire an image to be processed including an electrician test pen to be detected;
[0149] The detection module 1102 is used to input the image to be processed into a pre-trained single-stage target detection model for preliminary detection to obtain a preliminary detection result of the test pen, where the preliminary detection result of the test pen is the rectangular area where the test pen to be detected is located; call the pre-trained key point detection model to perform refined detection on the preliminary detection result of the test pen to obtain the tip point and tail point of the test pen to be detected; determine the body of the test pen based on the tip point and the tail point, and determine the tip point, the tail point and the body of the test pen as the target detection result of the test pen to be detected.
[0150] It should be noted that the implementation principle of the aforementioned deep learning-based electrician pen detection device 110 can refer to the implementation principle of the aforementioned deep learning-based electrician pen detection method, which will not be repeated here. It should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the deep learning-based electrician pen detection device 110 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above-mentioned deep learning-based electrician pen detection device 110. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in a processor element or an instruction in software form.
[0151] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0152] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned deep learning-based electrician pen detection device 110. Figure 7 As shown, Figure 7The computer device 100 is a block diagram of a structure of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a detection device 110 for an electrician's pen based on deep learning, a memory 111, a processor 112 and a communication unit 113.
[0153] In order to realize data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly. For example, the electrical connection between these elements can be realized through one or more communication buses or signal lines. The deep learning-based electrician test pen detection device 110 includes at least one software function module that can be stored in the memory 111 in the form of software or firmware or solidified in the operating system (OS) of the computer device 100. The processor 112 is used to execute the deep learning-based electrician test pen detection device 110 stored in the memory 111, such as the software function modules and computer programs included in the deep learning-based electrician test pen detection device 110.
[0154] An embodiment of the present invention provides a readable storage medium, which includes a computer program. When the computer program is running, it controls the computer device where the readable storage medium is located to execute the aforementioned deep learning-based electrician test pen detection method.
[0155] For illustrative purposes, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Numerous modifications and variations are possible in accordance with the above teachings. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.
Claims
1. A detection method for an electrician's pen based on deep learning. It is characterized in that include: Acquire an image to be processed including an electrician tester to be tested; Input the image to be processed into a pre-trained single-stage target detection model for preliminary detection to obtain a preliminary detection result of the electrician test pen, wherein the preliminary detection result of the electrician test pen is a rectangular area where the electrician test pen to be detected is located; Calling a pre-trained key point detection model to perform a detailed detection on the preliminary detection result of the electrician test pen, and obtaining the tip end point and the tail end point of the electrician test pen to be detected; Determine the body of the electrician test pen according to the pen tip point and the pen tail point, and determine the pen tip point, the pen tail point and the electrician test pen body as the target detection result of the electrician test pen to be detected; The single-stage target detection model is built based on the SSD model, and the single-stage target detection model includes a backbone network, an additional feature layer and a head structure; The backbone network is a MobileNet V2 structure, including a Conv12 convolutional layer and a Conv16 convolutional layer, which are used to generate two feature maps of corresponding sizes; The additional feature layer includes a Conv18 convolution layer and a Conv20 convolution layer, which are used to generate two feature maps of corresponding sizes; The head structure includes a default box generation, a classification subnetwork and a regression subnetwork, which are used to combine the four feature maps generated by the backbone network and the additional feature layer, perform multi-layer prediction fusion and non-maximum suppression, and obtain the type of the electrician pen to be detected and the rectangular area; The key point detection model is constructed based on PFLD, and the key point detection model includes a backbone network and a head structure; The backbone network is built based on MobileNet V2, including four stacked MobileNet V2 bottleneck modules; The bottleneck module includes an improved channel attention module; The channel attention module includes a global pooling layer, a fully connected layer, a normalization layer and an attention mechanism; the global pooling layer is used to perform global average pooling or global maximum pooling operations on each channel of the input feature map; the fully connected layer is used to extract the channel attention weights related to the task from the global statistical information; the normalization layer is used to standardize the attention weights; the attention mechanism is used to multiply the obtained channel attention coefficient vector by the original feature map element by channel to weight the features of each channel; The head structure includes a fully connected layer, the number of output channels of the fully connected layer is 4, and the fully connected layer is used to output the coordinate information of the tip point of the pen and the tail point of the pen.
2. The method according to claim 1, It is characterized in that The single-stage target detection model is trained in the following manner, including: The basic single-stage target detection model is trained and optimized based on the Adam optimizer; After each iteration of the training data for the basic single-stage target detection model is completed, the average model accuracy of the current cycle is obtained, and the model weight corresponding to the current highest average model accuracy is saved; When the preset training stop condition is reached, the single-stage target detection model that has completed training is obtained.
3. The method according to claim 1, It is characterized in that The key point detection model is trained in the following manner, including: The basic key point detection model is trained and optimized based on the Adam optimizer, and SmoothL1Loss is used as the loss function; After each iteration of the training data for the basic key point detection model is completed, the model mean square error of the current cycle is obtained, and the model weight corresponding to the current minimum mean square error is saved; When the preset training stop condition is reached, the key point detection model that has completed training is obtained.
4. The method according to claim 1, It is characterized in that The method further comprises: Obtain a pre-annotated electrician test pen rectangular frame and construct a training data set for training the single-stage target detection model; Expand the pre-marked test pen rectangular frame by a preset multiple to obtain a test pen area; A training data set for training the key point detection model is constructed based on the electrician test pen area.
5. The method according to claim 1, It is characterized in that The method of determining the body of the electrician test pen according to the pen tip end point and the pen tail end point comprises: Obtaining a line connecting the pen tip end point and the pen tail end point; According to the connection, the body of the electrician's pen is determined.
6. A detection device for an electrician's pen based on deep learning, It is characterized in that include: An acquisition module, used for acquiring an image to be processed including an electrician test pen to be detected; A detection module, used for inputting the image to be processed into a pre-trained single-stage target detection model for preliminary detection to obtain a preliminary detection result of the electrician test pen, wherein the preliminary detection result of the electrician test pen is a rectangular area where the electrician test pen to be detected is located; Calling the pre-trained key point detection model to perform a detailed detection on the preliminary detection result of the electrician test pen, and obtaining the pen tip point and the pen tail end point of the electrician test pen to be detected; determining the pen body of the electrician test pen according to the pen tip point and the pen tail end point, and determining the pen tip point, the pen tail end point and the pen body of the electrician test pen as the target detection result of the electrician test pen to be detected; The single-stage target detection model is built based on the SSD model, and the single-stage target detection model includes a backbone network, an additional feature layer and a head structure; The backbone network is a MobileNet V2 structure, including a Conv12 convolutional layer and a Conv16 convolutional layer, which are used to generate two feature maps of corresponding sizes; The additional feature layer includes a Conv18 convolution layer and a Conv20 convolution layer, which are used to generate two feature maps of corresponding sizes; The head structure includes a default box generation, a classification subnetwork and a regression subnetwork, which are used to combine the four feature maps generated by the backbone network and the additional feature layer, perform multi-layer prediction fusion and non-maximum suppression, and obtain the type of the electrician pen to be detected and the rectangular area; The key point detection model is constructed based on PFLD, and the key point detection model includes a backbone network and a head structure; The backbone network is built based on MobileNet V2, including four stacked MobileNet V2 bottleneck modules; The bottleneck module includes an improved channel attention module; The channel attention module includes a global pooling layer, a fully connected layer, a normalization layer and an attention mechanism; the global pooling layer is used to perform global average pooling or global maximum pooling operations on each channel of the input feature map; the fully connected layer is used to extract the channel attention weights related to the task from the global statistical information; the normalization layer is used to standardize the attention weights; the attention mechanism is used to multiply the obtained channel attention coefficient vector by the original feature map element by channel to weight the features of each channel; The head structure includes a fully connected layer, the number of output channels of the fully connected layer is 4, and the fully connected layer is used to output the coordinate information of the tip point of the pen and the tail point of the pen.
7. A computer device, It is characterized in that The computer device comprises a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 5.
8. A readable storage medium, It is characterized in that The readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 5.
