A communication maintenance tool identification method based on YOLO V5

Abstract

The present invention relates to a communication maintenance tool identification method based on YOLO V5, and belongs to the technical field of communication maintenance tool identification. The method of the present invention uses MOT semi-automatic annotation tool and VoTT visual object annotation tool to annotate communication line maintenance tool samples, generate a dedicated data set, and has practical value in the field of communication; the YOLO V5 model is improved by using attention mechanism, and the improved YOLO V5 model is used to perform data enhancement and model training on the communication line maintenance tool data set to improve detection efficiency and recognition accuracy; OpenVINO is used to optimize and accelerate model reasoning to improve detection speed; by deploying to a portable mobile terminal Raspberry Pi to perform real-time detection and result display of communication line maintenance tools, the current communication line maintenance tool identification problem is better solved, and the communication operation provides convenience for classification, storage and selection of communication line maintenance tool identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of communication maintenance tool identification, and in particular relates to a communication maintenance tool identification method based on YOLO V5. Background Art

[0002] There are various types of communication line maintenance tools, including program-controlled switching, network transmission, optical transmission, image transmission, communication power supply, etc., which are quite professional and complex. In the process of communication line maintenance operations, operators often use tools improperly, which not only affects the efficiency of "rushing to replace communication", but also may cause irreversible damage to communication lines and interfaces. In order to reduce the error rate of communication operators in selecting communication maintenance tools, this paper takes the identification of communication maintenance tools as the starting point, and provides guidance for communication operators to correctly select tools and identify violations by accurately identifying them. There are many communication line maintenance tools, and with the continuous optimization and updating of information network and optical communication technology and equipment, the types of communication line maintenance tools are increasing, which brings certain difficulties to identification and detection. Traditional machine learning related algorithms cannot better identify a considerable number of communication line maintenance tools. However, there are few studies on the identification and classification of communication line maintenance tools, which cannot better facilitate communication operators to correctly select tools, nor can they provide prompts for the discovery of violations. Currently, deep learning detection technology is widely used in the communications technology industry. Among them, the YOLO system algorithm has become one of the important models for engineering applications and implementation due to its good real-time performance and high accuracy.

[0003] Referring to a method for detecting electric tools based on the YOLO V5 model with reference to Chinese patent publication number CN113947527A, the method realizes the recognition and classification of electric tools based on the YOLO V5 algorithm, and performs image enhancement on the communication line maintenance tool samples, and performs transfer learning on the data set through preprocessing, so as to further improve the accuracy of electronic maintenance tool recognition. To transfer this research method from the field of electric tools to the field of communication line maintenance tools, it is necessary to produce a new data set, focus on solving the problem of the algorithm's reduced recognition rate in multiple application scenarios such as indoors, outdoors, at night, and in strong light, and realize deployment to mobile terminals for easy portability. To this end, the present invention fully considers the application scenarios, shapes, sizes, colors and other characteristics of communication line detection tools, introduces a self-attention mechanism to improve the YOLO V5 algorithm, and improves the recognition accuracy. Considering the mobility requirements, OpenVINO is used to infer and accelerate the improved YOLO V5 algorithm, and it is deployed on the mobile terminal to achieve better results. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] The technical problem to be solved by the present invention is how to provide a communication maintenance tool recognition method based on YOLO V5 to solve the problems of decreased recognition rate and inconvenience in use in multiple application scenarios such as indoors, outdoors, at night, and in strong light.

[0006] (II) Technical solution

[0007] In order to solve the above technical problems, the present invention proposes a communication maintenance tool identification method based on YOLO V5, which comprises the following steps:

[0008] S1: Use the MOT semi-automatic annotation tool and the VoTT visual object annotation tool to annotate the video clips and images of the communication maintenance tools to obtain the communication maintenance tool dataset;

[0009] S2: Based on the deep separable residual network structure, SE, CA or CBAM attention mechanism is introduced to improve the YOLO V5 backbone network;

[0010] S3: Use the improved YOLO V5 backbone network to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain the trained model parameters and weights;

[0011] S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained YOLO V5 backbone network;

[0012] S5: Deploy the optimized model to the Raspberry Pi mobile terminal, detect the standardized image of the communication line maintenance tool, and output the category of the corresponding communication line maintenance tool.

[0013] Furthermore, step S1 specifically includes: collecting video clips and pictures of communication line maintenance tools through a camera device to form a sample, using the MOT semi-automatic annotation tool to semi-automatically annotate the video clips of the communication maintenance tools, and using the VoTT visual object annotation tool to perform data annotation on the pictures of the communication maintenance tools, and the two annotation results are summarized to obtain a communication maintenance tool data set.

[0014] Furthermore, the step S2 specifically includes the following steps: select one of the three network components SENet (Squeeze-and-Excitation Networks), CA (Channel Attention), and CBAM (Convolutional Block Attention Module) to replace the CSP1_1 network component of the Backbone part of the YOLO V5 network structure;

[0015] For the SENet network component, the channel attention mechanism is introduced. Through the two steps of squeezing and excitation, the channel dimension is optimized and a small number of parameters are added, so that the YOLO V5 backbone network can obtain features on different channels.

[0016] For the CA network component, the channel attention is decomposed into two 1D feature encoding processes, which aggregate features along two spatial directions to obtain long-range dependencies and retain precise location information, respectively;

[0017] For the CBAM network component, the above SENet and CA are connected in series as two independent components, and the spatial attention mechanism and channel attention mechanism are comprehensively considered.

[0018] Furthermore, the S3 specifically includes the following steps:

[0019] S31: The improved YOLO V5 backbone network uses the Mosaic data enhancement method to enhance the features of the labeled communication maintenance tool image to obtain enhanced image data;

[0020] S32: dividing the communication maintenance tool image dataset into a training set, a validation set, and a test set in a ratio of 3:1:1;

[0021] S33: When calculating the improved YOLO V5 backbone network, three loss functions are used: BCE, Focal loss, and QFocal loss. The effects of these three loss functions in training and detection are compared, and the optimal one is selected;

[0022] S34: Use the improved YOLO V5 backbone network to train the communication detection tool data set, and after the training is completed, obtain the target parameters and target weights recognized by the communication line maintenance tool.

[0023] Furthermore, the BCE loss function is:

[0024]

[0025] in Represents the probability that the i-th sample of the YOLO V5 backbone network belongs to a certain category, N is the total number of samples; y(i) represents the true category of the i-th sample, and its value is 0 or 1.

[0026] Furthermore, the Focal loss function is:

[0027]

[0028] Where p represents the predicted probability. The larger its value is, the closer it is to category y, that is, the more accurate the classification is. γ is an introduced hyperparameter with a value range of [0,5]. It can significantly reduce the loss value of positive samples with higher probability and negative samples with lower probability, thereby improving the model's ability to distinguish more difficult samples.

[0029] Furthermore, the QFocal loss function is:

[0030] QFL(σ)=-|y-σ| β ((1-y)log(1-σ)+ylog(σ))

[0031] Where σ represents the prediction result, the probability of each image corresponding to a certain category ranges from 0 to 1, and β is the proportional coefficient.

[0032] Furthermore, the feature enhancement includes adding noise, changing brightness and changing chrominance.

[0033] Furthermore, the step S4 specifically includes the following steps:

[0034] S41: Convert the pytorch model file of the improved YOLO V5 backbone network into the universal open model format ONNX, and further convert the ONNX format file through the model conversion script of OpenVINO to generate IR model files, namely .bin files and .xml files;

[0035] S42: Complete model parsing and calling based on OpenVINO SDK, set input and output formats, the YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order, the input and output format data are set to floating point numbers, set the input image data and implement inference prediction, and obtain the optimized YOLO V5 backbone network.

[0036] Furthermore, the step S5 specifically includes the following steps:

[0037] S51: Improve the operation interface, add three buttons: test picture, test video and test camera, obtain image sources and perform tests through these three ways, and convert the images of the communication line maintenance tools to be tested into images with a resolution of 640*640;

[0038] S52: Deploy the optimized YOLO V5 model and the converted IR model file to the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

[0039] (III) Beneficial effects

[0040] The present invention proposes a communication maintenance tool identification method based on YOLO V5. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2 This is the improved YOLO V5 network structure diagram;

[0043] Figure 3 This is the internal structure diagram of the attention mechanism. DETAILED DESCRIPTION

[0044] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0045] The present invention provides a communication maintenance tool identification method based on YOLO V5, which can better solve the current communication line maintenance tool identification problem and provide convenience for communication operations to classify, store and select communication line maintenance tools.

[0046] The invention provides a communication maintenance tool identification method based on YOLO V5. The method adopts MOT semi-automatic annotation tool and VoTT visual object annotation tool to annotate communication line maintenance tool samples, generates a dedicated data set, and has practical value in the communication field; adopts attention mechanism to improve YOLO V5 model, and uses the improved YOLO V5 model to perform data enhancement and model training on the communication line maintenance tool data set, so as to improve detection efficiency and recognition accuracy; adopts OpenVINO to optimize and accelerate model reasoning, so as to improve detection speed; and performs real-time detection and result display on communication line maintenance tools by deploying them on a portable mobile terminal Raspberry Pi, so as to better solve the current communication line maintenance tool identification problem, and provide convenience for communication operations to classify, store and select communication line maintenance tools.

[0047] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0048] A communication maintenance tool identification method based on YOLO V5, comprising the following steps:

[0049] S1: Use the MOT semi-automatic annotation tool and the VoTT visual object annotation tool to annotate the video clips and images of the communication maintenance tools to obtain the communication maintenance tool dataset;

[0050] S2: Based on the deep separable residual network structure, three attention mechanisms, SE, CA, and CBAM, are introduced to improve the YOLOV5 backbone network;

[0051] S3: Use the improved YOLO V5 backbone network to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain the trained model parameters and weights;

[0052] S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained YOLO V5 backbone network;

[0053] S5: Deploy the optimized model to the Raspberry Pi mobile terminal, detect the standardized image of the communication line maintenance tool, and output the category of the corresponding communication line maintenance tool.

[0054] Furthermore, the step S1 specifically includes the following steps:

[0055] Video clips and pictures of communication line maintenance tools are collected by camera equipment to form samples. The video clips of communication maintenance tools are semi-automatically annotated using the MOT semi-automatic annotation tool. The pictures of communication maintenance tools are annotated using the VoTT visual object annotation tool. The two annotation results are summarized to obtain the communication maintenance tool dataset.

[0056] Furthermore, the step S2 specifically includes the following steps:

[0057] Choose one of the three network components: SENet (Squeeze-and-Excitation Networks), CA (Channel Attention), and CBAM (Convolutional Block Attention Module), and replace the CSP1_1 network component (512×20×20) of the Backbone part of the YOLO V5 network structure:

[0058] For the SENet network component, a channel attention mechanism is introduced to optimize the channel dimension through two steps of squeezing and excitation, and a small number of parameters are added to enable the YOLO V5 backbone network to obtain features on different channels, thereby improving the accuracy;

[0059] For the CA network component, the channel attention is decomposed into two 1D feature encoding processes, which aggregate features along two spatial directions to obtain long-range dependencies and retain precise location information, respectively, thereby enhancing the feature representation of object detection;

[0060] For the CBAM network component, the above SENet and CA are connected in series as two independent sub-modules, and the spatial attention mechanism and channel attention mechanism are comprehensively considered.

[0061] Furthermore, the S3 specifically includes the following steps:

[0062] S31: The improved YOLO V5 backbone network uses the Mosaic data enhancement method to enhance the features of the labeled communication maintenance tool image to obtain enhanced image data;

[0063] S32: dividing the communication maintenance tool image dataset into a training set, a validation set, and a test set in a ratio of 3:1:1;

[0064] S33: There are three loss functions that can be used to calculate the improved YOLO V5 backbone network: BCE, Focal loss, and QFocal loss. Compare the effects of these three loss functions in training and detection, and select the best one.

[0065] S34: Use the improved YOLO V5 backbone network to train the communication detection tool data set. After the training is completed, the target parameters and target weights for communication line maintenance tool recognition are obtained. The total number of samples N is 5000, and there are 7 categories of target detection: program-controlled switching, network testing, time unification, image transmission, optical fiber transmission, voice communication, and communication power supply. The target parameters and target weights are the parameters and weights used in the YOLO V5 backbone network, which are generally tens of millions and need to be placed in the model file.

[0066] Furthermore, the BCE loss function is:

[0067]

[0068] in Represents the probability that the i-th sample in the YOLO V5 backbone network belongs to a certain category, and N is the total number of samples. y(i) represents the true category of the i-th sample, and its value is 0 or 1.

[0069] Furthermore, the Focal loss function is:

[0070]

[0071] Among them, p represents the predicted probability. The larger its value is, the closer it is to category y, that is, the more accurate the classification is. p also reflects the difficulty of classification. The larger its value is, the higher the confidence of classification is, which means that the sample is easier to distinguish. The smaller its value is, the lower the confidence of classification is, which means that the sample is more difficult to distinguish. γ is an introduced hyperparameter with a value range of [0,5]. It can significantly reduce the loss value of positive samples with higher probability and negative samples with lower probability, and improve the model's ability to distinguish difficult samples.

[0072] Furthermore, the QFocal loss function is:

[0073] QFL(σ)=-|y-σ|β ((1-y)log(1-σ)+ylog(σ))

[0074] Where σ represents the prediction result, the probability of each image corresponding to a certain category ranges from 0 to 1, and β is the proportional coefficient.

[0075] Furthermore, the step S4 specifically includes the following steps:

[0076] S41: Convert the pytorch model file of the improved YOLO V5 backbone network into the universal open model format ONNX, and further convert the ONNX format file through the model conversion script of OpenVINO to generate IR model files, namely .bin files and .xml files;

[0077] S42: Complete model parsing and calling based on OpenVINO SDK, set input and output formats, the YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order, the input and output format data are set to floating point numbers, set the input image data and implement inference prediction, and obtain the optimized YOLO V5 backbone network.

[0078] Furthermore, the step S5 specifically includes the following steps:

[0079] S51: Improve the operation interface and add three buttons: test picture, test video and test camera. The image source can be obtained and tested through these three ways, and the image of the communication line maintenance tool to be tested is uniformly converted into an image with a resolution of 640*640;

[0080] S52: Deploy the optimized YOLO V5 model and the converted IR model file to the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

[0081] Furthermore, the feature enhancement includes adding noise, changing brightness and changing chrominance; and the value of β is 2.

[0082] The figures are for illustrative purposes only and should not be construed as limiting the present patent;

[0083] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0084] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0085] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0086] Example 1

[0087] A communication maintenance tool identification method based on YOLO V5, comprising the following steps:

[0088] S1: Use the MOT semi-automatic annotation tool and the VoTT visual object annotation tool to annotate the video clips and images of the communication maintenance tools to obtain the communication maintenance tool dataset;

[0089] S2: Based on the deep separable residual network structure, three attention mechanisms, SE, CA, and CBAM, are introduced to improve the YOLOV5 backbone network;

[0090] S3: Use the improved YOLO V5 backbone network to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain the trained model parameters and weights;

[0091] S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained YOLO V5 backbone network;

[0092] S5, deploying the optimized model to the Raspberry Pi mobile terminal, detecting the standardized image of the communication line maintenance tool, and outputting the category of the corresponding communication line maintenance tool.

[0093] Step S1 specifically includes the following steps:

[0094] Video clips and pictures of communication line maintenance tools are collected by camera equipment to form samples. The video clips of communication maintenance tools are semi-automatically annotated using the MOT semi-automatic annotation tool. The pictures of communication maintenance tools are annotated using the VoTT visual object annotation tool. The two annotation results are summarized to obtain 5,000 high-quality samples to form a communication maintenance tool dataset.

[0095] Step S2 specifically includes the following steps:

[0096] Choose one of the three network components: SENet (Squeeze-and-Excitation Networks), CA (ChannelAttention), and CBAM (Convolutional BlockAttention Module), and replace the CSP1_1 network component (512×20×20) in the Backbone part of the YOLO V5 network structure:

[0097] For the SENet network component, a channel attention mechanism is introduced to optimize the channel dimension through two steps of squeezing and excitation, and a small number of parameters are added to enable the YOLO V5 backbone network to obtain features on different channels, thereby improving the accuracy;

[0098] For the CA network component, the channel attention is decomposed into two 1D feature encoding processes, which aggregate features along two spatial directions to obtain long-range dependencies and retain precise location information, respectively, thereby enhancing the feature representation of object detection;

[0099] For the CBAM network component, the above SENet and CA are connected in series as two independent sub-modules, and the spatial attention mechanism and channel attention mechanism are comprehensively considered.

[0100] Step S3 specifically includes the following steps:

[0101] S31: The improved YOLO V5 backbone network uses the Mosaic data enhancement method to enhance the features of the labeled communication maintenance tool image to obtain enhanced image data;

[0102] S32: dividing the communication maintenance tool image dataset into a training set, a validation set, and a test set in a ratio of 3:1:1;

[0103] S33: There are three loss functions that can be used to calculate the improved YOLO V5 backbone network: BCE, Focal loss, and QFocal loss. Compare the effects of these three loss functions in training and detection, and select the best one.

[0104] S34: Use the improved YOLO V5 backbone network to train the communication detection tool data set, and after the training is completed, obtain the target parameters and target weights recognized by the communication line maintenance tool.

[0105] The BCE loss function is:

[0106]

[0107] in Represents the probability that the i-th sample of the YOLO V5 backbone network belongs to a certain category.

[0108] The focal loss function is:

[0109]

[0110] where p t Represents the predicted probability. The larger its value is, the closer it is to category y, that is, the more accurate the classification is. γ is an introduced hyperparameter that significantly reduces the loss value of positive samples with higher probability and negative samples with lower probability, thereby improving the model's ability to distinguish more difficult samples.

[0111] The QFocal loss function is:

[0112] QFL(σ)=-|y-σ|β ((1-y)log(1-σ)+ylog(σ))

[0113] The probability of each image corresponding to a certain category ranges from 0 to 1.

[0114] Step S4 specifically includes the following steps:

[0115] S41: Convert the pytorch model file of the improved YOLO V5 backbone network into the universal open model format ONNX, and further convert the ONNX format file through the model conversion script of OpenVINO to generate IR model files, namely .bin files and .xml files;

[0116] S42: Complete model parsing and calling based on OpenVINO SDK, set input and output formats, the YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order, the input and output format data are set to floating point numbers, set the input image data and implement inference prediction, and obtain the optimized YOLO V5 backbone network.

[0117] Step S5 specifically includes the following steps:

[0118] S51: Improve the operation interface and add three buttons: test picture, test video and test camera. The image source can be obtained and tested through these three ways, and the image of the communication line maintenance tool to be tested is uniformly converted into an image with a resolution of 640*640;

[0119] S52: Deploy the optimized YOLO V5 model and the converted IR model file to the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

[0120] Example 2

[0121] like Figure 1 As shown, a communication maintenance tool identification method based on YOLO V5 includes the following steps:

[0122] S1: Collect and annotate sample image data of communication line maintenance tools; S1 specifically includes the following steps: Collect sample image data of communication line maintenance tools; Use the MOT semi-automatic annotation tool to semi-automatically annotate video clips of communication maintenance tools. Use the VoTT visual object annotation tool to annotate the images of communication maintenance tools. The two annotation results are summarized to obtain about 5,000 high-quality samples to form a communication maintenance tool dataset.

[0123] S2: Introduce three attention mechanisms: SE, CA, and CBAM, and improve the YOLO V5 backbone network. Figure 2 and Figure 3 , including the following steps:

[0124] S201: As attached Figure 2 As shown in the figure, in the Backbone part of the YOLO V5 network structure, the CSP_1 network component can be replaced by three network components: SENet, CA, and CSMA. Different types of attention mechanisms are introduced to improve the YOLO V5 model and enhance the detection accuracy.

[0125] S202: As attached Figure 3 As shown, there are 4 network components in total. The SENet network component introduces the channel attention mechanism, optimizes the channel dimension through two steps of squeezing and excitation, and adds a small number of parameters, so that the model can better obtain the features on different channels, thereby improving the accuracy. The CA network component decomposes the channel attention into two 1-dimensional feature encoding processes, aggregates features along two spatial directions, obtains long-range dependencies and retains precise location information, thereby enhancing the feature representation of target detection. The CBAM network component combines the above SENet and CA as two independent sub-modules, comprehensively considering the spatial attention mechanism and the channel attention mechanism, which is a hybrid attention mechanism;

[0126] S3: Use the improved model to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain trained model parameters and weights, which specifically includes the following steps:

[0127] S301: The improved YOLO V5 model uses the Mosaic data enhancement method to perform feature enhancement on the annotated communication maintenance tool image to obtain enhanced image data, wherein the feature enhancement includes adding noise, changing brightness, and changing chromaticity;

[0128] Among them, the method of adding noise is: randomly selecting the number of pixels to add noise as 1 / 10 of the total pixels, and covering them with Gaussian noise, so as to avoid excessive feature loss under the condition of appropriately increasing noise; the method of changing brightness is: randomly changing the brightness of the image, and the random proportion of brightness change is 0.5, 0.75, 1.25, 1.5, so as to simulate the change of light intensity in the real working environment; the method of changing chromaticity is: randomly transforming the chromaticity of the image, randomly adjusting the three channels of RGB of the image by 0.5, 1, 1.5 respectively, and then changing the channels, so as to simulate the physical phenomena such as light refraction and reflection in the real environment;

[0129] S302: Divide the communication maintenance tool image dataset into training set, validation set and test set in a ratio of 3:1:1. The images and labels folders contain three folders: train, val and test, which represent the training set, validation set and test set respectively. The three folders in the images folder store the images, and the three folders in the labels folder store the corresponding labeled yolo format data labels;

[0130] S303: The improved YOLO V5 model uses three loss functions: BCE (Binary Cross Entropy loss), Focal loss, and QFocal loss.

[0131] The BCE loss function is:

[0132]

[0133] in Represents the probability that the i-th sample of the YOLO V5 backbone network belongs to a certain category.

[0134] The Focal loss function is:

[0135]

[0136] where p t Represents the predicted probability. The larger its value is, the closer it is to category y, that is, the more accurate the classification is. γ is an introduced hyperparameter that significantly reduces the loss value of positive samples with higher probability and negative samples with lower probability, thereby improving the model's ability to distinguish more difficult samples.

[0137] The QFocal loss function is:

[0138] QFL(σ)=-|y-σ| β ((1-y)log(1-σ)+ylog(σ))

[0139] The probability of each image corresponding to a certain category ranges from 0 to 1, and the effect is best when β is 2.

[0140] S304: The communication detection tool data set is trained using the improved YOLO V5 model, and after the training is completed, target parameters and target weights for communication line maintenance tool recognition are obtained.

[0141] S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained model, which includes the following steps:

[0142] S401: Convert the pytorch model file of the improved YOLO V5 model to the universal open model format ONNX. Further convert the ONNX format file through the model conversion script of OpenVINO to generate an IR model file (.bin file and .xml file);

[0143] S402: Complete model parsing and calling based on OpenVINO SDK. Set the input and output formats. The YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order. The input and output format data are set to floating point numbers. Set the input image data and implement inference prediction to obtain the optimized YOLO V5 model.

[0144] S5: Deploy the optimized model to the Raspberry Pi mobile terminal, detect the standardized images of the communication line maintenance tools, and output the corresponding categories of the communication line maintenance tools. Specifically, the following steps are included:

[0145] S501: Improve the operation interface and add three buttons: test picture, test video and test camera. The image source can be obtained and tested through these three ways. The image of the communication line maintenance tool to be tested is uniformly converted into a 640*640 resolution image;

[0146] S502: deploy the optimized YOLO V5 model and the converted IR model file on the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

[0147] Example 3

[0148] like Figure 1 As shown, a communication maintenance tool identification method based on YOLO V5 includes the following steps:

[0149] S1: Collect sample image data of communication line maintenance tools and annotate them; S1 specifically includes the following steps: Collect sample image data of communication line maintenance tools; Use the MOT semi-automatic annotation tool to semi-automatically annotate video clips of communication maintenance tools. Use the VoTT visual object annotation tool to annotate the images of communication maintenance tools. The two annotation results are combined to obtain a communication maintenance tool dataset; Let the annotated labels have a total of 7 categories: (1) program-controlled switching, (2) network testing, (3) time unification, (4) image transmission, (5) optical fiber transmission, (6) voice communication, (7) communication power supply;

[0150] S2: Introduce three attention mechanisms: SE, CA, and CBAM, and improve the YOLO V5 backbone network. Figure 2 and Figure 3 , including the following steps:

[0151] S201: As attached Figure 2 As shown in the figure, in the Backbone part of the YOLO V5 network structure, the CSP_1 network component can be replaced by three network components: SENet, CA, and CSMA. Different types of attention mechanisms are introduced to improve the YOLO V5 model and enhance the detection accuracy.

[0152] S202: As attached Figure 3 As shown, there are 4 network components in total. The SENet network component introduces the channel attention mechanism, optimizes the channel dimension through two steps of squeezing and excitation, and adds a small number of parameters, so that the model can better obtain the features on different channels, thereby improving the accuracy. The CA network component decomposes the channel attention into two 1-dimensional feature encoding processes, aggregates features along two spatial directions, obtains long-range dependencies and retains precise location information, thereby enhancing the feature representation of target detection. The CBAM network component combines the above SENet and CA as two independent sub-modules, comprehensively considering the spatial attention mechanism and the channel attention mechanism, which is a hybrid attention mechanism;

[0153] S3: Use the improved model to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain trained model parameters and weights, which specifically includes the following steps:

[0154] S301: The improved YOLO V5 model uses the Mosaic data enhancement method to perform feature enhancement on the annotated communication maintenance tool image to obtain enhanced image data, wherein the feature enhancement includes adding noise, changing brightness, and changing chromaticity. The method of adding noise is: randomly selecting the number of pixels to be added with noise as 1 / 10 of the total pixels, and covering them with Gaussian noise, so as to avoid excessive feature loss while appropriately increasing the noise. The method of changing the brightness is: randomly changing the brightness of the image, and the random proportions of the brightness changes are 0.5, 0.75, 1.25, and 1.5, thereby simulating the changes in light intensity in a real working environment. The method of changing the chromaticity is: randomly transforming the chromaticity of the image, randomly adjusting the three channels of RGB of the image by 0.5, 1, and 1.5 respectively, and then changing the channels, thereby simulating the physical phenomena such as light refraction and reflection in a real environment;

[0155] S302: Divide the communication maintenance tool image dataset into training set, validation set and test set in a ratio of 3:1:1. The images and labels folders contain three folders: train, val and test, which represent the training set, validation set and test set respectively. The three folders in the images folder store the images, and the three folders in the labels folder store the corresponding labeled yolo format data labels;

[0156] S303: The improved YOLO V5 model uses three loss functions: BCE (Binary Cross Entropy loss), Focal loss, and QFocal loss.

[0157] The BCE loss function is:

[0158]

[0159] in Represents the probability that the i-th sample of the YOLO V5 backbone network belongs to a certain category.

[0160] The Focal loss function is:

[0161]

[0162] where p t Represents the predicted probability. The larger its value is, the closer it is to category y, that is, the more accurate the classification is. γ is an introduced hyperparameter that significantly reduces the loss value of positive samples with higher probability and negative samples with lower probability, thereby improving the model's ability to distinguish more difficult samples.

[0163] The QFocal loss function is:

[0164] QFL(σ)=-|y-σ| β ((1-y)log(1-σ)+ylog(σ))

[0165] The probability of each image corresponding to a certain category ranges from 0 to 1, and the effect is best when β is 2.

[0166] S304: The communication detection tool data set is trained using the improved YOLO V5 model, and after the training is completed, target parameters and target weights for communication line maintenance tool recognition are obtained.

[0167] S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained model, which includes the following steps:

[0168] S401: Convert the pytorch model file of the improved YOLO V5 model to the universal open model format ONNX. Further convert the ONNX format file through the model conversion script of OpenVINO to generate an IR model file (.bin file and .xml file);

[0169] S402: Complete model parsing and calling based on OpenVINO SDK. Set the input and output formats. The YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order. The input and output format data are set to floating point numbers. Set the input image data and implement inference prediction to obtain the optimized YOLO V5 model.

[0170] S5: Deploy the optimized model to the Raspberry Pi mobile terminal, detect the standardized images of the communication line maintenance tools, and output the corresponding categories of the communication line maintenance tools. Specifically, the following steps are included:

[0171] S501: Improve the operation interface and add three buttons: test picture, test video and test camera. The image source can be obtained and tested through these three ways. The image of the communication line maintenance tool to be tested is uniformly converted into a 640*640 resolution image;

[0172] S502: deploy the optimized YOLO V5 model and the converted IR model file on the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

[0173] The optimized model obtained by the present invention is tested and compared with the original model on the same communication maintenance tool test set under the same number of training times and the same hyperparameter conditions. The detection accuracy of the optimized model is improved by 0.9%, and the detection speed is increased by 5.1 times. The present invention collects pictures and videos of communication line maintenance tools to form samples, and produces a training data set to train the improved YOLO V5 model, and accelerates reasoning through the optimizer to realize the recognition of communication line maintenance tools, reduce the error rate of communication line maintenance tool selection, and facilitate the classification and storage of communication tools, the selection of communication operation tools, and the identification of violations.

[0174] The same or similar reference numerals correspond to the same or similar components;

[0175] The positional relationships described in the drawings are only for illustrative purposes and should not be construed as limiting the present patent.

[0176] Embodiment 4:

[0177] A communication maintenance tool identification method based on YOLO V5, comprising the following steps:

[0178] S1: Use the MOT semi-automatic annotation tool and the VoTT visual object annotation tool to annotate the video clips and images of the communication maintenance tools to obtain the communication maintenance tool dataset;

[0179] S2: Based on the deep separable residual network structure, three attention mechanisms, SE, CA, and CBAM, are introduced to improve the YOLOV5 backbone network;

[0180] S3: Use the improved YOLO V5 backbone network to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain the trained model parameters and weights;

[0181] S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained YOLO V5 backbone network;

[0182] S5: Deploy the optimized model to the Raspberry Pi mobile terminal, detect the standardized image of the communication line maintenance tool, and output the category of the corresponding communication line maintenance tool.

[0183] Furthermore, the step S1 specifically includes the following steps:

[0184] Video clips and pictures of communication line maintenance tools are collected by camera equipment to form samples. The video clips of communication maintenance tools are semi-automatically annotated using the MOT semi-automatic annotation tool. The pictures of communication maintenance tools are annotated using the VoTT visual object annotation tool. The two annotation results are summarized to obtain the communication maintenance tool dataset.

[0185] Furthermore, the step S2 specifically includes the following steps:

[0186] Use SENet, CA, and CSMA network components to replace the CSP_1 network component of the Backbone part of the YOLO V5 network structure:

[0187] For the SENet network component, a channel attention mechanism is introduced to optimize the channel dimension through two steps of squeezing and excitation, and a small number of parameters are added to enable the YOLO V5 backbone network to obtain features on different channels, thereby improving the accuracy;

[0188] For the CA network component, the channel attention is decomposed into two 1D feature encoding processes, which aggregate features along two spatial directions to obtain long-range dependencies and retain precise location information, respectively, thereby enhancing the feature representation of object detection;

[0189] For the CBAM network component, the above SENet and CA are combined as two independent sub-modules, and the spatial attention mechanism and channel attention mechanism are comprehensively considered.

[0190] Furthermore, the S3 specifically includes the following steps:

[0191] S31: The improved YOLO V5 backbone network uses the Mosaic data enhancement method to enhance the features of the labeled communication maintenance tool image to obtain enhanced image data;

[0192] S32: dividing the communication maintenance tool image dataset into a training set, a validation set, and a test set in a ratio of 3:1:1;

[0193] S33: Calculate the three loss functions BCE, Focal loss, and QFocalloss of the improved YOLO V5 backbone network;

[0194] S34: Use the improved YOLO V5 backbone network to train the communication detection tool data set, and after the training is completed, obtain the target parameters and target weights recognized by the communication line maintenance tool.

[0195] Furthermore, the BCE loss function is:

[0196]

[0197] in Represents the probability that the i-th sample of the YOLO V5 backbone network belongs to a certain category.

[0198] Furthermore, the Focal loss function is:

[0199]

[0200] where p t Represents the predicted probability. The larger its value is, the closer it is to category y, that is, the more accurate the classification is. γ is an introduced hyperparameter that significantly reduces the loss value of positive samples with higher probability and negative samples with lower probability, thereby improving the model's ability to distinguish more difficult samples.

[0201] Furthermore, the QFocal loss function is:

[0202] QFL(σ)=-|y-σ| β ((1-y)log(1-σ)+ylog(σ))

[0203] The probability of each image corresponding to a certain category ranges from 0 to 1.

[0204] Furthermore, the step S4 specifically includes the following steps:

[0205] S41: Convert the pytorch model file of the improved YOLO V5 backbone network into the universal open model format ONNX, and further convert the ONNX format file through the model conversion script of OpenVINO to generate IR model files, namely .bin files and .xml files;

[0206] S42: Complete model parsing and calling based on OpenVINO SDK, set input and output formats, the YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order, the input and output format data are set to floating point numbers, set the input image data and implement inference prediction, and obtain the optimized YOLO V5 backbone network.

[0207] Furthermore, the step S5 specifically includes the following steps:

[0208] S51: Improve the operation interface and add three buttons: test picture, test video and test camera. The image source can be obtained and tested through these three ways, and the image of the communication line maintenance tool to be tested is uniformly converted into an image with a resolution of 640*640;

[0209] S52: Deploy the optimized YOLO V5 model and the converted IR model file to the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

[0210] Furthermore, the feature enhancement includes adding noise, changing brightness and changing chrominance; and the value of β is 2.

[0211] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0212] The present invention adopts MOT semi-automatic labeling tool and VoTT visual object labeling tool to label communication line maintenance tool samples, generates a special data set, and has practical value in the communication field; adopts attention mechanism to improve YOLO V5 model, and uses the improved YOLO V5 model to perform data enhancement and model training on the communication line maintenance tool data set, so as to improve detection efficiency and recognition accuracy; adopts OpenVINO to optimize and accelerate model reasoning, so as to improve detection speed; and by deploying to the portable mobile terminal Raspberry Pi, the communication line maintenance tool is detected in real time and the result is displayed, so as to better solve the current communication line maintenance tool identification problem and provide convenience for communication operations to classify, store and select the communication line maintenance tool.

[0213] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A communication maintenance tool identification method based on YOLO V5, characterized in that: The method comprises the following steps: S1: Use the MOT semi-automatic annotation tool and the VoTT visual object annotation tool to annotate the video clips and images of the communication maintenance tools to obtain the communication maintenance tool dataset; S2: Based on the deep separable residual network structure, SE, CA or CBAM attention mechanism is introduced to improve the YOLO V5 backbone network; S3: Use the improved YOLO V5 backbone network to perform image enhancement on the communication maintenance tool dataset and perform model training to obtain the trained model parameters and weights; S4: Use OpenVINO optimizer to perform inference and performance optimization on the trained YOLO V5 backbone network; S5: deploy the optimized model to the Raspberry Pi mobile terminal, detect the standardized image of the communication line maintenance tool, and output the category of the corresponding communication line maintenance tool; in, The step S2 specifically includes the following steps: select one of the three network components SENet (Squeeze-and-Excitation Networks), CA (Channel Attention), and CBAM (Convolutional Block Attention Module) to replace the CSP1_1 network component of the Backbone part of the YOLO V5 network structure; For the SENet network component, the channel attention mechanism is introduced. Through the two steps of squeezing and excitation, the channel dimension is optimized and a small number of parameters are added, so that the YOLO V5 backbone network can obtain features on different channels. For the CA network component, the channel attention is decomposed into two 1D feature encoding processes, which aggregate features along two spatial directions to obtain long-range dependencies and retain precise location information, respectively; For the CBAM network component, the above SENet and CA are connected in series as two independent components, and the spatial attention mechanism and channel attention mechanism are comprehensively considered; The step S4 specifically comprises the following steps: S41: Convert the pytorch model file of the improved YOLO V5 backbone network into the universal open model format ONNX, and further convert the ONNX format file through the model conversion script of OpenVINO to generate IR model files, namely .bin files and .xml files; S42: Complete model parsing and calling based on OpenVINO SDK, set input and output formats, the YOLO V5 input image is normalized between 0 and 1, and is in RGB channel order, the input and output format data are set to floating point numbers, set the input image data and implement inference prediction, and obtain the optimized YOLO V5 backbone network.

2. The communication maintenance tool identification method based on YOLO V5 as claimed in claim 1, characterized in that: The step S1 specifically includes: collecting video clips and pictures of the communication line maintenance tool by means of a camera to form a sample, semi-automatically annotating the video clips of the communication maintenance tool by means of the MOT semi-automatic annotation tool, and annotating the pictures of the communication maintenance tool by means of the VoTT visual object annotation tool, and aggregating the two annotation results to obtain a communication maintenance tool data set.

3. The communication maintenance tool identification method based on YOLO V5 as claimed in claim 1, characterized in that: The S3 specifically includes the following steps: S31: The improved YOLO V5 backbone network uses the Mosaic data enhancement method to enhance the features of the labeled communication maintenance tool image to obtain enhanced image data; S32: dividing the communication maintenance tool image dataset into a training set, a validation set, and a test set in a ratio of 3:1:1; S33: When calculating the improved YOLO V5 backbone network, three loss functions are used: BCE, Focalloss, and QFocalloss. The effects of these three loss functions in training and detection are compared, and the optimal one is selected; S34: Use the improved YOLO V5 backbone network to train the communication detection tool data set, and after the training is completed, obtain the target parameters and target weights recognized by the communication line maintenance tool.

4. The communication maintenance tool identification method based on YOLO V5 as claimed in claim 3, characterized in that: The BCE loss function is: in represents the probability that the i-th sample of the YOLO V5 backbone network belongs to a certain category, and N is the total number of samples; y (i) Indicates the true category of the i-th sample, and its value is 0 or 1.

5. The communication maintenance tool identification method based on YOLO V5 as claimed in claim 3, characterized in that: The feature enhancement includes adding noise, changing brightness and changing chrominance.

6. The communication maintenance tool identification method based on YOLO V5 as claimed in claim 1, characterized in that: The step S5 specifically comprises the following steps: S51: Improve the operation interface, add three buttons: test picture, test video and test camera, obtain image sources and perform tests through these three ways, and convert the images of the communication line maintenance tools to be tested into images with a resolution of 640*640; S52: Deploy the optimized YOLO V5 model and the converted IR model file to the Raspberry Pi, and display and record the detection results of the communication line maintenance tool in real time.

Images