A lightweight pointer instrument identification method for local deployment of a patrol robot

Abstract

The application provides a light-weight pointer instrument identification method for local deployment of an inspection robot, comprising: obtaining an instrument identification training set and a pointer and scale line segmentation training set; training an improved YOLOv5 model using the instrument identification training set; training an improved Deeplabv3+ model using the pointer and scale line segmentation training set; for a pointer instrument image collected by the inspection robot, locating a region where the instrument is located in the instrument image using the trained improved YOLOv5 model, clipping the instrument positioning frame according to four vertex coordinates of the obtained positioning frame, and obtaining an instrument region image; inputting the instrument region image obtained after clipping into the trained improved Deeplabv3+ model for segmentation and extraction of the pointer and the scale line; performing perspective transformation on a pointer and scale line binaryzation mask image based on an ellipse method, fitting a pointer straight line according to a centroid of the pointer, and finally performing reading according to an angle method; the method has small calculation consumption, high accuracy, and is easy to be locally deployed on the inspection robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of instrument identification, in particular to a light-weight pointer-type instrument identification method for local deployment of an inspection robot. BACKGROUND

[0002] Pointer-type instruments are simple in structure, easy to install and maintain, have the advantages of dustproof, waterproof, cold-proof, not affected by electromagnetic field, high reliability, etc., and are widely used in power systems, railway systems, factories and mines, and measurement departments. However, the manual inspection method is time-consuming, labor-intensive, low in inspection efficiency, large in inspection error, and poor in reliability. At present, with the development of robot technology, the method of identifying and reading instruments by an inspection robot has become the mainstream.

[0003] There are two modes for processing the pointer-type instrument images collected by the inspection robot: one is to transmit the images to a backend computer for processing through a wireless network, and the other is to perform edge computing on the inspection robot. In long-distance detection applications such as coal mines, power pipe corridors, and oil and gas pipelines, the cost of establishing a network is very high. Moreover, direct edge computing on the inspection robot can directly avoid problems caused by wireless networks, such as frame loss and data theft. Nowadays, more and more application sites require inspection robots with local recognition capabilities. However, most neural networks rely on complex models and calculations, and the computing resources available on the IPC (industrial personal computer) of the inspection robot for instrument identification are limited. Therefore, it is crucial to develop a light-weight, accurate pointer-type instrument identification method.

[0004] Current pointer-type instrument identification methods based on deep learning have large model parameters and complex structures, making it difficult to deploy and run on inspection robots. For example, in patent CN 113837178 A, the segmentation of the pointer and the scale uses a semantic segmentation Yolact model with ResNet101+FPN as the backbone, which has a complex model structure, large parameters, and high computational consumption, making it difficult to run locally on an inspection robot. SUMMARY

[0005] The main purpose of the present application is to overcome the above-mentioned defects in the prior art, and to provide a light-weight pointer-type instrument identification method for local deployment of an inspection robot, which has low computational consumption, high accuracy, and is easy to deploy locally on an inspection robot.

[0006] The present application adopts the following technical solutions:

[0007] A light-weight pointer-type instrument identification method for local deployment of an inspection robot, comprising:

[0008] An instrument recognition dataset is acquired and labeled, and a pointer and scale line segmentation dataset is acquired and labeled, to obtain an instrument recognition training set and a pointer and scale line segmentation training set;

[0009] An improved YOLOv5 model is trained using the instrument recognition training set, to obtain a trained improved YOLOv5 model;

[0010] An improved Deeplabv3+ model is trained using the pointer and scale line segmentation training set, to obtain a trained improved Deeplabv3+ model;

[0011] For a pointer instrument image collected by an inspection robot, the trained improved YOLOv5 model is used to locate a region where an instrument is located in the instrument image, the four vertex coordinates of the obtained positioning frame are used to crop the instrument positioning frame, and an instrument region image is obtained;

[0012] The instrument region image obtained after cropping is input into the trained improved Deeplabv3+ model for segmentation and extraction of a pointer and scale lines, to obtain a pointer and scale line binary mask;

[0013] Perspective transformation is performed on the pointer and scale line binary mask image based on an ellipse method, a pointer straight line is fitted according to the centroid of the pointer, and reading is performed according to an angle method.

[0014] Specifically, the improved YOLOv5 model specifically comprises:

[0015] An adaptive momentum random optimization strategy Adam is used to update parameters of the improved YOLOv5 network model, a dropout method is used for regularization, and an improved loss function is used as a loss function of the network, and an expression is as follows:

[0016]

[0017] In the formula, the first term EIoUloss represents a boundary box regression loss, the second and third terms represent confidence prediction losses, S represents the height and width of a feature map, B represents the number of basic frames corresponding to each pixel point in the feature map, and C represents a classification category. Specifically, the improved Deeplabv3+ model specifically comprises:

[0018] The improved Deeplabv3+ model comprises an encoding module and a decoding module;

[0019] The encoding module comprises a deep convolution block, a JPU module and a separate convolution of different rates, the deep convolution block acquires a feature map of an input image, the JPU module and the separate convolution of different rates acquire sufficient global information;

[0020] The decoding module comprises a 1*1 convolution block, a 3*3 convolution block and a plurality of up-sampling blocks.

[0021] Specifically, the JPU module is specifically:

[0022] First, the input feature map is processed by a convolution block to realize feature fusion.

[0023] The feature map is up-sampled and connected to obtain a first result map, and then the features of the first result map are extracted by four different rates of separable convolution.

[0024] The features of the extracted first result map are converted into a prediction result by a convolution block.

[0025] Another aspect of the embodiment of the application provides a lightweight pointer instrument identification system for local deployment of an inspection robot, comprising:

[0026] The training set acquisition unit acquires and labels an instrument identification dataset, and acquires and labels a pointer and scale line segmentation dataset to obtain an instrument identification training set and a pointer and scale line segmentation training set.

[0027] The improved YOLOv5 model training unit trains an improved YOLOv5 model using the instrument identification training set to obtain a trained improved YOLOv5 model.

[0028] The improved Deeplabv3+ model training unit trains an improved Deeplabv3+ model using the pointer and scale line segmentation training set to obtain a trained improved Deeplabv3+ model.

[0029] The instrument region image acquisition unit uses the trained improved YOLOv5 model to locate the region of the instrument in the pointer instrument image collected by the inspection robot, clips the instrument positioning frame according to the four vertex coordinates of the obtained positioning frame to obtain an instrument region image.

[0030] The pointer and scale line segmentation unit inputs the instrument region image obtained after clipping into the trained improved Deeplabv3+ model to segment and extract the pointer and scale line, and obtains a pointer and scale line binary mask.

[0031] The reading unit performs perspective transformation on the pointer and scale line binary mask image based on the ellipse method, fits a pointer straight line according to the centroid of the pointer, and finally performs reading according to the angle method.

[0032] Specifically, in the improved YOLOv5 model training unit, the improved YOLOv5 model is specifically:

[0033] The improved YOLOv5 network model is updated by using a random optimization strategy Adam with adaptive momentum, and a dropout method is used for regularization; an improved loss function is used as the loss function of the network, and the expression is:

[0034]

[0035] In the formula, the first term EIoUloss represents the boundary box regression loss, and the second and third terms represent the confidence prediction loss, wherein S represents the height and width of the feature map, B represents the basic frame number corresponding to each pixel point in the feature map, and C represents the classification category.

[0036] Specifically, in the improved Deeplabv3+ model training unit, the improved Deeplabv3+ model specifically comprises:

[0037] The improved Deeplabv3+ model comprises an encoding module and a decoding module.

[0038] The encoding module comprises a deep convolution block, a JPU module and a separate convolution of different rates, the deep convolution block obtains a feature map of an input image, the JPU module and the separate convolution of different rates obtain sufficient global information.

[0039] The decoding module comprises a 1*1 convolution block, a 3*3 convolution block and a plurality of up-sampling blocks.

[0040] Specifically, the JPU module specifically comprises:

[0041] First, the input feature map is processed by a convolution block to realize feature fusion.

[0042] The feature map is up-sampled and connected to obtain a first result map, and then four different rates of separable convolution are used to extract the features of the first result map.

[0043] The extracted features of the first result map are converted into prediction results by a convolution block.

[0044] In another aspect, the embodiment of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned light-weight pointer instrument identification method for local deployment of a patrol robot when executing the computer program.

[0045] In another aspect, the embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned light-weight pointer instrument identification method for local deployment of a patrol robot.

[0046] As can be known from the above description of the present application, compared with the prior art, the present application has the following beneficial effects:

[0047] (1) The present application provides a lightweight pointer instrument identification method for local deployment of an inspection robot, comprising: obtaining an instrument identification training set and a pointer and scale line segmentation training set; training an improved YOLOv5 model using the instrument identification training set; training an improved Deeplabv3+ model using the pointer and scale line segmentation training set; for a pointer instrument image collected by the inspection robot, locating the area where the instrument is located in the instrument image using the trained improved YOLOv5 model, cropping the instrument positioning frame according to the four vertex coordinates of the obtained positioning frame to obtain an instrument area image; inputting the instrument area image obtained after cropping into the trained improved Deeplabv3+ model to segment and extract the pointer and scale line, obtaining a pointer and scale line binary mask; performing perspective transformation on the pointer and scale line binary mask image based on the ellipse method, then fitting a pointer straight line according to the centroid of the pointer, and finally performing reading according to the angle method; the present application method has small calculation consumption, high accuracy, is easy to deploy locally on an inspection robot, has strong robustness and adaptability, and can be used in various environments.

[0048] (2) The improved loss function in the improved YOLOv5 model of the present application, compared with the original loss function, uses EIoUloss to represent the boundary box regression loss, which considers the overlapping area, center point distance, and real difference of length and width, so that the calculation is more accurate; the improved loss removes the class prediction loss, because there is only one category in the scene application of the present application, so that the loss calculation is simpler.

[0049] (3) The present application improves the Deeplabv3+ model, uses ResNet50 as the backbone of the model, uses the JPU module instead of the dilated convolution in the DCNN, and uses a separation convolution with different rates instead of the ASPP module in the original Deeplabv3+; so that the improved Deeplabv3+ can significantly reduce the calculation consumption and improve the segmentation accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A lightweight pointer instrument identification method for local deployment of an inspection robot is provided for the embodiments of the present application;

[0051] Figure 2 An improved Deeplabv3+ model structure diagram is provided for the embodiments of the present application;

[0052] Figure 3 A JPU module structure diagram is provided for the embodiments of the present application;

[0053] Figure 4 An explanatory diagram of perspective transformation is provided for the embodiment of the present application.

[0054] Figure 5 An explanatory diagram of pointer straight line extraction is provided for the embodiment of the present application.

[0055] Figure 6 An explanatory diagram of angle method calculation reading is provided for the embodiment of the present application.

[0056] Figure 7 An architecture diagram of a light-weight pointer instrument identification system locally deployed by a patrol robot is provided for the embodiment of the present application.

[0057] Figure 8 An electronic device schematic diagram is provided for the embodiment of the present application.

[0058] Figure 9 An embodiment schematic diagram of a computer readable storage medium is provided for the embodiment of the present application.

[0059] The present application is further described below in combination with the drawings and specific embodiments. DETAILED DESCRIPTION

[0060] The present application is further described below in combination with the drawings and specific embodiments.

[0061] The present application provides a light-weight pointer instrument identification method locally deployed by a patrol robot, which has small calculation consumption, high accuracy, is easy to be locally deployed by a patrol robot, has strong robustness and adaptability, and can be used in various environments.

[0062] As Figure 1 , a light-weight pointer instrument identification method locally deployed by a patrol robot is provided for the embodiment of the present application, which comprises:

[0063] S101: Obtain and label an instrument identification dataset, and obtain and label a pointer and scale line segmentation dataset, to obtain an instrument identification training set and a pointer and scale line segmentation training set;

[0064] A total of 2000 substation instrument images are collected under different illuminations and environmental backgrounds using a picture collection device. The substation instrument images include circular ammeter, circular voltmeter and circular lightning arrester. The labelimg and labelme labeling software are respectively used to complete the labeling of the instrument identification dataset and the labeling of the pointer and scale line segmentation dataset.

[0065] S102: Train the improved YOLOv5 model using the instrument identification training set, to obtain a trained improved YOLOv5 model;

[0066] The improved YOLOv5 model, in the embodiment of the application, adopts a random optimization strategy Adam (Adaptive momentum) of adaptive momentum to update parameters of the network, adopts a dropout method for regularization to prevent overfitting of the network training. An improved loss function is adopted as the loss function of the network, and the expression is:

[0067]

[0068] In the formula, the first term EIoUloss represents the bounding box regression loss, and the second and third terms represent the confidence prediction loss, wherein S represents the height and width of the feature map, B represents the number of basic boxes corresponding to each pixel point in the feature map, and C represents the classification category.

[0069] In the formula, the first term EIoUloss represents the bounding box regression loss, and the second and third terms represent the confidence prediction loss. Compared with the original loss function, the improved loss function adopts EIoUloss to represent the bounding box regression loss, which considers the overlapping area, center point distance and real difference of length and width, so that the loss calculation is more accurate; the improved loss removes the category prediction loss, because there is only one category in the application, so that the loss calculation is more simple.

[0070] Subsequently, channel pruning is performed on the improved YOLOv5s, and then the pruned model is optimized.

[0071] The model construction is programmed by Pytorch 1.9 and Python 3.6. The training parameters of the improved YOLOv5 are as follows: epoch = 200; batch-size = 4; learning rate lr = 0.001; and the optimizer is Adam.

[0072] S103: The improved Deeplabv3+ model is trained by using the pointer and the scale line to divide the training set, and a trained improved Deeplabv3+ model is obtained.

[0073] The improved Deeplabv3+ model structure is as shown in Figure 2 The JPU module is as shown in Figure 3

[0074] Given an input image, in the Encoder module, a deep convolutional block (DCNN) is used to obtain a feature map of the input image, and then a JPU module (Joint Pyramid Upsampling) and a separate convolution at different rates are used to obtain sufficient global information, and then a convolution block and an upsample are used to obtain a feature Figure 1 ​(Feature map1);In the Decoder module, the low-level features obtained in the CNN are reduced in dimension by using a convolution block to obtain feature Figure 2 (Featuremap2). The features Figure 1 and feature Figure 2 are concatenated and up-sampled to obtain the final output.

[0075] The JPU module can be divided into three parts: first, the input feature map is processed by a convolution block to achieve better feature fusion; then, the feature map is up-sampled and connected to obtain y c , and the features of y c are extracted by four different rate separable convolutions; finally, a convolution block is used to convert the features into a prediction result. JPU can replace the dilated convolution to obtain multi-scale context information on multiple feature maps, and compared with the dilated convolution, its computational consumption is greatly reduced.

[0076] Compared with the original Deeplabv3+, the improved Deeplabv3+ in the application uses ResNet50 as the backbone of the model instead of Xception; the application uses the JPU module to replace the dilated convolution in the DCNN; and the application uses different rate separable convolutions to replace the ASPP module in the original Deeplabv3+. The improved Deeplabv3+ can significantly reduce the computational consumption and improve the segmentation accuracy.

[0077] The model is constructed by programming with Pytorch 1.9 and Python 3.6, and the training parameters of the improved Deeplabv3+ are: epoch=500; batch-size=4; learning rate lr=0.001; out-stride=16.

[0078] S104: For the pointer instrument image collected by the inspection robot, an improved YOLOv5 model trained is used to locate the area where the instrument in the instrument image is located, and according to the four vertex coordinates of the obtained positioning box, the instrument positioning box is cropped to obtain an instrument region image;

[0079] S105: The instrument region image obtained after cropping is input into an improved Deeplabv3+ model trained to segment and extract the pointer and the scale line, and a pointer and scale line binary mask is obtained;

[0080] S106: Based on the ellipse method, the pointer and scale line binary mask image is perspective transformed, then a pointer straight line is fitted according to the centroid of the pointer, and finally the reading is obtained according to the angle method.

[0081] Perspective transformation: First, the minimum circumscribed circle and the minimum circumscribed ellipse of the scale line binary mask are fitted by using opencv, as shown in Figure 4 C1-C4 represents the four foci of the minimum circumscribed circle and the horizontal line, the vertical line; E1-E4 represents the four foci of the minimum circumscribed ellipse and the major axis, the minor axis; O represents the center; α represents the deflection angle of the ellipse.

[0082] The equations of the circle and the ellipse are respectively [(O x ,O y ), r] and [(E x ,E y ), 2a, 2b, α]. Therefore, the coordinates of C1-C4 and E1-E4 can be calculated by the following formulas:

[0083] [C1, C2, C3, C4] = [(O x ± r, O y ± r)]

[0084] [E1, E2, E3, E4] = [E x ± k × a(b), E y ± k × a(b)]

[0085] Where k = sin(α) or cos(α). According to the coordinates of C1-C4 and E1-E4, the perspective transformation matrix A can be calculated according to the following formula:

[0086]

[0087] Pointer straight line extraction:

[0088] First, the minimum circumscribed triangle of the pointer binary mask is fitted by using opencv, as shown in Figure 5

[0089] The three vertices of the triangle are respectively T1(x1, y1), T2(x2, y2) and T3(x3, y3). Therefore, the centroid T(x, y) of the triangle can be calculated by the following formula:

[0090]

[0091] The distances from the three vertices to the centroid T are calculated respectively, and the vertex T1 farthest from the point T is found. Finally, the pointer straight line is determined by connecting T1 and T.

[0092] Angle method to calculate the reading

[0093] The principle of angle method is as follows Figure 6 ​The coordinate system is established with the pointer center as the coordinate origin. Lf represents the extracted pointer straight line; the scale mask points with the minimum Y-axis coordinates in the third and fourth quadrants are recorded as P1 and P2, respectively; L1 and L2 represent straight lines from P1 and P2 to O point, respectively; θ represents the angle between L1 and Lf; and β represents the angle between L1 and L2.

[0094] According to prior information, the range of the instrument is r, and therefore, the instrument reading can be calculated by the following formula:

[0095]

[0096] As Figure 7 , the embodiment of the present application provides a light-weight pointer instrument recognition system for local deployment of an inspection robot, which comprises:

[0097] The training set acquisition unit 701 acquires and labels an instrument recognition dataset and acquires and labels a pointer and scale line segmentation dataset to obtain an instrument recognition training set and a pointer and scale line segmentation training set.

[0098] A total of 2000 substation instrument images are collected under different illuminations and environmental backgrounds using a picture collection device. The substation instrument images include circular ammeters, circular voltmeters and circular lightning arresters. The labelimg and labelme labeling software are used to complete the labeling of the instrument recognition dataset and the labeling of the pointer and scale line segmentation dataset.

[0099] The improved YOLOv5 model training unit 702 trains the improved YOLOv5 model using the instrument recognition training set to obtain a trained improved YOLOv5 model.

[0100] In the embodiment of the present application, the random optimization strategy Adam (Adaptive momentum) is used to update the parameters of the network, the dropout method is used for regularization to prevent network overfitting during training, and the improved loss function is used as the loss function of the network, and the expression is as follows:

[0101]

[0102] In the formula, the first term EIoUloss represents the boundary box regression loss, and the second and third terms represent the confidence prediction loss, wherein S represents the height and width of the feature map, B represents the number of basic boxes corresponding to each pixel point in the feature map, and C represents the classification category.

[0103] In the formula, the first term EIoUloss represents the bounding box regression loss, and the second and third terms represent the confidence prediction loss. Compared with the original loss function, the improved loss function uses EIoUloss to represent the bounding box regression loss, which considers the overlap area, center point distance, and real difference of length and width, so that the loss calculation is more accurate. The improved loss removes the class prediction loss because there is only one category in the present application, so that the loss calculation is simpler.

[0104] Subsequently, channel pruning is performed on the improved YOLOv5s, and then the pruned model is optimized.

[0105] The model is constructed by programming with Pytorch 1.9 and Python 3.6. The training parameters of the improved YOLOv5 are: epoch=200; batch-size=4; learning rate lr=0.001; and the optimizer is Adam.

[0106] The improved Deeplabv3+ model training unit 703: the improved Deeplabv3+ model is trained by using a pointer and a scale line to divide the training set, and a trained improved Deeplabv3+ model is obtained.

[0107] The improved Deeplabv3+ model structure is as shown in Figure 2 , wherein the JPU module is as shown in Figure 3 .

[0108] Given an input image, in the Encoder module, a deep convolutional block (DCNN) is used to obtain the feature map of the input image, and then a JPU module (Joint Pyramid Upsampling) and a separate convolution with different rates are used to obtain sufficient global information, and then a convolution block and up sampling (Upsample) are used to obtain a feature Figure 1 (Feature map1); in the Decoder module, a convolution block is used to reduce the dimension of the low-level feature obtained in the CNN to obtain a feature Figure 2 (Feature map2). The feature Figure 1 and the feature Figure 2 are concatenated and up sampled to obtain the final output.

[0109] The JPU module can be divided into three parts: first, the input feature map is processed by a convolution block to achieve better feature fusion; then the feature map is up sampled and connected to obtain y c , and then four different rates of separable convolution are used to extract y cThe improved Deeplabv3+ in the application uses ResNet50 as the backbone of the model instead of Xception, uses the JPU module to replace the dilated convolution in the DCNN, and uses a separation convolution with different rates to replace the ASPP module in the original Deeplabv3+. The improved Deeplabv3+ can significantly reduce the computational consumption and improve the segmentation accuracy.

[0110] Compared with the original Deeplabv3+, the improved Deeplabv3+ in the application uses ResNet50 as the backbone of the model instead of Xception, uses the JPU module to replace the dilated convolution in the DCNN, and uses a separation convolution with different rates to replace the ASPP module in the original Deeplabv3+. The improved Deeplabv3+ can significantly reduce the computational consumption and improve the segmentation accuracy.

[0111] The model is constructed by programming with Pytorch 1.9 and Python 3.6, and the training parameters of the improved Deeplabv3+ are as follows: epoch = 500, batch-size = 4, learning rate lr = 0.001, and out-stride = 16.

[0112] The instrument area image acquisition unit 704: for the pointer instrument image collected by the inspection robot, the improved YOLOv5 model is used to locate the area where the instrument is located in the instrument image, the four vertex coordinates of the obtained positioning box are used to crop the instrument positioning box, and the instrument area image is obtained;

[0113] The pointer and scale line segmentation unit 705: the instrument area image obtained after cropping is input into the improved Deeplabv3+ model to segment and extract the pointer and scale line, and the pointer and scale line binary mask is obtained;

[0114] The reading unit 706: the pointer and scale line binary mask image is subjected to perspective transformation based on the ellipse method, a pointer straight line is fitted according to the centroid of the pointer, and finally the reading is performed according to the angle method.

[0115] Figure 8 As shown in the figure, the embodiment of the application provides an electronic device 800, which includes a memory 810, a processor 820, and a computer program 811 stored in the memory 820 and capable of running on the processor 820, and the processor 820 implements the method for light-weight pointer instrument recognition of the local deployment of the inspection robot provided by the embodiment of the application when executing the computer program 811.

[0116] Since the electronic device introduced in the embodiment is the device used in the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific implementation of the electronic device of the embodiment and its various forms, so the electronic device how to implement the method in the embodiment of the present application is not introduced in detail, as long as the person skilled in the art implements the method in the embodiment of the present application, the device used belongs to the scope of the present application.

[0117] Please refer to Figure 9 , Figure 9 An embodiment of a computer readable storage medium provided in the embodiment of the present application is shown.

[0118] As Figure 9 shown, the embodiment provides a computer readable storage medium 900, which stores a computer program 911, and the computer program 911 is executed by a processor to implement a lightweight pointer type instrument identification method for local deployment of a patrol robot provided in the embodiment of the present application.

[0119] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0120] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0121] It is to be noted that, in the present text, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element. The above specification is merely a specific implementation of the present application, enabling one of ordinary skill in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0122] The above is only a specific embodiment of the present application, but the design concept of the present application is not limited thereto, and any non-essential modification of the present application using the concept should be regarded as an infringement of the scope of protection of the present application.

Claims

1. A method for light-weight pointer instrument identification for local deployment of inspection robots, characterized in that, The application relates to an improved YOLOv5 model and an improved Deeplabv3+ model for meter reading. The application comprises the following steps: acquiring and labeling a meter recognition dataset and acquiring and labeling a pointer and scale line segmentation dataset to obtain a meter recognition training set and a pointer and scale line segmentation training set; training the improved YOLOv5 model by using the meter recognition training set to obtain a trained improved YOLOv5 model; training the improved Deeplabv3+ model by using the pointer and scale line segmentation training set to obtain a trained improved Deeplabv3+ model; for a pointer meter image collected by an inspection robot, the trained improved YOLOv5 model is used to locate a region where a meter is located in the meter image, a meter positioning frame is cropped according to four vertex coordinates of the obtained positioning frame to obtain a meter region image; the meter region image obtained after the cropping is input into the trained improved Deeplabv3+ model to segment and extract a pointer and a scale line, and a pointer and scale line binary mask is obtained; perspective transformation is performed on the pointer and scale line binary mask image based on an ellipse method, a pointer straight line is fitted according to a centroid of the pointer, and reading is performed according to an angle method; the improved YOLOv5 model is specifically as follows: an adaptive momentum random optimization strategy Adam is used to update parameters of the improved YOLOv5 network model, a dropout method is used for regularization, and an improved loss function is used as a loss function of the network, and an expression is as follows: in the formula, the first term EIoUloss represents a boundary box regression loss, the second and third terms represent confidence prediction losses, S represents the height and width of a feature map, B represents a basic frame number corresponding to each pixel point in the feature map, and C represents a classification category; 2. The light-weight pointer instrument identification method for local deployment of a patrol robot according to claim 1, characterized in that, the improved Deeplabv3+ model uses ResNet50 as a backbone of the model, uses a JPU module to replace a dilated convolution in a DCNN, and uses a separation convolution with different rates to replace an ASPP module in the original Deeplabv3+. the improved Deeplabv3+ model is specifically as follows: the improved Deeplabv3+ model comprises an encoding module and a decoding module; the encoding module comprises a deep convolution block, a JPU module and a separation convolution with different rates, the deep convolution block acquires a feature map of an input image, and the JPU module and the separation convolution with different rates acquire sufficient global information; 3. The method of claim 2, wherein the method is a lightweight pointer instrument identification method for local deployment of a patrol robot. the decoding module comprises a 1*1 convolution block, a 3*3 convolution block and a plurality of up-sampling blocks. the JPU module is specifically as follows: first, a convolution block is used to process an input feature map to realize feature fusion; the feature map is up-sampled and connected to obtain a first result image, and four separation convolutions with different rates are used to extract features of the first result image; 4. A lightweight pointer instrument identification system for local deployment of a patrol robot, characterized in that, a convolution block is used to convert the extracted features of the first result image into a prediction result. The application relates to an improved YOLOv5 model and an improved Deeplabv3+ model for meter reading. The application comprises the following steps: a training set acquisition unit is used to acquire and label a meter recognition dataset and acquire and label a pointer and scale line segmentation dataset to obtain a meter recognition training set and a pointer and scale line segmentation training set; The improved YOLOv5 model training unit trains the improved YOLOv5 model by using the instrument recognition training set, and obtains the trained improved YOLOv5 model; The improved Deeplabv3+ model training unit trains the improved Deeplabv3+ model by using the pointer and scale line segmentation training set, and obtains the trained improved Deeplabv3+ model; The instrument region image acquisition unit uses the trained improved YOLOv5 model to locate the region where the instrument is located in the pointer instrument image collected by the inspection robot, clips the instrument positioning frame according to the four vertex coordinates of the positioning frame, and obtains the instrument region image; The pointer and scale line segmentation unit inputs the instrument region image obtained after clipping into the trained improved Deeplabv3+ model to segment and extract the pointer and scale line, and obtains the pointer and scale line binary mask; The reading unit performs perspective transformation on the pointer and scale line binary mask image based on the ellipse method, fits a straight line of the pointer according to the centroid of the pointer, and finally performs reading according to the angle method; The improved YOLOv5 model is specifically: The improved YOLOv5 network model is updated by using the random optimization strategy Adam with adaptive momentum, and the dropout method is used for regularization; an improved loss function is used as the loss function of the network, and the expression is: In the formula, the first term EIoUloss represents the boundary box regression loss, and the second and third terms represent the confidence prediction loss, wherein S represents the height and width of the feature map, B represents the basic frame number corresponding to each pixel point in the feature map, and C represents the classification category; The improved Deeplabv3+ model uses ResNet50 as the backbone of the model, uses the JPU module to replace the dilated convolution in the DCNN, and uses the separated convolution with different rates to replace the ASPP module in the original Deeplabv3+.

5. The lightweight pointer-type instrument identification system for local deployment of a patrol robot according to claim 4, wherein, The improved Deeplabv3+ model in the improved Deeplabv3+ model training unit is specifically: The improved Deeplabv3+ model includes an encoding module and a decoding module; The encoding module includes a deep convolution block, a JPU module and a separated convolution with different rates, the deep convolution block obtains the feature map of the input image, and the JPU module and the separated convolution with different rates obtain sufficient global information; The decoding module includes a 1*1 convolution block, a 3*3 convolution block and a plurality of up-sampling blocks.

6. The lightweight pointer-type instrument identification system for local deployment of a patrol robot according to claim 5, wherein, The JPU module is specifically: First, the input feature map is processed by a convolution block to realize feature fusion; The feature map is up-sampled and connected to obtain a first result image, and then four different rates of separable convolution are used to extract the features of the first result image; A convolution block is used to convert the extracted features of the first result image into a prediction result.

7. An electronic device, comprising: It includes: A memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method steps of any one of claims 1 to 3 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 1-3.

Images