Instrument dial reading identification method and device, equipment and storage medium

Abstract

The invention provides an instrument dial reading identification method, device and equipment and a storage medium, and the method comprises the steps: firstly obtaining a to-be-identified image collected by an image collection device, and calling a pre-trained YOLOv5 model to carry out the identification of the to-be-identified image, so as to position a dial region image; next, calling a pre-trained DeepLabV3 + model to perform feature extraction on the dial region image so as to generate deep features and shallow features; and finally, fusing the deep layer features and the shallow layer features to generate fused features, performing semantic segmentation on the fused features, and identifying and outputting a reading result of the dial plate. The problems of insufficient identification precision and reliability caused by poor quality of shot images are solved.

Description

Technical Field

[0001] The present invention relates to the field of instrument identification, and in particular to an instrument dial reading identification method, device, equipment and storage medium. Background Art

[0002] Traditional instrument dial readings are done manually and the data is transmitted to the cloud and recorded. Traditional manual readings have low accuracy and high labor costs. With the development of science and technology, intelligent recognition technology is gradually adopted for instrument dial readings, and computer vision technology is used to realize intelligent recognition of instrument dial readings. However, most of the current reading recognition is based on traditional image processing technology. There are too many styles of instrument dials, and the pictures taken in the environment where many instrument dials are located are not clear. In many cases, high quality requirements are placed on image processing, resulting in low accuracy and poor reliability of instrument dial image processing.

[0003] In view of this, this application is filed. Summary of the invention

[0004] The invention discloses a method, device, equipment and storage medium for recognizing instrument dial readings, aiming to solve the problem of insufficient recognition accuracy and reliability caused by poor quality of captured images.

[0005] A first embodiment of the present invention provides a method for identifying instrument dial readings, comprising:

[0006] Acquire the image to be identified acquired by the image acquisition device, and call the pre-trained YOLOv5 model to identify the image to be identified, so as to locate the dial area image;

[0007] Call the pre-trained DeepLabV3+ model to extract features from the dial area image to generate deep features and shallow features, wherein the shallow features include: the specific shape of the pointer, the precise position of the scale line and the outline of the number, and the deep features include the shape and type of the dashboard;

[0008] The deep features and the shallow features are fused to generate fused features, and the fused features are semantically segmented and the reading results of the output dial are recognized, wherein the fusion process is: using a feature pyramid network to fuse the shallow features with the deep features in a top-down and lateral connection manner.

[0009] Preferably, the calling of the pre-trained YOLOv5 model to identify the image to be identified so as to locate the dial area image is specifically as follows:

[0010] The image to be identified is divided into n*n grids, and a confidence evaluation result is generated based on whether there is an instrument in each grid, where if there is an instrument, the confidence Pc=1, and if not, the confidence Pc=0;

[0011] An image annotation tool is called, and a prediction box is drawn on the image to be identified according to the confidence evaluation result, and the dial area image is located based on the position information of the prediction box.

[0012] Preferably, the calling of the pre-trained DeepLabV3+ model to extract features from the dial area image to generate deep features and shallow features is as follows:

[0013] Perform shallow feature extraction on the dial area image by using the resnet deep residual network algorithm to generate basic shallow features and feature images of the dial area image;

[0014] The feature image is extracted through three different hole convolutions and global features of the image to generate a feature layer of the image, and then the deep features are obtained through a 1*1 convolution layer.

[0015] Preferably, the semantic segmentation of the fused features and identification of the reading result of the output dial are specifically:

[0016] The fusion features are identified based on the DeepLabV3+ algorithm to identify the starting scale line and digital display, the end scale line and digital display,

[0017] According to the coordinates of the scale line, the pointer reading and the pointer axis, the mathematical equation of the scale circle, the angle between the scales, and the angle between the scale and the pointer are generated, and then the pointer reading is generated.

[0018] Preferably, before calling the pre-trained YOLOv5 model to identify the image to be identified so as to locate the dial area image, the YOLOv5 model is further trained, and the training process is specifically as follows:

[0019] Obtain a dashboard data set, identify a dashboard area in the dashboard data set, and mark the area-identified pictures and non-dashboard pictures as a first sample set, wherein the dashboard data set includes dashboards of different styles and environments and pictures similar to the dashboard.

[0020] Using the first sample set as model data, and extracting training set features by inputting different image data;

[0021] According to the training set features, an instrument dial recognition model is generated based on the YOLOv5 algorithm;

[0022] The pictures that have not been collected before are marked as the second sample set, and the second sample set is further collected. The second sample set is input into the trained instrument dial recognition model, and the model is further tuned. If the output result is incorrect, the weight parameters of the model are optimized.

[0023] Preferably, the first sample set is used as model data, and different image data are input to extract training set features, specifically:

[0024] Based on the algorithm matrix, each grid algorithm matrix is ​​set to 1*1*6, and all grid algorithm matrices of the picture are 19*19*6. The expression for obtaining the algorithm matrix is:

[0025] y=[P c b x b y b h b w ] -1

[0026] Where: P c is the confidence level, b x is the x coordinate of the center point of the prediction box in the whole picture, b y is the y coordinate of the center point of the prediction box in the whole picture, b h is the predicted box height, b w is the predicted box length;

[0027] All grid algorithms are mapped through CNN, and the spatial size of the input image is adjusted to a fixed size using the Resize and Normalize functions, and the homogeneous variance statistically obtained from the data set is normalized.

[0028] Preferably, before calling the pre-trained DeepLabV3+ model to extract features from the dial area image, the DeepLabV3+ model is further trained, and the training process is specifically as follows:

[0029] Obtaining the regional image after the instrument dial area is extracted, marking the starting scale line and digital display, the end scale line and digital display of the instrument, and using the regionally marked image as the third sample set;

[0030] Extracting features of the third sample set, and generating a dashboard reading model based on the DeepLabV3+ model,

[0031] The instrument dial area pictures different from those collected previously are marked as the fourth sample set, and the fourth sample set is input into the instrument dial reading model. If the input result is incorrect, the weight parameters of the instrument dial intelligent recognition model are optimized.

[0032] A second embodiment of the present invention provides a device for identifying instrument dial readings, comprising:

[0033] A dial area image positioning unit, used to obtain the image to be identified acquired by the image acquisition device, and call the pre-trained YOLOv5 model to identify the image to be identified, so as to locate the dial area image;

[0034] A feature extraction unit, used to call the pre-trained DeepLabV3+ model to perform feature extraction on the dial area image to generate deep features and shallow features, wherein the shallow features include: the specific shape of the pointer, the precise position of the scale line and the outline of the number, and the deep features include the shape and type of the instrument panel;

[0035] A fusion unit is used to fuse the deep features and the shallow features to generate fused features, perform semantic segmentation on the fused features and identify the reading results of the output dial, wherein the fusion process is: using a feature pyramid network to fuse the shallow features with the deep features in a top-down and lateral connection manner.

[0036] A third embodiment of the present invention provides an instrument dial reading recognition device, including a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement an instrument dial reading recognition method as described in any one of the above.

[0037] A fourth embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program can be executed by a processor of a device where the computer-readable storage medium is located to implement a method for identifying an instrument dial reading as described in any one of the above items.

[0038] Based on the recognition method, device, equipment and storage medium of the instrument dial reading provided by the present invention, the image to be recognized collected by the image acquisition device is first obtained, and the pre-trained YOLOv5 model is called to recognize the image to be recognized to locate the dial area image; then, the pre-trained DeepLabV3+ model is called to extract features of the dial area image to generate deep features and shallow features; finally, the deep features and the shallow features are fused to generate fused features, and the fused features are semantically segmented and the dial reading result is recognized and output. The problem of insufficient recognition accuracy and reliability due to poor quality of the captured images is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a method for identifying instrument dial readings provided by the first embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of the process of recognizing the instrument dial reading provided by the present invention;

[0041] Figure 3 It is a schematic diagram of the training set feature extraction process provided by the present invention;

[0042] Figure 4 It is a module schematic diagram of a device for identifying instrument dial readings provided in a second embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0045] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0047] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0048] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0049] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0050] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] The invention discloses a method, device, equipment and storage medium for recognizing instrument dial readings, aiming to solve the problem of insufficient recognition accuracy and reliability caused by poor quality of captured images.

[0052] See also Figure 1 The first embodiment of the present invention provides a method for identifying instrument dial readings, which can be performed by an instrument dial reading identification device (hereinafter referred to as identification device), and in particular, is performed by one or more processors in the identification device to implement at least the following steps:

[0053] S101, obtaining an image to be identified acquired by an image acquisition device, calling a pre-trained YOLOv5 model to identify the image to be identified, so as to locate a dial area image;

[0054] In this embodiment, the recognition device may be a terminal with data processing capability such as a desktop computer, a laptop computer, a server, a workstation, etc., and the recognition device may be installed with a corresponding operating system and application software, and realize the functions required by this embodiment through the combination of the operating system and the application software; the recognition device can establish communication with the image acquisition device, can obtain the image acquired by the image acquisition device, and recognize the image to determine whether it is a dashboard image;

[0055] Specifically, in this embodiment, the image to be identified is divided into n*n grids, and a confidence evaluation result is generated based on whether there is an instrument in each grid, wherein if there is an instrument, the confidence Pc=1, and if not, the confidence Pc=0;

[0056] An image annotation tool is called, and a prediction box is drawn on the image to be identified according to the confidence evaluation result, and the dial area image is located based on the position information of the prediction box.

[0057] It should be noted that this embodiment can, but is not limited to, divide the image to be identified into 19*19 grids, and evaluate each grid to reduce the situation of misidentification and missed identification. The use of binary confidence evaluation (1 or 0) simplifies the calculation process, which can improve the processing speed while ensuring accuracy. At the same time, this grid evaluation method can cope with images under various complex backgrounds and different lighting conditions. Even if the position and size of the instrument in the image changes, it can be effectively located.

[0058] S102, calling a pre-trained DeepLabV3+ model to perform feature extraction on the dial area image to generate deep features and shallow features;

[0059] Specifically, in this embodiment: shallow feature extraction can be performed on the dial area image through the resnet deep residual network algorithm to generate basic shallow features and feature images of the dial area image;

[0060] The shallow features are features extracted from the network layer closer to the input layer, with higher resolution and closer to the original image. In instrument panel recognition, shallow features can capture the specific shape of the pointer, the precise position of the scale line, and the outline of the number. The resnet deep residual network algorithm can effectively solve the problems of gradient disappearance and gradient explosion in deep neural network training, thereby extracting rich feature information in deeper networks. At the same time, resnet can provide reliable shallow features in image recognition tasks;

[0061] Among them, the deep features are features extracted from the network layer far away from the input layer. These features are more abstract and global, contain higher-level semantic information, have less noise, but have lower resolution. In instrument panel recognition, deep features can help identify the layout and overall structure of the entire instrument panel, such as distinguishing different types of instrument panels (circular, square) or identifying the overall pattern on the instrument panel (such as speedometer, fuel gauge). The feature image is extracted through three different types of void convolution and image global features to generate the feature layer of the image, and then the deep features are obtained through a 1*1 convolution layer.

[0062] It should be noted that, in this embodiment, multiple feature maps of the same size can be extracted, specifically through 1*1Conv, 3*3Convrate6, 3*3Convrate12, 3*3Convrate18, and image pooling, so as to expand the receptive field of the convolution kernel without increasing the number of parameters and computational complexity, thereby capturing a wider range of contextual information; then, after splicing the feature maps of multiple feature maps of the same size, a 1*1 convolution layer is used to obtain deep features, which can effectively compress and integrate the information of the feature layer, map the multi-channel features to a low-dimensional space, reduce the amount of calculation and improve the representation ability of the features, and further, the global feature extraction can capture the global information of the image and make up for the details that may be missed by local convolution.

[0063] S103, fusing the deep layer features and the shallow layer features to generate fused features, performing semantic segmentation on the fused features and identifying and outputting the reading results of the dial.

[0064] Specifically, in this embodiment, the fusion feature is identified based on the DeepLabV3+ algorithm to identify the starting scale line and digital display, the end scale line and digital display,

[0065] According to the coordinates of the scale line, the pointer reading and the pointer axis, the mathematical equation of the scale circle, the angle between the scales, and the angle between the scale and the pointer are generated, and then the pointer reading is generated.

[0066] In this embodiment, the specific identification principle is:

[0067] Assume that the center point of the instrument dial is (0, 0), the coordinates of the instrument start scale line are (x1, y1), the digital display is M, the coordinates of the instrument end scale line are (x2, y2), the digital display is N, the current coordinates of the instrument pointer are (α, β), the internal scale of the instrument dial is an ellipse or a circle, and the rectangular coordinate equation of the ellipse is known from the three points:

[0068] Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0

[0069] The rectangular coordinate equation of an ellipse can be converted to a polar coordinate equation:

[0070] x=ρcosθ;y=ρsinθ

[0071] The angle between the position of the instrument pointer and the starting scale line is:

[0072]

[0073] The angle between the instrument's start scale line and end scale line is:

[0074]

[0075] The starting scale line is displayed as M, and the ending scale line is displayed as N. The angle between the three coordinates can be used to obtain the pointer position reading X:

[0076]

[0077] Please combine Figure 3 In a possible implementation of the present invention, before calling the pre-trained YOLOv5 model to identify the image to be identified to locate the dial area image, the YOLOv5 model is further trained. The training process is as follows:

[0078] We collected dashboard images of various styles and environments from multiple sources to ensure the diversity and wide coverage of the dataset. The dataset includes not only clear dashboard images, but also pictures similar to dashboards, which can increase the model's discernment ability.

[0079] Manually annotate the dial area in the collected instrument dial image to generate an area marked picture, and generate a first sample set including the marked instrument dial area picture and the non-instrument dial picture.

[0080] The images in the first sample set are used as model training data to input into the YOLOv5 model. Different image data can be preprocessed first, resized and normalized, and the image pixel values ​​can be scaled from the range of [0,255] to [0,1] or other ranges to help the model converge better. The normalization formula is as follows:

[0081]

[0082] In the formula, x is the original pixel value, mean and std are the mean and standard deviation of the data set respectively.

[0083] After preprocessing the image, the YOLOv5 model automatically extracts features from the preprocessed image through a convolutional neural network and finally generates training set features.

[0084] Each convolution layer scans the image through a filter (or kernel) to extract local features. The convolution operation can be expressed by the following formula:

[0085]

[0086] Where W is the weight matrix, X is the input image, and * represents the convolution operation.

[0087] Among them, the pooling operation usually uses maximum pooling or average pooling to reduce the data dimension while retaining the most important information. The formula for maximum pooling can be expressed as:

[0088]

[0089] Where stride is the step size of the pooling.

[0090] Based on the extracted training set features, the YOLOv5 algorithm is used to train the model and generate a preliminary instrument dial recognition model. During the training process, data enhancement techniques such as rotation, scaling, and flipping can be used to improve the generalization ability of the model.

[0091] After the initial training of the model is completed, new images are further collected from the image source that has not been used to generate a second sample set. This includes more diverse dial images to further improve the recognition accuracy and robustness of the model. The second sample set is input into the trained instrument dial recognition model for further training and tuning. If the model output results are not ideal, the weight parameters of the model are optimized and adjusted. Through continuous iterative training and optimization, the recognition accuracy of the model is gradually improved. The model performance is evaluated after each iteration until the expected recognition effect is achieved.

[0092] It should be noted that to optimize the weight parameters of the model, we first need to use the loss function to measure the difference between the model prediction value and the actual value. Here we use the cross entropy loss function:

[0093]

[0094] Where: y i is the true label, is the probability predicted by the model.

[0095] Then, the gradient descent optimization algorithm is used to minimize the loss function by iteratively updating the weights:

[0096] w new =w old -η·abla w L

[0097] Where: w new and w old are the weights before and after the update, η is the learning rate, and bla w L is the gradient of the loss function with respect to the weights.

[0098] Next, we use the back-propagation method to calculate the gradient, passing the error from the output layer to the input layer through the chain rule, and calculate the gradient of each layer weight. For the weight of the lth layer, its gradient can be expressed as:

[0099]

[0100] In the formula, is the weighted input of the jth neuron in the lth layer.

[0101] Finally, after getting the gradients of all weights, we update the weights using the gradient descent rule:

[0102]

[0103] Repeat the above process, update the weights with new data in each iteration, and evaluate the model performance on the validation set. If the performance does not meet expectations, you can adjust the hyperparameters or continue training for more rounds.

[0104] In a possible implementation of the present invention, the first sample set is used as model data, and different image data are input to extract training set features, as follows:

[0105] The image to be recognized is divided into n*n small grids, and each grid is represented by an algorithm matrix.

[0106] The algorithm matrix size of each grid is 1*1*6, and the total generated matrix is ​​19*19*6;

[0107] The expression of the algorithm matrix is ​​as follows:

[0108] y=[P c b x b y b h b w ] -1

[0109] Where: P c is the confidence level, b x is the x coordinate of the center point of the prediction box in the whole picture, b y is the y coordinate of the center point of the prediction box in the whole picture, b h is the predicted box height, b w is the predicted box length;

[0110] Furthermore, the algorithm matrix of all grids can be mapped through a convolutional neural network (CNN). Through the cascading operation of CNN, important features in the image are extracted. The Resize function is used to adjust the spatial size of the input image to a fixed size to meet the input requirements of the model. The Normalize function is used to normalize the image and adjust the pixel values ​​to a uniform range. Among them, the normalization process uses the homogeneous variance statistically obtained from the data set to make the numerical distribution of the image more uniform, thereby improving the training effect of the model.

[0111] The preprocessed image data is input into the model, and feature extraction is performed through CNN to generate training set features. The extracted features contain the spatial information and contextual information of the image, providing comprehensive data support for model training.

[0112] In a possible implementation of the present invention, before calling the pre-trained DeepLabV3+ model to extract features from the dial area image, the DeepLabV3+ model is further trained. The training process is as follows:

[0113] Extract images of the dial area from images of instrument dials of various types and environments.

[0114] Mark the instrument start scale line and digital display, end scale line and digital display in the extracted dial area image to ensure that the model can recognize and distinguish these key areas. The above marked images are used as the third sample set for initial model training.

[0115] Use a deep learning algorithm to extract features from the third sample set. By extracting these features, the model can learn the key areas and identification information in the image. In the DeepLabV3+ model, feature extraction is mainly achieved through the dilated convolution and spatial pyramid pooling modules:

[0116] The shallow features first use dilated convolution to expand the receptive field. Dilated convolution expands the effective receptive field of the convolution kernel by inserting blank space (holes) between standard convolution kernels, while maintaining high-resolution feature maps. For a given dilation rate d, the convolution operation can be expressed as:

[0117]

[0118] Where x[i] is the value of the i-th position of the input feature map, w[k] is the k-th weight of the convolution kernel, and K is the size of the convolution kernel.

[0119] Secondly, the Spatial Pyramid Pooling (ASPP) module uses parallel dilated convolutions with different dilation rates to capture multi-scale contextual information. The output of ASPP is the combined result of these parallel convolution operations:

[0120] y[i]=Concat(y1[i],y2[i],...,y n [i])

[0121] Where y1,y2,...,y n is the output of dilated convolution with different dilation rates.

[0122] After ASPP, there is usually a 1×11×1 convolutional layer to reduce the number of channels and fuse features:

[0123]

[0124] Where y[i,j] is the value of the jth channel at the ith position of the ASPP output, C is the number of channels, and w[j] is the weight of the 1×11×1 convolution kernel.

[0125] Through these steps, the DeepLabV3+ model is able to extract rich shallow features while maintaining high resolution, which is essential for capturing fine details in images (such as the pointers and ticks of the dashboard). These features are then used in deeper network layers for further feature abstraction and semantic understanding.

[0126] The deep features also first use dilated convolution to expand the receptive field to capture larger context information while maintaining high-resolution feature maps. Then the ASPP module uses parallel dilated convolution layers with different expansion rates to capture multi-scale features, and the global average pooling layer is used to obtain the overall image features. The calculation formula for global average pooling in the ASPP module is:

[0127]

[0128] Where, H and W are the height and width of the feature map, respectively.

[0129] Then, the feature map processed by dilated convolution and ASPP is passed through a 1x1 convolution layer for dimensionality reduction and feature fusion to generate the final deep features. Finally, by combining dilated convolution, ASPP and 1x1 convolution layers, the DeepLabV3+ model can effectively extract highly abstract and global deep features from network layers far away from the input layer. These deep features are crucial for identifying the overall layout and structure of the dashboard, which helps to distinguish different types of dashboards or identify the overall pattern on the dashboard.

[0130] The feature pyramid network (FPN) is used to fuse the shallow features and deep features through top-down and lateral connections:

[0131] 1. Top-down path: Upsample the high-level feature map to align the spatial size of the low-level feature map. Assume that the current layer is the i-th layer and its feature map is C i , then the upsampled feature map is recorded as P i . The upsampling operation can be achieved through interpolation method;

[0132] 2. Horizontal connection: At each level, the feature map sampled from top to bottom is horizontally connected with the feature map of the corresponding level from bottom to top. This connection method can effectively combine the strong semantic information of the high-level and the detailed information of the low-level to generate a fused feature map. Assume that the feature map of the current level is C i-1 , then the fused feature map P i It can be expressed as:

[0133] P i =C i-1 +Upsample(P i+1 )

[0134] Where Upsample represents the upsampling operation.

[0135] Based on the DeepLabV3+ model, a model that can accurately read and identify dashboard readings is generated by training the extracted features. New dial area images are further collected from dashboard images of different types and environments. These images are different from the previous sample set and provide more diverse data for model optimization.

[0136] The fourth sample set is input into the generated dashboard reading model for verification and testing. If the model output is incorrect, the model weight parameters are optimized. The model is adjusted and optimized repeatedly to ensure that it can accurately identify and read the readings in different environments and different styles of dashboard images.

[0137] See also Figure 4 The second embodiment of the present invention provides a device for identifying instrument dial readings, comprising:

[0138] The dial area image positioning unit 201 is used to obtain the image to be recognized collected by the image acquisition device, and call the pre-trained YOLOv5 model to recognize the image to be recognized, so as to locate the dial area image;

[0139] The feature extraction unit 202 is used to call the pre-trained DeepLabV3+ model to perform feature extraction on the dial area image to generate deep features and shallow features, wherein the shallow features include: the specific shape of the pointer, the precise position of the scale line and the outline of the number, and the deep features include the shape and type of the instrument panel;

[0140] The fusion unit 203 is used to fuse the deep features and the shallow features to generate fused features, perform semantic segmentation on the fused features and identify the reading results of the output dial, wherein the fusion process is: using a feature pyramid network to fuse the shallow features with the deep features in a top-down and lateral connection manner.

[0141] A third embodiment of the present invention provides an instrument dial reading recognition device, including a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement an instrument dial reading recognition method as described in any one of the above.

[0142] A fourth embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program can be executed by a processor of a device where the computer-readable storage medium is located to implement a method for identifying an instrument dial reading as described in any one of the above items.

[0143] Based on the recognition method, device, equipment and storage medium of the instrument dial reading provided by the present invention, the image to be recognized collected by the image acquisition device is first obtained, and the pre-trained YOLOv5 model is called to recognize the image to be recognized to locate the dial area image; then, the pre-trained DeepLabV3+ model is called to extract features of the dial area image to generate deep features and shallow features; finally, the deep features and the shallow features are fused to generate fused features, and the fused features are semantically segmented and the dial reading result is recognized and output. The problem of insufficient recognition accuracy and reliability due to poor quality of the captured images is solved.

[0144] Exemplarily, the computer program described in the third and fourth embodiments of the present invention may be divided into one or more modules, and the one or more modules are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the recognition device for realizing a meter dial reading. For example, the device described in the second embodiment of the present invention.

[0145] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the method for identifying the reading of an instrument dial, and uses various interfaces and lines to connect the various parts of the method for identifying the reading of an instrument dial.

[0146] The memory can be used to store the computer program and / or module, and the processor realizes various functions of a method for identifying instrument dial readings by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, a text conversion function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, text message data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0147] Wherein, if the implemented module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0148] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0149] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for identifying instrument dial readings, characterized in that: include: Acquire the image to be identified acquired by the image acquisition device, and call the pre-trained YOLOv5 model to identify the image to be identified, so as to locate the dial area image; Call the pre-trained DeepLabV3+ model to extract features from the dial area image to generate deep features and shallow features, wherein the shallow features include: the specific shape of the pointer, the precise position of the scale line and the outline of the number, and the deep features include the shape and type of the dashboard; The deep features and the shallow features are fused to generate fused features, and the fused features are semantically segmented and the reading results of the output dial are recognized, wherein the fusion process is: using a feature pyramid network to fuse the shallow features with the deep features in a top-down and lateral connection manner.

2. A method for identifying instrument dial readings according to claim 1, characterized in that: The calling of the pre-trained YOLOv5 model to identify the image to be identified so as to locate the dial area image is specifically as follows: The image to be identified is divided into n*n grids, and a confidence evaluation result is generated based on whether there is an instrument in each grid, where if there is an instrument, the confidence Pc=1, and if not, the confidence Pc=0; An image annotation tool is called, and a prediction box is drawn on the image to be identified according to the confidence evaluation result, and the dial area image is located based on the position information of the prediction box.

3. A method for identifying instrument dial readings according to claim 1, characterized in that: The pre-trained DeepLabV3+ model is called to extract features from the dial area image to generate deep features and shallow features, specifically: Perform shallow feature extraction on the dial area image by using the resnet deep residual network algorithm to generate basic shallow features and feature images of the dial area image; The feature image is extracted through three different hole convolutions and global features of the image to generate a feature layer of the image, and then the deep features are obtained through a 1*1 convolution layer.

4. A method for identifying instrument dial readings according to claim 1, characterized in that: The semantic segmentation of the fused features and identification of the output dial reading result are specifically as follows: The fusion features are identified based on the DeepLabV3+ algorithm to identify the starting scale line and digital display, the end scale line and digital display, According to the coordinates of the scale line, the pointer reading and the pointer axis, the mathematical equation of the scale circle, the angle between the scales, and the angle between the scale and the pointer are generated, and then the pointer reading is generated.

5. A method for identifying instrument dial readings according to claim 1, characterized in that: Before calling the pre-trained YOLOv5 model to identify the image to be identified so as to locate the dial area image, the YOLOv5 model is also trained. The training process is specifically as follows: Obtain a dashboard data set, identify a dashboard area in the dashboard data set, and mark the area-identified pictures and non-dashboard pictures as a first sample set, wherein the dashboard data set includes dashboards of different styles and environments and pictures similar to the dashboard. Using the first sample set as model data, and extracting training set features by inputting different image data; According to the training set features, an instrument dial recognition model is generated based on the YOLOv5 algorithm; The pictures that have not been collected before are marked as the second sample set, and the second sample set is further collected. The second sample set is input into the trained instrument dial recognition model, and the model is further tuned. If the output result is incorrect, the weight parameters of the model are optimized.

6. A method for identifying instrument dial readings according to claim 1, characterized in that: The first sample set is used as model data, and different image data are input to extract training set features, specifically: Based on the algorithm matrix, each grid algorithm matrix is ​​set to 1*1*6, and all grid algorithm matrices of the picture are 19*19*6. The expression for obtaining the algorithm matrix is: y=[Pbxbybhbw]-1 Where: P c is the confidence level, b x is the x coordinate of the center point of the prediction box in the whole picture, b y is the y coordinate of the center point of the prediction box in the whole picture, b h is the predicted box height, b w is the predicted box length; All grid algorithms are mapped through CNN, and the spatial size of the input image is adjusted to a fixed size using the Resize and Normalize functions, and the homogeneous variance statistically obtained from the data set is normalized.

7. A method for identifying instrument dial readings according to claim 1, characterized in that: Before calling the pre-trained DeepLabV3+ model to extract features from the dial area image, the DeepLabV3+ model is also trained. The training process is specifically as follows: Obtaining the regional image after the instrument dial area is extracted, marking the starting scale line and digital display, the end scale line and digital display of the instrument, and using the regionally marked image as the third sample set; Extracting features of the third sample set, and generating a dashboard reading model based on the DeepLabV3+ model, The instrument dial area pictures different from those collected previously are marked as the fourth sample set, and the fourth sample set is input into the instrument dial reading model. If the input result is incorrect, the weight parameters of the instrument dial intelligent recognition model are optimized.

8. A device for identifying instrument dial readings, characterized in that: include: A dial area image positioning unit, used to obtain the image to be identified acquired by the image acquisition device, and call the pre-trained YOLOv5 model to identify the image to be identified, so as to locate the dial area image; A feature extraction unit, used to call the pre-trained DeepLabV3+ model to perform feature extraction on the dial area image to generate deep features and shallow features, wherein the shallow features include: the specific shape of the pointer, the precise position of the scale line and the outline of the number, and the deep features include the shape and type of the instrument panel; A fusion unit is used to fuse the deep features and the shallow features to generate fused features, perform semantic segmentation on the fused features and identify the reading results of the output dial, wherein the fusion process is: using a feature pyramid network to fuse the shallow features with the deep features in a top-down and lateral connection manner.

9. A device for identifying instrument dial readings, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a method for identifying an instrument dial reading as claimed in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program can be executed by a processor of the device where the computer-readable storage medium is located to implement a method for identifying an instrument dial reading as described in any one of claims 1 to 6.

Images