A method for identifying and reading pointer instruments

Abstract

The present invention is applicable to the field of image recognition, and discloses a pointer instrument recognition and reading method, comprising: performing target detection on an image to be read, cropping the image to be read according to the detection result, and obtaining a pointer instrument area image; performing semantic segmentation on the pointer instrument area image, and obtaining a multi-channel mask of a pointer and a scale in the pointer instrument area image; performing data conversion on the multi-channel mask, and obtaining a corresponding two-dimensional rectangular label map; obtaining the height and width of the two-dimensional rectangular label map, and creating a first zero array and a second zero array, the first zero array being used to store a pointer position, and the second zero array being used to store scale information; traversing each pixel in the two-dimensional rectangular label map, and obtaining a pointer array containing a pointer position and a scale array containing scale information; determining the scale position, the pointer position, and the scale position pointed to by the pointer based on the pointer array and the scale array, and performing reading; the method can improve the reading recognition speed and recognition accuracy.

Description

Technical Field

[0001] The invention relates to the field of image recognition, and in particular to a pointer instrument recognition reading method. Background Art

[0002] As a relatively traditional measuring instrument, pointer instruments are widely used in scientific experiments, industrial production and other fields due to their simple structure, easy maintenance, low price, high reliability, strong adaptability and little environmental impact. This makes them still have an irreplaceable position after the emergence of digital instruments.

[0003] The traditional method of obtaining the readings of the pointer instrument is to manually observe the readings of the pointer on the instrument dial. This method is greatly affected by subjective factors and objective conditions. For example, the different observation angles, observation distances, and judgment and estimation abilities of the readers will cause large errors in the readings, and the reliability is not high. Once real-time monitoring is required, staff will be required to be on duty, which increases the labor intensity and reduces safety. On the other hand, if the environment in which the instrument is used and installed is relatively harsh, it may also be unfavorable for manual readings, such as the installation location is too high from the ground, or the working environment is too high. Therefore, with the continuous development and maturity of technologies such as computer vision technology and image processing technology, computer vision technology has been introduced in instrument readings to achieve automatic recognition of readings.

[0004] For pointer instruments, although the automatic recognition algorithm of pointer instrument readings based on image processing has made some progress to a certain extent, there are still some shortcomings and limitations. The following are some of the main problems:

[0005] The use environment of pointer instruments is complex: Since pointer instruments may be used in various environments, factors such as lighting conditions, background noise, pointer color, scale line clarity, etc. may affect the recognition algorithm. These environmental factors may cause image quality to deteriorate, thereby affecting recognition accuracy.

[0006] Recognition accuracy is difficult to guarantee: Although some advanced recognition algorithms can achieve high recognition accuracy under ideal conditions, in actual applications, due to the influence of various factors, recognition accuracy is often difficult to guarantee. In particular, when the pointer overlaps with the scale line, the pointer shakes, or the image quality is poor, the recognition accuracy is more easily affected.

[0007] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0008] The object of the present invention is to provide a pointer instrument identification reading method, which can improve the identification speed and identification accuracy in the intelligent reading of pointer instrument identification reading.

[0009] To achieve the above object, the solution provided by the present invention is:

[0010] A pointer instrument recognition and reading method, the pointer instrument recognition and reading method comprising: obtaining an image to be read, and performing target detection on the image to be read, cropping the image to be read according to the detection result to obtain a pointer instrument area image; performing semantic segmentation on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image; performing data conversion on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask; obtaining the height and width of the two-dimensional rectangular label map, and creating a first zero array and a second zero array, the size of the first zero array and the size of the second zero array are both the same as the width of the two-dimensional rectangular label map, the first zero array is used to store the pointer position, and the second zero array is used to store the pointer position. A two-zero array is used to store scale information; traverse each pixel in the two-dimensional rectangular label map, if the current pixel is a pointer label, increase the pointer count of the column where the current pixel is located, and update the first zero array; if the current pixel is a scale label, increase the scale count of the column where the current pixel is located, and update the second zero array, to obtain a pointer array containing pointer positions and a scale array containing scale information; determine the scale position and pointer position based on the pointer array and the scale array, respectively, and determine the scale position pointed to by the pointer according to the scale position and the pointer position; obtain a predicted scale number according to the scale position pointed to by the pointer, and determine the reading of the pointer instrument according to the predicted scale number, the range of the preset pointer instrument and the total scale number of the preset pointer instrument.

[0011] Preferably, the semantic segmentation of the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image also includes: semantic segmentation preprocessing of the pointer instrument area image to obtain a standardized pointer instrument area image.

[0012] Preferably, the semantic segmentation preprocessing of the pointer instrument area image to obtain a standardized pointer instrument area image specifically includes: acquiring the shape of the pointer instrument area image, calculating the ratio between the target size and the size of the pointer instrument area image, and obtaining the resized pointer instrument area image; performing channel transposition on the resized pointer instrument area image; and performing normalization and standardization on the channel transposed pointer instrument area image to obtain a standardized pointer instrument area image.

[0013] Preferably, when semantic segmentation is performed on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image, semantic segmentation is performed on the pointer instrument area image by using a trained pointer scale detection model based on the DeepLabv3+ semantic segmentation algorithm.

[0014] Preferably, the data conversion of the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask also includes: performing an erosion operation on the multi-channel mask.

[0015] Preferably, the data conversion of the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask includes: creating circular labels and a second-order zero matrix, wherein the second-order zero matrix is ​​used to store processed data; traversing the second-order zero matrix to calculate the angle and polar radius of the current position of the second-order zero matrix; determining the position coordinates of the circular label according to the calculated angle and polar radius, and calculating the rectangular coordinates of the multi-channel mask at the corresponding position according to the position coordinates of the circular label; mapping the rectangular coordinates of the multi-channel mask to the second-order zero matrix, and converting the mapped second-order zero matrix to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask.

[0016] Preferably, the center of the circular label is , the radius is , a second-order zero matrix with m rows and n columns It is expressed as:

[0017]

[0018] The angle of the current position of the second-order zero matrix It is expressed as:

[0019]

[0020] The polar radius of the current position of the second-order zero matrix It is expressed as:

[0021]

[0022] Cartesian coordinates of multi-channel masks for:

[0023]

[0024] Where a and b are constants.

[0025] Preferably, the method of determining the scale position and the pointer position based on the pointer array and the scale array, respectively, and determining the scale position pointed to by the pointer according to the scale position and the pointer position, includes: traversing each column of the scale array to find continuous scales, and when continuous scales are detected, recording the starting position and the ending position of the current scale, and calculating its center position, and storing the center position of each scale in a preset first list, thereby determining the scale position; traversing each column of the pointer array, if the values ​​of the current column and the next column are both greater than 0, recording the starting position of the pointer, and when the values ​​of the two adjacent columns are 0, recording the ending position of the pointer, and calculating the center position of the pointer, and storing the center position of the pointer in a preset second list, thereby determining the pointer position; traversing the first list, comparing each pair of adjacent scale positions, if the center position of the pointer points to a certain scale position, using the scale number corresponding to the scale position as the predicted scale number, and if the pointer position is between two adjacent scale positions, using linear interpolation to calculate the predicted scale number.

[0026] Preferably, the reading of the pointer instrument It is expressed as:

[0027]

[0028] In the formula, represents the number of prediction scales, Indicates the total number of scales of the pointer instrument. Indicates the measuring range of a pointer instrument.

[0029] Preferably, if there are multiple pointer instruments in the image to be read, the predicted scale number is obtained according to the scale position pointed to by the pointer, and the reading of the pointer instrument is determined according to the predicted scale number, the preset range of the pointer instrument and the preset total scale number of the pointer instrument, which also includes: comparing the scale number of the scale array with a preset scale threshold to determine the type of the current pointer instrument; and obtaining the range corresponding to the pointer instrument according to the type of the current pointer instrument.

[0030] The pointer instrument recognition and reading method provided by the present invention realizes automatic recognition of pointer instrument readings through image processing technologies such as target detection and semantic segmentation, greatly improves the degree of automation of reading instrument readings, reduces the need for manual intervention, and converts a multi-channel mask into a two-dimensional rectangular label map, and creates two zero arrays for storing pointer position and scale information respectively, which not only simplifies the subsequent data processing flow, allowing the program to traverse and analyze information in the image more efficiently, but also realizes precise positioning of the pointer and scale, thereby obtaining more accurate readings. In addition, in the process of traversing the two-dimensional rectangular label map, by increasing the pointer count and scale count of the corresponding column, the approximate position of the pointer and the distribution of the scale information can be quickly counted, and the error caused by pixel noise or misidentification can be reduced, which is faster and more accurate than traditional pixel-by-pixel analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0032] Figure 1 The present invention is a flowchart of a pointer instrument identification and reading method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 In an embodiment of the present invention, a method for identifying a reading of a pointer instrument includes:

[0035] S101, obtaining an image to be read, performing target detection on the image to be read, and cropping the image to be read according to the detection result to obtain an image of the pointer instrument area;

[0036] S102, performing semantic segmentation on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image;

[0037] S103, performing data conversion on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask;

[0038] S104, obtaining the height and width of the two-dimensional rectangular label map, and creating a first zero array and a second zero array, the size of the first zero array and the size of the second zero array are the same as the width of the two-dimensional rectangular label map, the first zero array is used to store the pointer position, and the second zero array is used to store scale information;

[0039] S105, traverse each pixel in the two-dimensional rectangular label map, if the current pixel is a pointer label, increase the pointer count of the column where the current pixel is located, and update the first zero array, if the current pixel is a scale label, increase the scale count of the column where the current pixel is located, and update the second zero array, to obtain a pointer array containing a pointer position and a scale array containing scale information;

[0040] S106, determining the scale position and the pointer position based on the pointer array and the scale array respectively, and determining the scale position pointed to by the pointer according to the scale position and the pointer position;

[0041] S107, obtaining a predicted scale number according to the scale position pointed by the pointer, and determining a reading of the pointer instrument according to the predicted scale number, a preset measuring range of the pointer instrument and a preset total scale number of the pointer instrument.

[0042] In this embodiment, the image to be read is a focused image.

[0043] In this embodiment, if there are multiple pointer instruments in the image to be read, multiple cropping is performed to obtain multiple pointer instrument area images, and steps S102 to S105 are performed on each pointer instrument area image.

[0044] In this embodiment, automatic recognition of pointer instrument readings is achieved through image processing technologies such as target detection and semantic segmentation, which greatly improves the degree of automation in reading instrument readings and reduces the need for manual intervention. In addition, the multi-channel mask is converted into a two-dimensional rectangular label map, and two zero arrays are created to store the pointer position and scale information respectively. This not only simplifies the subsequent data processing process, allowing the program to traverse and analyze the information in the image more efficiently, but also achieves precise positioning of the pointer and scale, thereby obtaining more accurate readings. In addition, in the process of traversing the two-dimensional rectangular label map, by increasing the pointer count and scale count of the corresponding column, the approximate position of the pointer and the distribution of the scale information can be quickly counted, and the errors caused by pixel noise or misidentification can be reduced. Compared with traditional pixel-by-pixel analysis, it is faster and more accurate.

[0045] In this embodiment, in step S101, when performing target detection on the image to be read, the trained YOLOv5 target detection model is used to perform target detection on the target image. The trained YOLOv5 target detection model outputs a detection result including a pointer instrument bounding box, and finally the pointer instrument area image is cropped according to the detected bounding box.

[0046] It can be understood that the YOLOv5 target detection model is based on the real-time target detection algorithm, which divides the image into grids and makes detailed predictions for each grid to quickly and accurately locate and identify the target. By inputting the image to be read into the YOLOv5 target detection model, the system can quickly and accurately locate the instrument and obtain its reading, thereby realizing real-time monitoring of the target instrument.

[0047] In this embodiment, before performing target detection on the image to be read, it also includes: preprocessing the image to be read to ensure that the image to be read meets the input requirements of the YOLOv5 target detection model, thereby improving the accuracy and efficiency of the YOLOv5 target detection model.

[0048] Preprocessing the image to be read may include scaling the image to an appropriate size based on the input requirements of the model.

[0049] Preprocessing the image to be read can also include scaling the pixel values ​​of the image to a smaller range (usually between 0 and 1 or between -1 and 1), which helps the YOLOv5 object detection model to better learn image features.

[0050] Preprocessing the image to be read can also include converting the image to a format that matches the YOLOv5 object detection model, such as a NumPy array or a tensor in a deep learning framework such as PyTorch or TensorFlow.

[0051] In some cases, to improve the generalization ability of the YOLOv5 object detection model, image augmentation techniques such as random cropping, rotation, flipping, brightness or contrast adjustment, etc. can be applied during the preprocessing stage.

[0052] In this embodiment, semantic segmentation is performed on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale parts of the meter, which also includes: semantic segmentation preprocessing is performed on the pointer instrument area image to obtain a standardized pointer instrument area image.

[0053] The semantic segmentation preprocessing is performed on the pointer instrument area image to obtain a standardized pointer instrument area image, specifically including: obtaining the shape of the pointer instrument area image, calculating the ratio between the target size and the size of the pointer instrument area image, and obtaining the resized pointer instrument area image; performing channel transposition on the resized pointer instrument area image; and performing normalization and standardization on the channel transposed pointer instrument area image to obtain a standardized pointer instrument area image.

[0054] In this embodiment, performing channel transposition on the resized pointer meter region image is performed by changing the order of channels from channel×height×width to channel×width×height.

[0055] In this embodiment, each pixel value of the pointer instrument area image after channel transposition is divided by 255 for normalization, and then the resized pointer instrument area image is standardized using the mean and standard deviation, that is, the mean is subtracted and divided by the standard deviation (the mean and standard deviation are both set to 0.5) to obtain a standardized pointer instrument area image.

[0056] In this embodiment, in step S102, semantic segmentation is performed on the pointer instrument area image by using a trained pointer scale detection model based on the DeepLabv3+ semantic segmentation algorithm to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image. In this process, a segmentation map (or mask) of the same size as the pointer instrument area image is generated, in which each pixel is assigned a category label (such as background, pointer, scale, etc.).

[0057] Find the multi-channel mask of the pointer and scale parts according to the category label. Each channel corresponds to a category, for example, one channel represents the pointer and another channel represents the scale.

[0058] In the segmentation result, each pixel has a corresponding confidence score, which indicates how reliable the pointer scale detection model is in predicting the pixel category.

[0059] In this embodiment, the category with the highest confidence in the segmentation result can be found by averaging or taking the maximum value of the confidence scores of all pixels belonging to the same category.

[0060] In this embodiment, the segmentation result may be converted into a NumPy array for subsequent processing.

[0061] In this embodiment, before performing data conversion on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask, the method further includes: performing an erosion operation on the multi-channel mask, including removing burrs on the multi-channel mask, and the erosion operation on the multi-channel mask may also include separating two slightly connected objects in the multi-channel mask to obtain a clearer multi-channel mask. Specifically, a matrix with all elements being 1 is created, used as a core of the erosion operation, and the erosion result is stored after the erosion operation is performed on the multi-channel mask.

[0062] It is understandable that the size of the core of the corrosion operation can be adjusted according to the actual application conditions and changes. In this embodiment, the size of the core of the corrosion operation is 3.

[0063] In this embodiment, a circular pointer instrument is used as an example. In step S103, data conversion is performed on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask, which specifically includes:

[0064] Create a circle with the center at , the radius is Circular labels and create a second-order zero matrix , the second-order zero matrix Used to store processed data; traverse the second-order zero matrix , calculate the second-order zero matrix The angle of the current position and polar radius ;According to the calculated angle and polar radius Determine the position coordinates of the circular label , and according to the position coordinates of the circular label Calculate the rectangular coordinates of the multi-channel mask at the corresponding position ; The rectangular coordinates of the multi-channel mask Mapping to a second-order zero matrix On, that is, with the rectangular coordinates of the multi-channel mask The corresponding value replace ; For the mapped second-order zero matrix The conversion process is performed to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask.

[0065] In this embodiment, the second-order zero matrix As a meter, the second-order zero matrix The size of is the same as the size of the multi-channel mask.

[0066] A second-order zero matrix with m rows and n columns It is expressed as:

[0067]

[0068] In this embodiment, the second-order zero matrix The angle of the leftmost column is 0 degrees, the angle of the rightmost column is 2Π.

[0069] The angle of the current position of the second-order zero matrix It is expressed as:

[0070]

[0071] Where c represents the current column of the second-order zero matrix;

[0072] The polar radius of the current position of the second-order zero matrix It is expressed as:

[0073]

[0074] Where r represents the current row of the second-order zero matrix;

[0075] Cartesian coordinates of multi-channel masks It is expressed as:

[0076]

[0077] Where a and b are constants.

[0078] In this embodiment, in step S104, the two-dimensional rectangular label map is traversed. , get a 2D rectangular label map Height and width , and create a The first zero array and the second zero array , the first zero array and the second zero array They are used to store pointer position and scale information respectively. When initialized, the first zero array and the second zero array Each element in is set to 0.

[0079] In step S105, for the two-dimensional rectangular label map For each pixel in, traverse each column , each row If the current pixel is a pointer label, the pointer count of the column is increased, i.e. If the current pixel is a tick label, increase the tick count of the column, i.e. , get the pointer array containing the pointer position and the scale array containing the scale information.

[0080] In this embodiment, the pointer instrument identification reading method further includes binarizing the pointer array and the scale array, which is as follows: looping the pointer array and the scale array, calculating their length (i.e., the number of data) and the average value of the data. Then looping the pointer array and the scale array, checking whether the value of each element is less than the average value. If it is less than the average value, the value of the column is set to 0, otherwise it is set to 1, and finally the binarized pointer array and scale array are obtained, which are easier to be subsequently processed or analyzed.

[0081] In this embodiment, in step S106, the scale position and the pointer position are determined based on the pointer array and the scale array, respectively, and the scale position pointed to by the pointer is determined according to the scale position and the pointer position, including: traversing each column of the scale array to find continuous scales (continuous 1s), when continuous scales are detected, recording the starting position and the ending position of the current scale, and calculating its center position, storing the center position of each scale in a preset first list, thereby determining the scale position; traversing each column of the pointer array, if the values ​​of the current column and the next column are both greater than 0, recording the starting position of the pointer, and when the values ​​of the two adjacent columns are 0, recording the ending position of the pointer, and calculating the center position of the pointer, storing the center position of the pointer in a preset second list, thereby determining the pointer position; initializing the predicted scale number variable, traversing the first list, comparing each pair of adjacent scale positions, if the center position of the pointer points to a certain scale position, then the scale number corresponding to the scale position is used as the predicted scale number, if the pointer position is between two adjacent scale positions, using linear interpolation to calculate the predicted scale number.

[0082] In this embodiment, since it is necessary to compare with the next column, each column of the scale array is traversed from the first column to the second to last column during operation. In each column, it is checked whether the values ​​of the current column and the next column are both greater than 0. If so, it means that a new start of the scale may be found. This is because consecutive scales are considered to be the same scale. Therefore, if the values ​​of the current column and the next column are both greater than 0, it means that the starting position of the scale has been found. If a new starting position of the scale is found, the position of the current column is set to the starting position of the scale, indicating that a new start of the scale has been found.

[0083] Check if the start position of the scale has been found. If so, continue to check if the values ​​of the current column and the next column are both 0. If so, it means that the current scale has ended. At this time, subtract 1 from the current column position as the end position of the scale (to avoid counting the last point in the next scale). Next, calculate the center position of the current scale, which is obtained by taking the average of the start position and the end position. Finally, store this center position in the first list Output is performed in.

[0084] In this embodiment, the method for determining the pointer position is similar to the method for determining the scale position. Each column of the pointer array is traversed. If the values ​​of the current column and the next column are both greater than 0, it means that a new starting position of the pointer may be found. If the starting position of the pointer has not been found, the position of the current column is set as the starting position of the new pointer.

[0085] If the starting position of the pointer has been found, continue to check whether the values ​​of the current column and the next column are both 0. If so, it means that the end position of the current pointer has been found. Set the position of the current column to the end position of the current pointer, calculate the center position of the pointer, and add the calculated center position to the second table. Output is performed in.

[0086] In this embodiment, when determining the scale position pointed to by the pointer according to the scale position and the pointer position, the zip function in Python is used to iterate the scale position and the pointer position at the same time, and initialize the predicted scale number variable , then traverse the first list and compare each pair of adjacent tick positions.

[0087] It can be understood that linear interpolation is used to estimate the value of an unknown point between two known data points. It is based on the linear relationship between the two known points and is estimated in the form of a linear equation. Linear interpolation assumes that the change between the two known points is linear, so a straight line can be used to connect the two points and use this line to estimate the value of other points. For example, for the known point and , linear interpolation can be used to calculate the y value corresponding to any x value between these two points using the following formula.

[0088]

[0089] In this embodiment, Assume that the total number of scales of the pointer instrument is , the range is , then the calculation result of the reading part is It is expressed as:

[0090] In the formula, Indicates the number of forecast scales.

[0091] If there are multiple pointer instruments in the image to be read, the type of instrument is determined based on whether the number of scales is greater than a predefined threshold, and then the corresponding instrument configuration information is selected. According to the meter type and pointer position, the corresponding scale interval value is used to calculate the instrument reading, and the calculated reading is finally returned.

[0092] In this embodiment, the threshold value and the total number of scales, the range and the unit corresponding to each meter type are preset. The meter type is determined by comparing the number of scales in the first list with the threshold value.

[0093] It should be noted that the scale threshold The number is set according to the actual situation. For example, when there are multiple pointer instruments in the image to be read, the scale threshold is determined according to the number of pointer instruments. For example, there are a voltmeter and a pressure gauge in the image to be read, the maximum scale number of the voltmeter is 10, and the maximum scale number of the pressure gauge is 50, then a threshold is set, the threshold is 30, when the maximum scale number of the pointer instrument area image is greater than 30, it means that the current reading is the pressure gauge reading, when the maximum scale number of the pointer instrument area image is less than 30, it means that the current reading is the voltmeter reading.

[0094] For another example, the image of the to-be-read data includes an ammeter, a voltmeter, and a pressure gauge, the maximum scale number of the ammeter is 20, the maximum scale number of the voltmeter is 10, and the maximum scale number of the pressure gauge is 50, then two thresholds are set, 15 and 30 respectively, when the maximum scale number of the pointer instrument area image is greater than 30, it indicates that the current reading is the pressure gauge reading, when the maximum scale number of the pointer instrument area image is greater than 10 and less than 30, it indicates that the current reading is the ammeter reading, and when the maximum scale number of the pointer instrument area image is less than 15, it indicates that the current reading is the voltmeter reading.

[0095] Furthermore, if the maximum number of scales of two or more pointer instruments is the same, a classification model may be added to classify the pointer instrument area image.

[0096] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for identifying and reading a pointer instrument, characterized in that: The pointer instrument identification reading method comprises: Acquire an image to be read, perform target detection on the image to be read, and crop the image to be read according to the detection result to obtain an image of the pointer instrument area; Performing semantic segmentation on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image; Performing data conversion on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask; Obtaining the height and width of the two-dimensional rectangular label map, and creating a first zero array and a second zero array, wherein the size of the first zero array and the size of the second zero array are both the same as the width of the two-dimensional rectangular label map, the first zero array is used to store the pointer position, and the second zero array is used to store scale information; Traverse each pixel in the two-dimensional rectangular label map, if the current pixel is a pointer label, increase the pointer count of the column where the current pixel is located, and update the first zero array; if the current pixel is a scale label, increase the scale count of the column where the current pixel is located, and update the second zero array, to obtain a pointer array containing a pointer position and a scale array containing scale information; Determine the scale position and the pointer position based on the pointer array and the scale array respectively, and determine the scale position pointed to by the pointer according to the scale position and the pointer position; The predicted scale number is obtained according to the scale position pointed by the pointer, and the reading of the pointer instrument is determined according to the predicted scale number, the preset range of the pointer instrument and the preset total scale number of the pointer instrument.

2. The pointer instrument identification reading method according to claim 1, characterized in that: The method of performing semantic segmentation on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image also includes: performing semantic segmentation preprocessing on the pointer instrument area image to obtain a standardized pointer instrument area image.

3. The pointer instrument identification reading method according to claim 1, characterized in that: The performing semantic segmentation preprocessing on the pointer instrument area image to obtain a standardized pointer instrument area image specifically includes: The shape of the pointer instrument area image is obtained, and the ratio between the target size and the size of the pointer instrument area image is calculated to obtain the pointer instrument area image after the size is adjusted; Transpose the channels of the resized pointer instrument area image; The pointer instrument area image after channel transposition is normalized and standardized to obtain a standardized pointer instrument area image.

4. The pointer instrument identification reading method according to claim 1, characterized in that: When semantic segmentation is performed on the pointer instrument area image to obtain a multi-channel mask of the pointer and scale in the pointer instrument area image, semantic segmentation is performed on the pointer instrument area image through a trained pointer scale detection model based on the DeepLabv3+ semantic segmentation algorithm.

5. The pointer instrument identification reading method according to claim 1, characterized in that: The data conversion is performed on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask, and before that, it also includes: performing an erosion operation on the multi-channel mask.

6. The pointer instrument identification reading method according to claim 1, characterized in that: The step of performing data conversion on the multi-channel mask to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask includes: Creating a circular label and a second-order zero matrix, wherein the second-order zero matrix is ​​used to store processed data; Traversing the second-order zero matrix, calculating the angle and polar radius of the current position of the second-order zero matrix; Determine the position coordinates of the circular tag according to the calculated angle and polar radius, and calculate the rectangular coordinates of the multi-channel mask at the corresponding position according to the position coordinates of the circular tag; The rectangular coordinates of the multi-channel mask are mapped onto the second-order zero matrix, and the mapped second-order zero matrix is ​​transformed to obtain a two-dimensional rectangular label map corresponding to the multi-channel mask.

7. The pointer instrument identification reading method according to claim 6, characterized in that: The center of the circular label is , the radius is , a second-order zero matrix with m rows and n columns It is expressed as: The angle of the current position of the second-order zero matrix It is expressed as: Where c represents the current column of the second-order zero matrix; The polar radius of the current position of the second-order zero matrix It is expressed as: Where r represents the current row of the second-order zero matrix; Cartesian coordinates of multi-channel masks for: Where a and b are constants.

8. The pointer instrument identification and reading method according to claim 1, characterized in that: The determining of the scale position and the pointer position based on the pointer array and the scale array, respectively, and determining the scale position pointed to by the pointer according to the scale position and the pointer position, comprises: Traversing each column of the scale array to find continuous scales, when continuous scales are detected, recording the starting position and the ending position of the current scale, calculating its center position, and storing the center position of each scale in a preset first list, thereby determining the scale position; Traverse each column of the pointer array, if the values ​​of the current column and the next column are both greater than 0, record the starting position of the pointer, and when the values ​​of the two adjacent columns are 0, record the ending position of the pointer, calculate the center position of the pointer, and store the center position of the pointer in a preset second list, thereby determining the pointer position; Traverse the first list and compare each pair of adjacent scale positions. If the center position of the pointer points to a certain scale position, the scale number corresponding to the scale position is used as the predicted scale number. If the pointer position is between two adjacent scale positions, the predicted scale number is calculated using linear interpolation.

9. The pointer instrument identification reading method according to claim 1, characterized in that: The reading of the pointer instrument It is expressed as: In the formula, represents the number of prediction scales, Indicates the total number of scales of the pointer instrument. Indicates the measuring range of a pointer instrument.

10. The pointer instrument identification and reading method according to claim 1, characterized in that: If there are multiple pointer instruments in the image to be read, the predicted scale number is obtained according to the scale position pointed to by the pointer, and the reading of the pointer instrument is determined according to the predicted scale number, the preset range of the pointer instrument and the preset total scale number of the pointer instrument, which also includes: comparing the scale number of the scale array with a preset scale threshold to determine the type of the current pointer instrument; and obtaining the range corresponding to the pointer instrument according to the type of the current pointer instrument.

Images