A meter reading recognition method based on self-generated and annotated canonical benchmark test sets

Abstract

The application discloses a kind of based on self-generation and annotation specification benchmark test set instrument reading identification method, applied to instrument transmission technical field, including: using Python script to carry out random sampling setting to measurement parameter and appearance parameter, and generate instrument data set by Blender;Annotation is carried out using the whole automatic specification annotation framework, and specification benchmark test set is obtained;Scale mark and pointer prediction are carried out using heat map prediction model based on DINOv2 and key point extraction algorithm, and multi-stage extraction method is used for intermediate scale mark that may contain uncertain number of key points;After text detection and identification, the required homography matrix is directly regressed from original image end to end using regression network to carry out perspective correction, and the relationship between polar angle and gauge reading is fitted using RANSAC algorithm, and the numerical value corresponding to pointer polar angle is predicted.The application effectively improves instrument reading identification precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of instrument transmission, more particularly, to an instrument reading identification method based on self-generated and annotated standard benchmark test set. BACKGROUND

[0002] In industrial automation systems, many devices transmit critical data through analog instruments, which are not only used to detect system abnormalities, but also maintain the stability of operations through control systems. The traditional manual interpretation method is inefficient, especially impractical in large-scale industrial applications. With the advancement of industrial detection devices, including straightness gauges, drones, and robot platforms integrated with computer vision algorithms, the automation of the instrument reading process has gradually been realized. Although these systems have achieved a relatively high level of automation, there are still limitations in reading accuracy, which requires secondary manual verification. This challenge is particularly prominent in the interpretation of analog instruments, as interpreting the instrument requires considering multiple visual elements (such as the dial, scale, pointer, and instrument numbers) to infer the measurement value, which poses strict requirements for the reading algorithm.

[0003] The scarcity of standardized instrument datasets is a major obstacle to the development of algorithms in this field, and the development of analog instrument reading algorithms is significantly limited by data quality issues. In addition, there is a lack of high-quality annotations required for effective algorithm training. The three main factors contributing to the scarcity of data are: (1) Data acquisition challenges: Under stable operating conditions, the captured instrument images change very little due to the minimal changes in measured physical quantities. On the contrary, manually taken photos are not only labor-intensive but also time-consuming, which hinders the effective creation of large-scale datasets. (2) High labeling cost: The most advanced instrument reading algorithms typically use deep learning architectures, breaking down the task into multiple subtasks such as instrument detection, pointer identification, and scale marker identification, each requiring specialized labeling. Certain implementations also require semantic segmentation or keypoint detection of scale markers, which requires meticulous labeling of many fine-grained features in a single image. (3) Data privacy restrictions: In industrial environments, instrument systems are embedded in complex operational frameworks and are subject to strict privacy regulations, which greatly limit access to third-party data.

[0004] At the same time, there is still room for further development of instrument data measurement reading algorithms.

[0005] Therefore, how to provide an instrument reading identification method based on self-generated and annotated standard benchmark test set, which can effectively solve the hindrance to the development of algorithms in this field due to the scarcity of standardized instrument datasets, and further develop high-precision instrument data measurement reading algorithms to effectively improve the accuracy of instrument reading identification, is a problem that needs to be solved by those skilled in the art. SUMMARY

[0006] In view of this, the present application provides a kind of instrument reading identification method based on self-generation and annotation standard benchmark test set.

[0007] In order to achieve the above object, the present application adopts the following technical solutions:

[0008] An instrument reading identification method based on self-generation and annotation standard benchmark test set, comprising:

[0009] Step 1: randomly sample and set the measurement parameters and appearance parameters using Python scripts, and generate instrument data set through Blender;

[0010] Step 2: build a fully automatic standard annotation framework based on Blender 3D model parameters, annotate the instrument data set, and obtain the standard benchmark test set;

[0011] Step 3: use the heat map prediction model based on DINOv2 and the key point extraction algorithm to perform scale marking and pointer prediction, and in the key point extraction, the intermediate scale marking that may contain an uncertain number of key points is extracted by using a multi-stage extraction method;

[0012] Step 4: after text detection and recognition, use regression network to directly regress the homography matrix required for perspective correction from the original image to perform perspective correction, and use RANSAC algorithm to fit the relationship between polar angle and gauge reading, and predict the value corresponding to the pointer polar angle through the fitted model.

[0013] Optionally, in step 1, the measurement parameters include the rotation angle of the pointer and the measurement range of the instrument.

[0014] Optionally, in step 1, the appearance parameters include the color and shape of the pointer, the length of the scale marking and the numerical distance between adjacent markings, the material and corrosion condition of the instrument shell, the angle and distance of the camera relative to the instrument, the style and size of the digital text and the radial position relative to the center, the texture and noise on the dial, and the diversity of the environment.

[0015] Optionally, in step 2, the fully automatic standard annotation framework based on Blender 3D model parameters is as follows:

[0016] Gauge detection and labeling of shell bounding box: traverse all vertices of the gauge shell in Blender, and calculate the projection coordinates of the vertices in the camera image; Convert the projection coordinates to a two-dimensional coordinate system with the upper left corner as the origin, and determine the bounding box of the gauge shell by collecting the maximum and minimum xy values of the projection coordinates;

[0017] Perspective correction: establish the correspondence between two point sets: the positions of four co-circular equidistant points in the world coordinate system and the corresponding points of the camera projection of the four co-circular equidistant points; normalize both coordinate sets relative to the detected gauge boundary box, and calculate the perspective transformation matrix between the normalized coordinate systems using the homography estimation method of OpenCV as the training target of the distortion correction model;

[0018] Scale mark detection: divided into key point detection and semantic segmentation; for key point detection, label the points adjacent to the center of the gauge at each major scale mark, and for semantic segmentation, label the polygons according to the geometric shapes of the scale marks;

[0019] Light point detection: divided into key point detection and semantic segmentation; for key point detection, define the intersection of the pointer rotation axis and the rotation axis surface as the starting position, and determine the end point by traversing all points on the pointer and selecting the point farthest from the starting position; for pointer segmentation labeling, a mask-based labeling method is used;

[0020] Optical character recognition technology OCR: including text positioning and text recognition, generating a boundary box for all text objects and associating the text content corresponding to the boundary box for recognition.

[0021] Optionally, in step 3, before using the DINOv2-based heat map prediction model and key point extraction algorithm to predict scale marks and pointers, it further includes: using YOLOv11s to position the instrument, and separating the instrument from the complex background.

[0022] Optionally, in step 3, using the DINOv2-based heat map prediction model and key point extraction algorithm to predict scale marks and pointers, specifically:

[0023] Select five key point categories, including: starting scale mark, ending scale mark, intermediate scale mark, starting position of pointer, and end point of pointer;

[0024] Use the heat map prediction model to predict the heat map for each key point category; the heat map prediction model is as follows:

[0025] Use the frozen pre-trained DINOv2 as an encoder to extract features from the input image; use a multi-stage upsampling network to generate a heat map of the same size as the original image, and use the BCE loss as the prediction loss; wherein each upsampling stage consists of a convolutional layer and a transposed convolutional layer, the latter performs 2x upsampling, and the final heat map is generated through a convolutional layer;

[0026] Extract key points from each key point category heat map; for intermediate scale marks, each heat map corresponds to a key point, and the mean shift algorithm is used for extraction.

[0027] Optionally, in step 3, a multi-stage extraction method is used for the intermediate scale markers that may contain an uncertain number of key points in key point extraction, specifically:

[0028] Significant region filtering: Set a response threshold c, and only keep the candidate regions whose heat map response satisfies H(x, y) ≥ c;

[0029] Local maximum extraction: Use a k x k sliding window to perform maximum filtering operation to detect local maximum values that satisfy the following conditions:

[0030]

[0031] Non-maximum suppression: Use a greedy algorithm based on spatial distance constraint to select key points in descending order of confidence, and remove low-confidence candidate points within a radius r of the selected key points;

[0032] Sub-pixel refinement: Use quadratic interpolation to adjust the integer pixel coordinates, and the peak shift is estimated by Taylor expansion:

[0033]

[0034] The key point coordinates (x, y) are optimized to (x + Δx, y + Δy), where |Δx| and |Δy| are both not more than 0.5.

[0035] Optionally, in step 4, YOLOv11s is used for text detection, and ABINet is used for text recognition.

[0036] Optionally, in step 4, a regression network is used to directly regress the homography matrix required for perspective correction from the original image to perform perspective correction, specifically:

[0037] A deep learning model is trained using homography annotations in synthetic data to predict homography transformation;

[0038] Homography transformation is represented with eight degrees of freedom, and the model predicts a vector h ∈ R 1×8 ; Use a pre-trained DINO backbone network as an encoder to extract features from the input image, and predict the final homography transformation through a linear layer, and use L1 loss function for training;

[0039] In the inference process, add a 1 at the end of the predicted vector h, and reshape it to a homogenous matrix suitable for images with normalized width and height For images with shape (h, w), the homographic transformation matrix H homo is calculated as follows:

[0040]

[0041] After obtaining the homographic transformation, it is applied to the key points and text coordinates detected in the previous stages and mapped onto the corrected perspective plane.

[0042] Optionally, in step 4, the relationship between the polar angle and the scale reading is obtained as follows:

[0043] By combining the confidence scores of detection and recognition, the OCR output is filtered based on confidence;

[0044] A polar coordinate system is established, with the starting position of the pointer as the polar point and the vertical direction downward as the polar axis, to calculate the polar angle of the scale mark, the end point of the pointer and the center of the OCR bounding box;

[0045] The relationship between the polar angle and the scale reading is established, and when the polar angle of the scale mark and the polar angle of the center of the OCR bounding box satisfy the following conditions, the two are matched:

[0046]

[0047] min(d ij ,(360-d ij ))≤d;

[0048] Where d is a threshold value; when two or more matching items are detected, the corresponding relationship between the scale mark polar angle and the numerical value is recorded.

[0049] According to the technical solution described above, compared with the prior art, the present disclosure provides an instrument reading recognition method based on a self-generated and annotated standard benchmark test set. By randomly sampling measurement and appearance parameters through Python scripts, and combining Blender to generate instrument data sets, a standard benchmark test set is obtained by means of a full-automatic standard annotation framework of Blender 3D model parameters, and then a heat map prediction model based on DINOv2 and a key point extraction algorithm (including a multi-stage extraction method) are used to predict scale marks and pointers. Finally, after text detection and recognition, a regression network is used to regress a homography matrix to correct the perspective, and a RANSAC algorithm is used to fit the relationship between the polar angle and the scale reading to predict the numerical value, effectively solving the problems of lack of standardized instrument data sets hindering algorithm development and insufficient accuracy of existing reading algorithms requiring secondary manual verification. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort on the basis of the provided drawings.

[0051] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments only represent some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the scope of protection of the present application.

[0053] Embodiment 1

[0054] Embodiment 1 of the present application discloses a meter reading recognition method based on self-generation and annotation of a standard benchmark test set, as shown in Figure 1 , which comprises:

[0055] Step 1: Randomly sample and set the measurement parameters and appearance parameters by using a Python script, and generate a meter data set by Blender.

[0056] In order to solve the meter data limitation in the prior art, the present application proposes a synthetic analog chart generation framework using parameterized control of Blender simulation software.

[0057] A well-designed meter synthesis framework should have two core features: parameter control and realistic reproduction. Parameter control enables the synthesized meter to be fully adjusted through configurable parameters, which cover various aspects from visual attributes to operating ranges, so as to truly simulate the meter in reality. Realistic reproduction not only requires realistic visual effects, but also systematically simulates environmental interference factors such as reflection of the glass surface, obstruction caused by pollutants, and dynamic shadow patterns.

[0058] Based on this concept, the present invention utilizes the modeling capabilities of Blender to construct gauges through six modular components, including: gauge lens, pointer, gauge number, scale marker, dial, and bezel, and establishes external parameters that control material properties and spatial configurations through system decomposition, including: measurement parameters and appearance parameters. Measurement parameters affect the numerical reading of the gauge, including the rotation angle of the pointer and the measurement range of the gauge; appearance parameters enhance the diversity of generated images by changing the appearance of gauge elements, environmental conditions, and camera perspectives, including: the color and shape of the pointer, the length of the scale marker and the numerical interval between adjacent markers, the material and corrosion condition of the gauge shell, the angle and distance of the camera, the style and size of the digital text and its radial position relative to the center, the texture and noise on the dial, and the diversity of the environment.

[0059] Pointer: The two most critical factors for the pointer are color and shape. The present invention pre-designs various pointer styles, which differ in length, thickness, and outline. To contrast with the dial, the pointer is black or white by default, while additional color variations are introduced with a certain probability by randomly adjusting the RGB properties of the pointer material.

[0060] Scale marker: The scale marker is parameterized by length and numerical interval between adjacent markers, allowing for a variety of measurement ranges and spatial distributions. Layout patterns include radial alignment variations and arc connection configurations, which are randomly selected to challenge the recognition model, although they have little impact on human readability.

[0061] Gauge shell: The material properties of the gauge shell differ in color, glossiness, and texture, all of which affect detection performance. To address these issues, the present invention randomly assigns different materials to the gauge shell. In addition, a rust texture is introduced to simulate the corrosion of metal shells in real-world environments.

[0062] Camera: Camera perspectives are simulated by constraining randomized camera positions while maintaining a 30° deviation from the surface normal. In addition, the distance between the camera and the gauge is also randomized to simulate different shooting distances.

[0063] Text diversity: The shape of the number varies in different fonts. The most notable difference is between serif and sans-serif fonts, with serif fonts having decorative strokes at the beginning and end of characters, while sans-serif fonts are designed more simply. In addition, some fonts appear slender, while others are more rounded. These differences in typography affect subsequent OCR performance. To increase diversity, the present invention introduces various font styles when generating gauges. In addition, the size of the digital text and its radial position relative to the center are parameterized, further increasing the diversity of the text.

[0064] Dial Plate: Many real-world gauges contain textual information on their dial plates to indicate the gauge type and measurement units. For example, an oil level gauge might display "OIL" and "%" on its dial plate. To simulate various real-world dial plate types, the present invention adjusts the dial plate texture accordingly. Additionally, random noise of different colors, shapes, positions, and sizes is overlaid on the dial plate to simulate effects such as rust and dust accumulation.

[0065] Environmental Diversity: The present invention collects a large number of environmental textures, including indoor environments such as apartments, factories, and bathrooms, as well as outdoor environments such as grasslands, mountains, and lakes. When generating each gauge model, a random environment is selected to provide a unique background for each synthetic image, enhancing the robustness of subsequent gauge detection models. Different environments also introduce diverse lighting conditions, including strong sunlight, soft indoor lighting, and dim ambient light, such as the faint light under a bridge. These settings result in diverse lighting conditions in terms of intensity, distance, and light source type (such as point light sources, area light sources). Utilizing the optical rendering capabilities of Blender, the present invention generates rich optical effects. With cameras at different angles, these light sources produce varying degrees of refraction on the gauge's glass cover, simulating real-world reflection effects. Additionally, the interaction of these light sources casts shadows of the needle and gauge housing on the dial plate, creating more complex image features.

[0066] To achieve random sampling of measurement and appearance parameters, the present invention utilizes Python scripts to automate the generation process and enhance output diversity. Through Blender's Python API, the present invention implements functions for generating gauge models, integrating scenes, transforming cameras, and rendering exports.

[0067] Step 2: Construct a fully automatic specification annotation framework based on Blender 3D model parameters, annotate the gauge dataset, and obtain a specification benchmark set.

[0068] The main advantage of synthetic gauges lies in their automatic annotation function. Spatial coordinates of gauge components can be directly obtained from 3D models, and data can be automatically extracted through Python scripts. For general gauge reading models, annotation is required in the following stages: gauge detection, perspective correction, scale marker detection, needle detection, and OCR recognition. The present invention lists the required annotations for each stage based on relevant algorithm types.

[0069] The fully automatic specification annotation framework based on Blender 3D model parameters is as follows:

[0070] Gauge detection bounding box annotation: using object detection algorithm, need to annotate the bounding box of the gauge shell. In the Blender environment, each object is a geometric entity composed of vertices and edges. Traverse all the vertices of the gauge shell in Blender, and calculate the projection coordinates of the vertices in the camera image; convert the projection coordinates into a two-dimensional coordinate system with the top-left corner as the origin, and determine the bounding box of the gauge shell by summarizing the maximum and minimum xy values of the projection coordinates;

[0071] Perspective correction: virtual instruments have inherent advantages for perspective correction by directly accessing real spatial relationships. The present invention establishes a correspondence between two point sets: the positions of the four equidistant points on a common circle in the world coordinate system (representing the front view) and the corresponding points of the camera projection of the four equidistant points on a common circle; To focus on the perspective distortion of a specific gauge, both coordinate sets are normalized relative to the detected gauge bounding box. Specifically, the four sampling points on the gauge and their projection coordinates in the camera view are converted to a coordinate system with the top-left corner of the detected gauge bounding box as the origin. These coordinates are then normalized according to the width and height of the bounding box. Finally, the homography estimation method of OpenCV is used to calculate the perspective transformation matrix between the normalized coordinate systems as the training target for the distortion correction model;

[0072] Scale mark detection: divided into two main algorithm types: key point detection and semantic segmentation; for key point detection, annotate points adjacent to the gauge center at each major scale mark, and for semantic segmentation, annotate polygons according to the geometric shape of the scale mark, i.e. select the outermost vertices along the contour of each scale mark in turn to construct a closed polygon;

[0073] Light point detection: similar to scale mark detection, divided into key point detection and semantic segmentation; for key point detection, the intersection of the pointer rotation axis and the rotation axis surface is defined as the starting position, and the end point is determined by traversing all points on the pointer and selecting the point farthest from the starting position; for pointer segmentation annotation, a mask-based annotation method is used, specifically, all objects except the pointer are disabled for rendering, and the self-luminous effect of the pointer is enhanced. This will generate a rendered image that serves as a mask for pointer segmentation;

[0074] Optical character recognition technology OCR: includes text positioning and text recognition, similar to size detection, generates a bounding box for all text objects and associates the text content corresponding to the bounding box for recognition.

[0075] The proposed annotation framework comprehensively meets the needs of mainstream instrument reading algorithms. Through post-processing, special requirements can be extracted from existing annotations, such as pointer bounding boxes. In addition, the invention records the basic properties of the instrument, including the measurement range, the number of major scale marks, and the interval between the pointer and the nearest major scale mark. Although these parameters are not necessary for direct reading tasks, they help build instrument-specific question-and-answer datasets, which are of great value for developing multi-modal base models in the instrument field.

[0076] Based on the above data synthesis and automatic annotation method, the invention constructs the specification benchmark SyncG. SyncG contains 16,000 training images and 4,000 test images, all rendered at a resolution of 1920x1080 using Blender's Cycles renderer with a sample count of 200. Through a GPU-accelerated pipeline, the generation, rendering, and annotation process of each image only takes 2-4 seconds, which can be achieved using a single NVIDIA RTX 3090 GPU. SyncG contains 145 different environments, 80% of which use transformed camera perspectives. The measurement parameters and appearance parameters of each instrument are randomly assigned, ensuring that different images can be considered to contain different instruments. Therefore, the invention successfully generates approximately 20,000 unique instruments.

[0077] Step 3: Scale mark and pointer prediction using DINOv2-based heatmap prediction model and keypoint extraction algorithm, and multi-stage extraction method for intermediate scale marks that may contain an uncertain number of key points.

[0078] The invention decomposes instrument reading into five different processing stages: instrument detection, scale mark and pointer detection, text detection and recognition, perspective correction, and final reading. The initial stage separates the instrument from the complex background to reduce environmental interference. Specifically, the invention uses YOLOv11 to locate the instrument. YOLOv11 is a lightweight single-stage object detection network. The invention chooses the smallest version of YOLOv11s and uses official pre-trained weights to initialize the training process. Subsequently, the detected instrument region is cropped and adjusted to a standard size as input for subsequent processing stages. Therefore, before scale mark and pointer prediction using the DINOv2-based heatmap prediction model and keypoint extraction algorithm, it also includes: locating the instrument using YOLOv11s and separating the instrument from the complex background.

[0079] Scale mark and pointer prediction using the DINOv2-based heatmap prediction model and keypoint extraction algorithm, specifically:

[0080] In the scale mark and pointer prediction stage, the target is to detect the positions of the pointer and scale marks in the instrument. Five key point categories are selected, including: the starting scale mark, the ending scale mark, the intermediate scale mark, the starting position of the pointer and the end point of the pointer; the key points of these scale marks are usually located near the center of the scale, while the starting point of the pointer is located at the center of rotation, and the end point is located away from the center. The traditional key point detection architecture is mainly used to solve the fixed number of prediction tasks, such as human pose estimation. However, due to the inherent variation of the number of main scale marks in different scale types, the traditional method is difficult to apply. Therefore, the present application develops a customized key point detection network, and takes the heat map prediction as a proxy task. The training process first generates a synthetic heat map, creating five different heat map channels for each key point category. Then, the present application designs a heat map prediction network, which aims to learn the mapping relationship from the original image to the heat map. Finally, in the test stage, the present application predicts the heat map from the input image, and uses a clustering algorithm to extract the key points.

[0081] A heat map prediction model is used to predict a heat map for each key point category; the heat map prediction model is as follows:

[0082] A frozen pre-trained DINOv2 is used as an encoder to extract features from the input image; a multi-stage upsampling network is used to generate a heat map with the same size as the original image, and a BCE loss is used as the prediction loss; since the encoder reduces the input image by a factor of 1 / 14 in height and width, the present application uses a multi-stage upsampling network instead of a single linear interpolation layer. Inspired by image super-resolution technology, each upsampling stage consists of a convolution layer and a transpose convolution layer, which performs 2x upsampling, and the final heat map is generated through a convolution layer;

[0083] Key points are extracted from each heat map of each key point category; for intermediate scale marks, each heat map corresponds to a key point, and the mean shift algorithm is used for extraction.

[0084] In the key point extraction, the intermediate scale marks, which may contain an uncertain number of key points, are extracted by a multi-stage extraction method, which is as follows:

[0085] Significant region filtering: set a response threshold c, and only keep the candidate regions whose heat map response satisfies H(x, y) ≥ c, so as to effectively suppress background noise;

[0086] Local maximum value extraction: a kxk sliding window is used for maximum value filtering operation to detect the local maximum value that satisfies the following conditions:

[0087]

[0088] Non-maximum suppression: A greedy algorithm based on spatial distance constraint is adopted to select key points in descending order of confidence. Low-confidence candidate points within the radius r of the selected key points are removed to reduce the problem of key point overlap in dense areas.

[0089] Sub-pixel refinement: In order to improve the positioning accuracy to the sub-pixel level, a quadratic interpolation method is used to adjust the integer pixel coordinates, and the peak shift is estimated by Taylor expansion:

[0090]

[0091] The key point coordinates (x, y) are optimized to (x+Δx, y+Δy), where |Δx| and |Δy| are both not more than 0.5, thereby significantly improving the accuracy at the sub-pixel level.

[0092] Step 4: After text detection and recognition, a regression network is used to directly regress the homography matrix required for perspective correction from the original image to perform perspective correction, and the RANSAC algorithm is used to fit the relationship between the polar angle and the gauge reading, and the value corresponding to the pointer polar angle is predicted through the fitted model.

[0093] In the text detection and recognition stage, the goal is to extract numerical text from the dashboard, allowing users to read the instrument reading without understanding the measurement range. This task is divided into two sub-tasks: Text detection: This sub-task is responsible for locating the text area and cropping for further processing. Similar to the instrument detection task, the present invention uses YOLOv11s to achieve this goal. Text recognition: This sub-task extracts the text content from the cropped image. For this purpose, the present invention uses ABINet from the MMOCR library.

[0094] A regression network is used to directly regress the homography matrix required for perspective correction from the original image to perform perspective correction, specifically:

[0095] In practical applications, due to camera instability or spatial limitations, it is often impossible to make the camera completely perpendicular to the dashboard. This distortion will change the original circular dial into an oval shape, resulting in errors in subsequent pointer angle calculation. To solve this problem, the present invention uses perspective correction technology.

[0096] To avoid dependence on the previous stage, a deep learning model is trained using homography annotations in synthetic data to predict homography transformation;

[0097] From the original image to end-to-end processing. Since homography transformation can be represented by eight degrees of freedom, the model predicts a vector h∈R 1×8; using a pre-trained DINO backbone network as an encoder to extract features from input images and predict the final homography transformation through a linear layer, trained using an L1 loss function;

[0098] During inference, a 1 is added to the end of the prediction vector h and reshaped to fit an isomorphic matrix with normalized width and height images For an image of shape (h, w), the isomorphic transformation matrix H homo is calculated as follows:

[0099]

[0100] After obtaining the isomorphic transformation, the transformation is applied to the key points and text coordinates detected in the previous stages and mapped to the corrected perspective plane for the final reading stage.

[0101] The process of obtaining the relationship between the polar angle and the scale reading is as follows:

[0102] The fixed limitations of the detection model plus the differences in image quality can lead to inaccurate detection in key point detection and OCR (including false positives and missed detections), which makes simple reading strategies unreliable. To solve this problem, the present invention proposes a more robust reading method. First, the OCR output is filtered based on confidence by combining the confidence scores of detection and recognition;

[0103] In order to more accurately determine the relative position of the scale mark and the digital text, the present invention establishes a polar coordinate system with the starting position of the pointer as the pole and the vertical direction downward as the polar axis, to calculate the polar angle of the scale mark, the end point of the pointer and the center of the OCR bounding box;

[0104] The relationship between the polar angle and the scale reading is established, and when the polar angle of the scale mark The polar angle of the center of the OCR bounding box When the following conditions are met, the two are matched:

[0105]

[0106] min(d ij ,(360-d ij ))≤d;

[0107] where d is a threshold value; when two or more matches are detected, record the corresponding relationship between the scale mark polar angle and the numerical value.

[0108] When the number of valid pairs obtained is less than two, this indicates that there can be a keypoint detection failure, and the system automatically switches back to the OCR-derived geometric cues. It is empirically observed that in a properly calibrated gauge, the angle of the scale markings is usually consistent with the angle of the OCR centroids.

[0109] The various embodiments described in this specification are presented by way of example, and each embodiment describes a specific implementation of the general principles described herein. The embodiments are not intended to limit or restrict the scope of the disclosure to the embodiments shown. The various embodiments set forth in the attached description are described in the context of the embodiments shown in the figures. However, the embodiments set forth in the attached description are not intended to limit or restrict the scope of the disclosure to the embodiments shown. Indeed, the embodiments set forth in the attached description are intended to encompass all embodiments set forth in the claims and their equivalents. Numerous specific details are set forth in the attached description to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that the embodiments described herein can be practiced without some or all of these specific details. In other instances, well known operations, components, and circuits have not been described in detail in order to avoid unnecessarily obscuring the embodiments. The specific

[0110] The above description of disclosed embodiments provides information sufficient to understand how to make and use the present application. Numerous modifications and adaptations will be apparent to those skilled in the art from this description. This description is provided as illustrative examples of the general principles of the application and the general principles are only intended to be illustrated and not to limit the scope of the application. The scope of the application should be determined from the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. A method for meter reading based on self-generated and annotated canonical benchmark test sets, characterized by, The application relates to a method for automatic reading of dial gauges, and belongs to the technical field of computer vision. The method comprises the following steps: Step 1: random sampling setting of measurement parameters and appearance parameters is carried out by using a Python script, and an instrument dataset is generated by using Blender; Step 2: a full-automatic standard annotation framework based on a Blender 3D model parameter is constructed, the instrument dataset is annotated, and a standard benchmark test set is obtained; Step 3: a hot map prediction model based on DINOv2 and a key point extraction algorithm are used for scale marking and pointer prediction, and in the key point extraction, a multi-stage extraction method is adopted for the intermediate scale marking which may contain an uncertain number of key points; 2. The method of claim 1, wherein, Step 4: after text detection and recognition, a regression network is used to directly regress a homography matrix required for perspective correction from an original image in an end-to-end mode to perform perspective correction, and a RANSAC algorithm is used to fit the relationship between an extreme angle and a gauge reading, and the value corresponding to the pointer extreme angle is predicted through the fitted model.

3. The method of claim 1, wherein, In step 1, the measurement parameters comprise a rotation angle of a pointer and a measurement range of an instrument.

4. The method of claim 1, wherein, In step 1, the appearance parameters comprise a color and a shape of the pointer, a length of a scale mark and a numerical distance between adjacent marks, a material and a corrosion condition of an instrument shell, an angle and a distance of a camera relative to the instrument, a style and a size of digital text and a radial position relative to a center, a texture and noise on a dial plate and diversity of an environment. In step 2, the full-automatic standard annotation framework based on the Blender 3D model parameter specifically comprises the following steps: Gauge detection and marking of a shell boundary box: all vertices of a gauge shell are traversed in Blender, and projection coordinates of the vertices in a camera image are calculated; the projection coordinates are converted into a two-dimensional coordinate system with an upper left corner as an origin, and maximum and minimum xy values of the projection coordinates are summarized to determine a boundary box of the gauge shell; Perspective correction: a corresponding relationship between two point sets is established, that is, positions of four equidistant points in a world coordinate system and corresponding points of the four equidistant points in camera projection; both the two coordinate sets are normalized relative to a detected gauge boundary box, and a perspective transformation matrix between the normalized coordinate systems is calculated by using a homography estimation method of OpenCV, so as to be used as a training target of a distortion correction model; Scale mark detection: key point detection and semantic segmentation; for the key point detection, points adjacent to a center of a gauge are marked at each main scale mark, and for the semantic segmentation, polygon marking is carried out according to a geometric shape of the scale mark; Light point detection: key point detection and semantic segmentation; for the key point detection, an intersection of a pointer rotation axis and a rotation axis surface is defined as a starting position, and an end point is determined by traversing all points on the pointer and selecting a point farthest from the starting position; for pointer segmentation marking, a mask-based marking method is adopted; Optical character recognition technology OCR: text positioning and text recognition are included, a boundary box is generated for all text objects, and text contents corresponding to the boundary box are associated for recognition.

5. The method of claim 1, wherein, In step 3, before scale marker and pointer prediction using the DINOv2-based heatmap prediction model and key point extraction algorithm, it also includes: using YOLOv11s to locate the instrument and separate the instrument from the complex background.

6. The method of claim 1, wherein, In step 3, scale marker and pointer prediction using the DINOv2-based heatmap prediction model and key point extraction algorithm, specifically: Five key point categories are selected, including: starting scale marker, ending scale marker, intermediate scale marker, starting position of pointer, and end point of pointer; A heatmap prediction model is used to predict a heatmap for each key point category; the heatmap prediction model is as follows: A frozen pre-trained DINOv2 is used as an encoder to extract features from the input image; a multi-stage upsampling network is used to generate a heatmap with the same size as the original image, and a BCE loss is used as the prediction loss; each upsampling stage consists of a convolutional layer and a transposed convolutional layer, which performs 2x upsampling, and the final heatmap is generated through a convolutional layer; Key points are extracted from each heatmap for each key point category; for intermediate scale markers, each heatmap corresponds to a key point, and the mean shift algorithm is used for extraction.

7. The method of claim 1, wherein, In step 3, for intermediate scale markers that may contain an uncertain number of key points, a multi-stage extraction method is used, specifically: Significant region filtering: set a response threshold c, and only keep candidate regions with heatmap response H(x, y) ≥ c; Local maximum value extraction: use a k × k sliding window to perform maximum value filtering to detect local maximum values that satisfy the following conditions: Non-maximum suppression: use a greedy algorithm based on spatial distance constraints to select key points in descending order of confidence, and remove low-confidence candidate points within a radius r of the selected key points; Sub-pixel refinement: use quadratic interpolation to adjust integer pixel coordinates, and estimate the peak shift through Taylor expansion: The key point coordinates (x, y) are optimized to (x + Δx, y + Δy), where |Δx| and |Δy| are both less than or equal to 0.

5.

8. The method of claim 1, wherein, In step 4, YOLOv11s is used for text detection, and ABINet is used for text recognition.

9. The method of claim 1, wherein, In step 4, a regression network is used to directly regress the homography matrix required for perspective correction from the original image to perform perspective correction, specifically: A deep learning model is trained using homography annotations in synthetic data to predict homography transformations; The homography transformation is represented with eight degrees of freedom, the model predicts a vector h e R 1×8 ; using a pre-trained DINO backbone network as an encoder to extract features from the input image, and predicting the final homography transformation through a linear layer, using an L1 loss function for training; During the inference process, a 1 is added at the end of the prediction vector h and reshaped to fit into an isomorphic matrix with normalized width and height images For an image of shape (h, w), the isomorphic transformation matrix H homo Is computed in the following way: After obtaining the homography transformation, it is applied to the key points and text coordinates detected in the previous stages and mapped to the corrected perspective plane.

10. The method of claim 1, wherein, In step 4, the relationship between the polar angle and the gauge reading is obtained as follows: By combining the confidence scores of detection and recognition, the OCR output is filtered based on confidence; A polar coordinate system is established with the starting position of the pointer as the pole and the vertical direction downward as the polar axis to calculate the polar angles of the scale markers, the end point of the pointer, and the center of the OCR bounding box. Establish the relationship between the polar angle and the gauge reading, when the polar angle of the scale mark Polar angle to the center of the OCR bounding box Match the two when the following conditions are met: min(d ij ,(360-d ij ))≤d; Where d is a threshold value; when two or more matches are detected, record the corresponding relationship between the scale marker polar angles and the numerical values.

Images