A multi-type industrial dial real-time segmentation and geometric reading method

Abstract

The present application relates to the technical field of image processing and recognition of industrial dials, and specifically discloses a real-time segmentation and geometric reading method for multiple types of industrial dials, S1, image data acquisition and preprocessing, obtaining a standardized image; S2, inputting the standardized image into an extended YOLOv8n double-task joint inference network for inference, obtaining an instance segmentation mask of multiple instances including a dial main body, a pointer, a dial center origin, and a zero scale reference line, and a coordinate and confidence triple of a key point predicted for each instance; S3, evaluating the shielding state of the pointer instance and adaptively selecting a fusion path, obtaining a final pointer tip coordinate; S4, performing distortion-free geometric vectorization and continuous reading calculation; the method adopts a "key point + segmentation" dual-path complementary strategy to solve the engineering problems of inaccurate reading, unreadable reading, and non-continuous reading of the dial in a shielding scene.

Description

Technical Field

[0001] This invention relates to the field of image processing and reading recognition technology for industrial dials. Background Technology

[0002] With the deepening development of industrial automation, traditional pointer-type instruments are still widely used in fields such as petrochemicals, power, and water utilities. Industrial sites are characterized by dense and complex piping systems, often obstructing instruments with pipes, supports, flanges, etc., or causing partial visibility of the dial due to oil or sewage mist, severely impacting the reliability of automatic readings. Existing automatic reading technologies lack dedicated processing mechanisms for obstructed scenarios, and can be mainly categorized into three types:

[0003] The first category is traditional image processing and feature matching methods. These methods rely on manual operators such as Hough transform and edge detection, for example, using Hough circle transform to locate the dial and Hough line detection to extract the pointer. These methods require complete circular contours and continuous edges. When the occlusion rate exceeds 20%, the center localization drops sharply, edge breakage leads to screening failure, and there is no overall occlusion tolerance, resulting in extremely low accuracy under occlusion.

[0004] The second category is deep learning methods that combine object detection with geometric unfolding. First, YOLO or similar detection methods are used to detect the instrument area, and then a scale-reading mapping is established through polar coordinate unfolding or perspective transformation. However, both polar coordinate unfolding and perspective correction rely on the complete dial area. When occlusion causes the dial to be incomplete or the pointer / scale to be obscured, the mapping becomes distorted or the reading fails, and there is a lack of occlusion adaptation mechanisms.

[0005] The third category is semantic / instance segmentation methods. These methods use DeepLabV3, U-Net, and other techniques to segment pointer and tick pixel regions, calculating readings through skeletonization or angle calculation. While these segmentation methods have advantages at the pixel level, occlusion fragments the pointer mask, and the skeleton orientation deviates significantly. Semantic segmentation cannot distinguish between multiple pointers, leading to instance confusion. Furthermore, all methods rely on a single path with no redundancy.

[0006] The main drawbacks of existing technologies are: lack of a dedicated occlusion handling mechanism; single-path failure under occlusion; no degradation or fault tolerance; traditional methods become essentially ineffective when occlusion exceeds 20%; and segmentation skeleton errors cause reading deviations exceeding ±5% of full scale. They also lack geometric constraints on key points, outputting only region masks during segmentation, failing to provide precise coordinates and confidence levels for the pointer tip; and severe interference from debris to the centroid or skeleton endpoints under occlusion, with no geometric repair methods. Furthermore, single-path lacks adaptive redundancy, preventing the system from switching to backup schemes under occlusion, leading to reading interruptions or invalidity, and failing to meet the 24 / 7 continuous high-reliability monitoring requirements of industrial sites. Summary of the Invention

[0007] The purpose of this invention is to provide a method for real-time segmentation and geometric reading of various types of industrial dials. This method adopts a dual-path complementary strategy of "key point + segmentation" to solve the engineering problems of inaccurate, unreadable, and uncontinuous dial reading in occluded scenarios.

[0008] To achieve the above objectives, the technical solution of the present invention is: a method for real-time segmentation and geometric reading of multiple types of industrial dials, characterized by comprising the following method steps:

[0009] S1. Image data acquisition and preprocessing to obtain standardized images;

[0010] S2. Input the standardized image into the extended YOLOv8n dual-task joint inference network for inference, and obtain instance segmentation masks for multiple types of instances, including the main body of the dial, the pointer, the center origin of the dial, and the zero scale baseline, as well as the coordinates and confidence triples of the key points predicted for each instance.

[0011] S3. Evaluate the occlusion state of the pointer instance and adaptively select the fusion path to obtain the final pointer tip coordinates;

[0012] S4. Perform distortion-free geometric vectorization and continuous reading calculation.

[0013] In step S2, the extended YOLOv8n dual-task joint inference network is obtained by extending and improving YOLOv8n-seg. The standardized image is extracted by the backbone feature extraction network in the extended YOLOv8n dual-task joint inference network and outputs multiple multi-scale feature maps. Then, it is fused by the neck feature fusion network in the extended YOLOv8n dual-task joint inference network to obtain multiple enhanced scale feature maps. The extended YOLOv8n dual-task joint inference network also includes parallel prediction head branches, namely, the instance segmentation prediction head branch for obtaining multiple types of instances and the keypoint regression prediction head branch for obtaining the keypoint coordinates and confidence triples for each instance.

[0014] In step S3, two occlusion state quantification indicators are first calculated for each pointer instance: key point confidence and segmentation mask integrity. Then, the evaluation is carried out through three-mode adaptive path selection. The three modes are: the key point main path in high confidence mode, the segmentation skeletonization backup path in low confidence complete mask mode, and the weighted fusion path in medium occlusion mode.

[0015] In step S4, the center coordinates of the dial are extracted and the zero-scale reference angle is obtained first. Then, the geometric angle is calculated and the physical quantity is mapped. Specifically, this includes the calculation of the absolute deflection angle and relative deflection angle of the pointer, the linear mapping of physical quantities, and the parallel calculation of multiple pointers.

[0016] The instance segmentation prediction head is equipped with three parallel independent branches for each scale: an object classification branch, a bounding box regression branch, and a mask coefficient branch. The object classification branch predicts the semantic category of each instance; the bounding box regression branch predicts the bounding box coordinates of each instance; and the mask coefficient branch predicts k prototype masks for each instance. The network outputs k prototype masks and mask coefficients for each object at the neck of the network. The mask coefficients are linearly combined with the k prototype masks, and after Sigmoid activation, the final instance segmentation mask for that instance is generated. The formula for generating the instance segmentation mask is: In the formula, For the first A prototype mask, For the first The mask coefficients of each target This is the Sigmoid activation function; the output instance of this branch includes: the dial body, the hands, the center point of the dial, and the zero mark baseline.

[0017] The keypoint regression prediction head predicts the coordinates and confidence triples of multiple keypoints for each instance. The multiple keypoints correspond to: the center of the dial, the tip of the pointer, the zero mark, and the full mark. Each keypoint outputs a triple consisting of x and y coordinates and a confidence score, where the confidence score is obtained through supervised learning of occlusion visibility annotations during training.

[0018] The extended YOLOv8n dual-task joint inference network uses a hybrid loss function for model training: ,in For bounding box loss, For the distribution focus loss, For classifying losses, For mask loss, For keypoint similarity loss, For keypoint regression loss based on target keypoint similarity, , , , , All were determined through a grid search of the validation set. The expression is:

[0019]

[0020] in Predict the Euclidean distance between the i-th keypoint and the ground truth keypoint. The target scale factor, Let be the normalization factor for the i-th key point. This is the visibility flag for the i-th keypoint. This represents the total number of visible keypoints.

[0021] The keypoint main path is set when the keypoint confidence level is... When the keypoint regression prediction head outputs the tip coordinates directly, the final pointer tip coordinates are used; the segmentation skeletonization backup path is set when the keypoint confidence level is... And the integrity of the segmentation mask At that time, a morphological opening operation is performed on the segmentation mask to smooth the boundary, and then a thinning skeletonization algorithm is performed to extract the single-pixel width main axis skeleton. The skeleton endpoints are traversed, and the Euclidean distance from each endpoint to the center of the circle is calculated. The skeleton endpoint with the largest Euclidean distance is selected as the final pointer tip coordinate. The weighted fusion path is set so that when the confidence of the key point is zero, the weights of the main path of the key point and the backup path of the segmentation skeletonization are calculated by normalizing the confidence. Then, the coordinates of the main path of the key point and the coordinates of the backup path of the segmentation skeletonization are weighted and fused to obtain the final pointer tip coordinate.

[0022] The extraction of the dial center coordinates specifically involves: when the confidence level of the key point at the center... When the dial center coordinates are extracted, the coordinates of the key point of the dial center are used; when the key point of the center is occluded, the process reverts to a weighted fusion of minimum circumcircle fitting and spatial moment centroid calculation on the mask contour of the instrument dial body; the zero-scale reference angle is obtained by: calculating the reference direction angle using the coordinates of the zero-scale mark key point; when the zero-scale mark key point is occluded, the calibration value recorded in advance is used to complete the calculation.

[0023] The absolute deflection angle of the pointer is calculated using the atan2 function; the relative deflection angle is... Then add The linear mapping expression for the physical quantity is:

[0024]

[0025] In the formula, The total angular span at full scale is entered through system initialization calibration, and the calibration constant is set accordingly. This represents the change in angle corresponding to each unit of physical quantity. This is the relative deflection angle;

[0026] The multi-pointer parallel computing refers to the system performing occlusion fusion and angle calculation in parallel for multiple pointer instances on the same dial, and independently outputting the physical readings of each pointer.

[0027] The formula for generating the instance segmentation mask is: In the formula, For the first A prototype mask, For the first The mask coefficients of each target Use the Sigmoid activation function;

[0028] The expression for the integrity of the segmentation mask is: ,in To predict the area of ​​mask pixels, To determine based on dial radius The expected area estimated by the aspect ratio of a typical pointer. ,coefficient Obtained through statistics from the training set;

[0029] The Euclidean distance from each endpoint to the center of the circle is calculated using the following expression:

[0030] In the formula, Let C be the i-th skeleton endpoint and C be the center of the circle. , Let be the coordinates of the i-th skeleton endpoint. , The coordinates of the center of the circle;

[0031] The weighted fusion of the main path coordinates of the key points and the alternative path coordinates of the segmented skeleton is expressed as follows:

[0032]

[0033]

[0034] In the formula, The fusion weights for the main path of key points, The confidence level of the key point at the tip of the pointer. To ensure the integrity of the segmentation mask, To segment the skeletonized backup path fusion weights, , The final weighted and merged pointer tip coordinates. , The coordinates of the pointer tip output by the main path of the key point. , The pointer tip coordinates are output for the segmented skeletonized backup path. In step S1, image data acquisition uses an HDR industrial camera, outputting the raw image stream in Raw Bayer format. The image preprocessing method is as follows: First, a de-mosaic operation is performed using a bilinear interpolation algorithm to convert the Raw Bayer format into a BGR three-channel color image; then, CLAHE brightness enhancement processing is performed on the L channel of the LAB color space, the image is divided according to a set grid, the Clip Limit constraint factor is set to 2.0, and a smooth transition between grids is achieved through bicubic interpolation; finally, the image is scaled proportionally according to a set number of pixels, and the short sides are filled with gray values ​​to complete the pixel value normalization processing.

[0035] The HDR industrial camera features a CMOS image sensor with a dynamic range of no less than 120dB, a resolution of no less than 1920×1200, and a frame rate of no less than 30 FPS in conventional monitoring scenarios and no less than 60 FPS in high-speed mobile inspection scenarios; and / or, the HDR industrial camera is equipped with an LED supplementary lighting module with a color rendering index (CRI) > 95 and a wide illumination range covering 50-50000 lux; and / or, during installation, the angle between the optical axis and the normal to the dial plane is controlled within the range of 0-15°; and / or, the HDR industrial camera synchronously records metadata such as timestamps and camera numbers during the acquisition process.

[0036] By adopting the above technical solution, the beneficial effects of this invention are as follows: Addressing the industry problem of low reading reliability of pointer instruments in complex scenarios such as pipe obstruction, bracket obstruction, and oil / sewage mist coverage, this invention proposes a real-time segmentation and geometric reading method for multiple types of industrial dials based on the fusion of key point detection and instance segmentation. Compared with existing technologies, this invention has the following beneficial effects:

[0037] (I) Significantly Improved Robustness of Readings in Occlusion Scenarios. This invention constructs a dual-path complementary occlusion adaptive fusion mechanism of "keypoint detection + instance segmentation," using keypoint confidence and segmentation mask integrity as dual indicators to drive the three-mode adaptive path switching (keypoint main path, segmentation skeletonization backup path, and weighted fusion path), transforming the two reading technologies from a "serial competition" relationship to a "parallel complementarity" relationship. Through this dual-path redundancy mechanism, this invention fundamentally solves the industry-wide problem of reading failure in industrial occlusion scenarios, achieving an order-of-magnitude improvement in reading reliability under occlusion conditions.

[0038] (II) Achieving Synchronous and Accurate Readings from Multiple Types of Multi-Pointer Instruments. This invention is based on a dual-task joint inference network extended with YOLOv8n, which synchronously outputs six types of instance segmentation masks and five types of keypoint coordinates with confidence levels (dial center, main pointer tip, secondary pointer tip, zero mark, and full-scale mark) in a single forward propagation. Keypoint detection can still provide independent coordinates and confidence levels for each pointer tip even in extreme scenarios where the two pointers are completely overlapping. A single training model is compatible with more than 54% of heterogeneous instrument types in industrial settings (covering single-pointer, dual-pointer, and triple-pointer instruments, black-and-white and color dials, and different measurement units), eliminating the need to develop separate algorithms for each type of instrument, effectively reducing system complexity and deployment costs in mixed monitoring scenarios with multiple instrument types.

[0039] (III) Ensuring industrial-grade accuracy and real-time performance. This invention employs a distortion-free geometric pipeline based on "instance mask + key points," completely eliminating resampling errors introduced by polar coordinate unfolding and perspective transformation, thus meeting the requirements for high-precision readings in industrial applications.

[0040] The technical solution of this invention is applicable to the automation of readings in obstructed scenarios of all types of analog pointer instruments, and has broad market application prospects and intellectual property protection value. Attached Figure Description

[0041] Figure 1 This invention relates to a schematic diagram of the system structure of a method for real-time segmentation and geometric reading of multiple types of industrial dials.

[0042] Figure 2 This invention relates to a system algorithm flowchart for a real-time segmentation and geometric reading method for multiple types of industrial dials.

[0043] Figure 3 This is a schematic diagram of the structure of the extended YOLOv8n dual-task joint inference network in a real-time segmentation and geometric reading method for multiple types of industrial dials, which is a method of the present invention.

[0044] Figure 4 This invention relates to a flowchart of the occlusion adaptive fusion reading path selection process in a real-time segmentation and geometric reading method for multiple types of industrial dials. Detailed Implementation

[0045] To further explain the technical solution of the present invention, the present invention will be described in detail below through specific embodiments.

[0046] This embodiment discloses a real-time segmentation and geometric reading method for multiple types of industrial dials, mainly comprising a front-end acquisition hardware layer and a back-end algorithm processing layer. The front-end acquisition hardware layer realizes high dynamic range image acquisition in occluded scenes, while the back-end algorithm processing layer realizes preprocessing, dual-task joint inference, occlusion adaptive fusion, geometric calculation, and reading output. The following describes in detail the real-time segmentation and geometric reading method for multiple types of industrial dials of this invention, such as... Figure 1 and Figure 2 As shown, the method steps are as follows:

[0047] S1. Image data acquisition and preprocessing.

[0048] In this implementation, the image acquisition equipment at the front-end hardware acquisition layer preferably uses an HDR industrial camera. HDR (High Dynamic Range) imaging technology is a visual sensing technology in machine vision that can simultaneously and clearly capture both strongly reflective areas in an industrial environment and details in the dark areas of a dial. Its dynamic range is typically over 120dB, far exceeding the 60-70dB of traditional industrial cameras, effectively avoiding pixel truncation distortion caused by mirror reflection from glass protective covers. HDR industrial cameras are usually mounted on industrial inspection robots or fixedly installed next to equipment cabinets, and are also equipped with LED supplementary lighting modules.

[0049] The experiments in this embodiment show that the HDR industrial camera uses a CMOS image sensor with a dynamic range of no less than 120dB (LUCID Triton TRI120S-C or equivalent is recommended), outputs raw image streams in Raw Bayer format, has a resolution of no less than 1920×1200, and a frame rate of no less than 30 FPS in conventional monitoring scenarios and no less than 60 FPS in high-speed mobile inspection scenarios. Simultaneously, it is paired with an LED fill light module with a color rendering index (CRI) > 95 to cover a wide illumination range of 50-50000 lux, ensuring clear capture of global dial details even in industrial environments with strong reflections and dark areas. Furthermore, preferably, during the installation of the HDR industrial camera, the angle between the camera's optical axis and the normal to the instrument dial plane is controlled within the range of 0-15° to minimize perspective geometric distortion. During the acquisition process, the HDR industrial camera can simultaneously record metadata such as timestamps and camera numbers, providing high-fidelity raw image input for subsequent preprocessing and geometric calculations.

[0050] When in use, the HDR industrial camera configured as described above is used to acquire image data and obtain raw HDR images in Raw Bayer format.

[0051] After obtaining the original HDR image, it is preprocessed. Specifically, in this embodiment, firstly, a de-mosaic operation is performed using a bilinear interpolation algorithm to convert the Raw Bayer format into a BGR three-channel color image; then, CLAHE (Contrast Limited Adaptive Histogram Equalization) brightness enhancement processing is performed on the L channel of the LAB color space, the image is divided into an 8×8 grid, the Clip Limit factor is set to 2.0, and bicubic interpolation is used to achieve a smooth transition between grids to effectively brighten the pointer outline edges in the shadow area; finally, the image is scaled proportionally to 640×640 pixels, the short sides are filled with a grayscale value of 114, and the pixel values ​​are normalized to the [0,1] range, outputting a normalized floating-point tensor (640×640×3), i.e., the adaptive preprocessing yields a normalized image, which can be directly used by the subsequent dual-task joint inference network.

[0052] S2. Perform inference using the extended YOLOv8n dual-task joint inference network.

[0053] In this embodiment, the extended YOLOv8n dual-task joint inference network is obtained by extending and improving upon YOLOv8n-seg. A standardized image (640×640×3) is input into the extended YOLOv8n dual-task joint inference network, where joint single-step inference of segmentation and keypoints is achieved. The detailed network structure is as follows... Figure 3As shown, in the backbone feature extraction network, the normalized image (640×640×3) undergoes multi-level downsampling in the network's convolutional modules, progressively extracting semantic features at different scales. The backbone feature extraction network outputs three effective feature layers, corresponding to feature representations at different spatial resolutions (scale from large to small, semantic level from low to high), for subsequent multi-scale fusion by the neck feature fusion network. In the neck feature fusion network, a feature pyramid-style bidirectional fusion operation (top-down and bottom-up path aggregation) is performed on the multi-scale feature maps output by the backbone feature extraction network, fully fusing high-level semantic information with low-level spatial detail information to generate three enhanced multi-scale feature maps. These feature maps maintain the spatial scale corresponding to the backbone output and are used to detect target instances of different sizes (large, medium, and small targets), ensuring that the network has good perception capabilities for both large dial subjects and slender hands. The three enhanced feature maps of the neck are fed into two parallel prediction head branches: the instance segmentation prediction head branch and the keypoint regression prediction head branch. The two branches share the same set of multi-scale features extracted from the main trunk and the neck, but each performs different regression and classification tasks independently.

[0054] Specifically, in this embodiment, the instance segmentation prediction head is equipped with three parallel independent branches for each scale, responsible for the target classification branch, bounding box regression branch, and mask coefficient branch, respectively. The target classification branch is used to predict the semantic category of each instance; the bounding box regression branch is used to predict the coordinates of the detection box for each instance; and the mask coefficient branch is used to predict k (k=32 in this embodiment) prototype masks for each instance. The network outputs k prototype masks (MaskPrototypes) and mask coefficients for each target at the neck. The mask coefficients are linearly combined with these k prototype masks, and after Sigmoid activation, the final instance segmentation mask for that instance is generated. The formula for generating the instance segmentation mask is: In the formula, For the first A prototype mask, For the first The mask coefficients of each target This is the Sigmoid activation function. In this embodiment, this branch outputs a total of 6 semantic instances, including: the dial body, pointers (main pointer, secondary pointer red, secondary pointer green), the dial center origin, and the zero scale baseline.

[0055] Specifically, in this embodiment, the keypoint regression prediction head predicts the coordinates and confidence triples of multiple keypoints for each detection instance. In this embodiment, 5 keypoints are predicted, and the expression is as follows: Where i is the i-th point, Let x and y be the coordinates of the i-th point. Let i be the confidence level of the i-th point. The five key points correspond to: the center of the dial circle, etc. The tip of the main pointer , tip of secondary pointer Zero mark Full scale mark Each keypoint outputs a (x, y, conf) triple, where the confidence score is obtained through supervised learning of occlusion visibility labels (0 = occluded and invisible, 1 = fully visible) during training, and is used to quantitatively evaluate the visibility and occlusion degree of the keypoint.

[0056] Through the above process, the network simultaneously outputs two types of prediction results in a single forward propagation: the first type is 6 types of instance segmentation masks (each type contains its own independent pixel-level mask); the second type is the coordinates and confidence of 5 types of key points for each instance. The two outputs share the same backbone and neck features, realizing a joint inference architecture of "one inference, dual task output". The number of parameters is about 4.1M, and the inference latency is about 42ms (≥24 FPS) when deployed on edge computing devices.

[0057] In this invention, the model training of the extended YOLOv8n dual-task joint inference network adopts a hybrid loss function: ,in, , , , , All were determined through a grid search of the validation set. (Bounding box loss) and (Distributed Focus Loss) constrains the "boundary box regression branch" in the instance segmentation prediction head to ensure accurate detection box localization. (Classification Loss): Constrain the "target classification branch" in the instance segmentation prediction head to ensure that the network can correctly identify six semantic instances, such as "main pointer" and "secondary pointer". (Mask Loss): The combination of the “mask coefficient branch” in the constrained instance segmentation prediction head with the prototype mask ensures that the generated pixel-level mask fits the real pointer and / or dial shape. (Keypoint Similarity Loss): This is a loss function specifically designed for the newly added keypoint regression prediction head. It directly supervises the coordinate regression accuracy of the five keypoints (center, tip, etc.) to ensure that their output positions are as close as possible to the true coordinates. The keypoint regression loss based on target keypoint similarity is expressed as follows:

[0058]

[0059] in Predict the Euclidean distance between the i-th keypoint and the ground truth keypoint. The target scale factor, Let be the normalization factor for the i-th key point. The visibility flag for the i-th keypoint. This represents the total number of visible keypoints.

[0060] S3. Evaluation of pointer instance occlusion state and adaptive fusion path selection.

[0061] In this embodiment, two occlusion state quantification indicators are calculated for each pointer instance. The detailed process is as follows: Figure 4 As shown, they are:

[0062] Indicator 1: Confidence of Key Points The keypoint regression prediction head outputs directly, reflecting the visibility of the pointer tip.

[0063] Indicator 2: Segmentation Mask Integrity Its expression is ,in To predict the area of ​​mask pixels, To determine based on dial radius The expected area estimated by the aspect ratio of a typical pointer. ,coefficient The value is obtained through statistics from the training set (e.g., 0.12 by default in this example experiment).

[0064] The adaptive path selection was evaluated using the following three modes:

[0065] Path 1: Key point main path, this path is in high confidence mode. (0.7 is the preset value in this embodiment), directly using the tip coordinates output by the keypoint regression prediction head. This serves as the final pointer tip coordinate. The path operation is simple; the system does not need to perform any morphological processing on the segmentation mask, nor does it need to calculate complex weighted fusion weights. In the example experiment, when the system determines that it has entered the "key point main path," the post-processing computation of this step is about 1 ms, compared to the already completed dual-task network inference (about 42 ms).

[0066] Path 2: Segmentation and skeletonization backup path, this path is a low-confidence complete mask mode. (0.4 is the preset value in this embodiment) and (0.6 is the preset value in this embodiment), perform morphological opening operation (3×3 rectangular structuring element) on the segmentation mask to smooth the boundary, and then perform a thinning skeletonization algorithm to extract the single-pixel width main axis skeleton; traverse the skeleton endpoints and calculate the Euclidean distance from each endpoint to the center of the circle: In the formula, Let C be the i-th skeleton endpoint and C be the center of the circle. , Let be the coordinates of the i-th skeleton endpoint. , The coordinates of the center of the circle are used. The endpoint of the skeleton with the largest Euclidean distance is selected as the final pointer tip coordinates. .

[0067] Path 3: Weighted fusion path, this path is a moderate occlusion mode. For the remaining cases, after calculating the weights of the main path and the segmented skeletonized alternative path using confidence normalization, the coordinates of the main path and the coordinates of the segmented skeletonized alternative path are weighted and fused, as shown in the following expression:

[0068]

[0069]

[0070] In the formula, The fusion weights for the main path of key points, The confidence level of the key point at the tip of the pointer. To ensure the integrity of the segmentation mask, To segment the skeletonized backup path fusion weights, , The final weighted and merged pointer tip coordinates. , The coordinates of the pointer tip output by the main path of the key point. , To segment the skeletonized alternative path, output the pointer tip coordinates, and output the final pointer tip coordinates. , ).

[0071] S4. Perform distortion-free geometric vectorization and continuous reading calculation.

[0072] First, the dial center coordinates are extracted and the zero-scale reference angle is obtained. Specifically, the dial center coordinate extraction involves determining the confidence level of the key center point. When 0.6 is the preset value in this embodiment, the dial center coordinates are extracted using the dial center key point. coordinates When the center keypoint is occluded, the process regresses to a weighted fusion (with weights of 0.3 and 0.7 respectively) calculated by performing minimum circumcircle fitting (using the dial's "circular outline" to infer the center) and spatial moment centroid (using the dial's "geometric centroid of all pixels" to infer the center) on the main mask contour of the instrument panel. The zero-scale reference angle is obtained by marking keypoints at the zero scale. The coordinate calculation reference direction angle: ;when When obstructed, use pre-calibrated records Complete the calibration values.

[0073] Then, geometric angle calculation and physical quantity mapping are performed, specifically including absolute pointer deflection angle, relative deflection angle calculation, linear mapping of physical quantities, and parallel calculation of multiple pointers.

[0074] The expression for the absolute deflection angle of the pointer is: The atan2 function returns the complete angle values ​​in all four quadrants (range). to This avoids the symbolic ambiguity of the ordinary arctan function.

[0075] The formula for calculating the relative deflection angle is: ,right Make direction corrections if Then add ,make sure .

[0076] The linear mapping expression for the physical quantity is:

[0077]

[0078] In the formula, The total angular span at full scale is entered through system initialization calibration; calibration constants. It represents the change in angle corresponding to each unit of physical quantity.

[0079] The multi-pointer parallel computing involves multiple pointer instances on the same dial. The system executes the occlusion fusion and angle calculation steps S3-S4 in parallel, and outputs the physical readings of each pointer independently, realizing synchronous monitoring of multi-range instruments. Parallel computing does not increase the overall latency.

[0080] Obtain the final reading on the instrument panel.

[0081] The technical solution of this invention addresses the industry challenge of low reading reliability of pointer instruments in complex scenarios such as pipe obstruction, bracket obstruction, and oil and sewage mist coverage in industrial settings. It proposes a real-time segmentation and geometric reading method for multiple types of industrial dials based on deep fusion of key point detection and instance segmentation. This method constructs a dual-path complementary occlusion adaptive fusion mechanism of "key point detection + instance segmentation." In experiments, the reading accuracy reaches 67.8% in a 30% occlusion scenario and 54.3% in a 50% occlusion scenario, representing an improvement of approximately 9.8 percentage points compared to some existing technologies. This invention transforms the two reading technologies from a "serial competition" relationship to a "parallel complementary" relationship through confidence-driven three-mode adaptive path switching, fundamentally solving the persistent industry problem of reading failure in industrial occlusion scenarios and achieving an order-of-magnitude improvement in reading reliability under occlusion conditions.

[0082] Regarding the synchronous and accurate reading of multiple types of multi-pointers, this invention is based on a dual-task joint inference network extended with YOLOv8n. In a single forward propagation, it simultaneously outputs six types of instance segmentation masks and five types of keypoint coordinates with confidence levels. Instance segmentation reduces the multi-pointer confusion rate to below 1.5% (compared to 100% for existing semantic segmentation schemes and 8.3% for two-stage schemes). Keypoint detection can still provide independent coordinates and confidence levels for each pointer tip even in extreme scenarios where the two pointers completely overlap. A single training model is compatible with over 54% of heterogeneous instrument types in industrial settings (covering single-pointer, dual-pointer, and triple-pointer instruments, black-and-white and color dials, and different measurement units), eliminating the need for separate algorithm development for each instrument type and significantly reducing system complexity and deployment costs in mixed monitoring scenarios with multiple instrument types.

[0083] In terms of overall accuracy and real-time performance, the distortion-free geometric pipeline of "instance mask + key points" adopted in this embodiment completely eliminates the resampling errors introduced by polar coordinate unfolding and perspective transformation. The sub-pixel positioning error is only 0.6 pixels, and the full-scale reading error is controlled within ±0.5%, meeting the requirements of industrial-grade high-precision reading. On the NVIDIA Jetson Xavier NX edge computing device, the end-to-end processing latency is approximately 42ms (≥24 FPS), and the high-performance GPU inference speed can reach over 130 FPS, fully meeting the needs of real-time continuous monitoring in industrial settings. The model fully supports ONNX (approximately 1.5x faster inference speed) and TensorRT FP16 (approximately 2.1x faster, with less than 0.1% accuracy loss) inference acceleration engines, providing flexible deployment options for edge devices with different computing power levels.

[0084] The three core innovations of this invention clearly differentiate it from existing solutions: First, the "YOLOv8n segmentation + keypoint dual-task extended network" architecture achieves feature sharing and collaborative optimization through dual-head joint training; second, the "three-mode occlusion adaptive fusion mechanism driven by keypoint confidence and mask integrity dual indicators" provides an adaptive redundant reading path for industrial occlusion scenarios, unlike existing single-path, non-redundant designs; third, the "keypoint geometric constraint-assisted skeleton fracture repair method" uses the predicted tip coordinates of keypoints to anchor the skeleton extrapolation direction, controlling the skeletonization direction extraction deviation angle within the reading error range. The above methods are applicable to the automation of readings in occlusion scenarios for all types of analog pointer instruments, possessing broad market application prospects and technological value.

[0085] The above embodiments and figures are not intended to limit the product form and style of the present invention. Any appropriate changes or modifications made by those skilled in the art should be considered as not departing from the patent scope of the present invention.

Claims

1. A method for real-time segmentation and geometric reading of multiple types of industrial dials, characterized in that, The methods and steps include the following: S1. Image data acquisition and preprocessing to obtain standardized images; S2. Input the standardized image into the extended YOLOv8n dual-task joint inference network for inference, and obtain instance segmentation masks for multiple types of instances, including the main body of the dial, the pointer, the center origin of the dial, and the zero scale baseline, as well as the coordinates and confidence triples of the key points predicted for each instance. S3. Evaluate the occlusion state of the pointer instance and adaptively select the fusion path to obtain the final pointer tip coordinates; S4. Perform distortion-free geometric vectorization and continuous reading calculation; In step S2, the extended YOLOv8n dual-task joint inference network is obtained by extending and improving YOLOv8n-seg. The standardized image is extracted by the backbone feature extraction network in the extended YOLOv8n dual-task joint inference network and outputs multiple multi-scale feature maps. Then, it is fused by the neck feature fusion network in the extended YOLOv8n dual-task joint inference network to obtain multiple enhanced scale feature maps. The extended YOLOv8n dual-task joint inference network also includes parallel prediction head branches, namely, the instance segmentation prediction head branch for obtaining multiple types of instances and the keypoint regression prediction head branch for obtaining the keypoint coordinates and confidence triplet for each instance. In step S3, two occlusion state quantification indicators are first calculated for each pointer instance, namely key point confidence and segmentation mask integrity. Then, the evaluation is carried out through three-mode adaptive path selection. The three modes are: key point main path in high confidence mode, segmentation skeletonization backup path in low confidence complete mask mode, and weighted fusion path in medium occlusion mode. In step S4, the center coordinates of the dial are extracted and the zero-scale reference angle is obtained first. Then, the geometric angle is calculated and the physical quantity is mapped. Specifically, this includes the calculation of the absolute deflection angle and relative deflection angle of the pointer, the linear mapping of physical quantities, and the parallel calculation of multiple pointers. The extended YOLOv8n dual-task joint inference network uses a hybrid loss function for model training: ,in For bounding box loss, For the distribution focus loss, For classifying losses, For mask loss, For keypoint similarity loss, For keypoint regression loss based on target keypoint similarity, , , , , All were determined through a grid search of the validation set. The expression is: in Predict the Euclidean distance between the i-th keypoint and the ground truth keypoint. The target scale factor, Let be the normalization factor for the i-th key point. This is the visibility flag for the i-th keypoint. This represents the total number of visible keypoints. The keypoint main path is set when the keypoint confidence level is... When using the preset values, the coordinates of the tip of the keypoint regression prediction head are directly used as the final pointer tip coordinates; The segmentation skeletonization backup path is set when the confidence level of the key point is... Preset value and segmentation mask integrity When the preset value is used, a morphological opening operation is performed on the segmentation mask to smooth the boundary, and then a thinning skeletonization algorithm is performed to extract the single-pixel width main axis skeleton. The skeleton endpoints are traversed, the Euclidean distance from each endpoint to the center of the circle is calculated, and the skeleton endpoint with the largest Euclidean distance is selected as the final pointer tip coordinate. The weighted fusion path is set such that when the confidence level of the key point is in other cases, the weights of the main path of the key point and the segmented skeletonized backup path are calculated by normalizing the confidence level, and then the coordinates of the main path of the key point and the coordinates of the segmented skeletonized backup path are weighted and fused to obtain the final pointer tip coordinates.

2. The method for real-time segmentation and geometric reading of multiple types of industrial dials as described in claim 1, characterized in that, The instance segmentation prediction head is equipped with three parallel independent branches for each scale, which are responsible for the target classification branch, the bounding box regression branch, and the mask coefficient branch, respectively. The target classification branch is used to predict the semantic category to which each instance belongs; the bounding box regression branch is used to predict the coordinates of the detection box for each instance. mask The coefficient branch is used to predict k prototype masks for each instance; the network outputs k prototype masks and mask coefficients for each target at the neck. The mask coefficients are linearly combined with the k prototype masks, and after Sigmoid activation, the final instance segmentation mask for that instance is generated. The formula for generating the instance segmentation mask is: In the formula, For the first A prototype mask, For the first The mask coefficients of each target This is the Sigmoid activation function; the output instances of this branch include: the dial body, the hands, the center point of the dial, and the zero mark baseline.

3. The method for real-time segmentation and geometric reading of multiple types of industrial dials as described in claim 2, characterized in that, The keypoint regression prediction head predicts the coordinates and confidence triples of multiple keypoints for each instance. The multiple keypoints correspond to: the center of the dial, the tip of the pointer, the zero mark, and the full mark. Each keypoint outputs a triple consisting of x and y coordinates and a confidence score, where the confidence score is obtained through supervised learning of occlusion visibility annotations during training.

4. A method for real-time segmentation and geometric reading of multiple types of industrial dials as described in any one of claims 1-3, characterized in that, The extraction of the dial center coordinates specifically involves: when the confidence level of the key point at the center... When the preset value is used, the coordinates of the dial center are extracted using the coordinates of the key point of the dial center; when the key point of the center is occluded, it reverts to the weighted fusion of the minimum circumcircle fitting and the spatial moment centroid calculation of the mask contour of the instrument dial body. The zero-scale reference angle is obtained by calculating the reference direction angle using the coordinates of key points marked on the zero scale. When the zero-scale marker is obscured, the calibration value recorded in the pre-calibration record is used to fill in the gap.

5. A method for real-time segmentation and geometric reading of multiple types of industrial dials as described in any one of claims 1-3, characterized in that, The absolute deflection angle of the pointer is calculated using the atan2 function; the relative deflection angle is... Then add The linear mapping expression for the physical quantity is: In the formula, The total angular span at full scale is entered through system initialization calibration. This is the relative deflection angle; The multi-pointer parallel computing refers to the system performing occlusion fusion and angle calculation in parallel for multiple pointer instances on the same dial, and independently outputting the physical readings of each pointer.

6. A method for real-time segmentation and geometric reading of multiple types of industrial dials as described in any one of claims 1-3, characterized in that, The formula for generating the instance segmentation mask is: In the formula, For the first A prototype mask, For the first The mask coefficients of each target Use the Sigmoid activation function; The expression for the integrity of the segmentation mask is: ,in To predict the area of ​​mask pixels, To determine based on dial radius The expected area estimated by the aspect ratio of a typical pointer. ,coefficient Obtained through statistics from the training set; The Euclidean distance from each endpoint to the center of the circle is calculated using the following expression: In the formula, Let C be the i-th skeleton endpoint and C be the center of the circle. , Let be the coordinates of the i-th skeleton endpoint. , The coordinates of the center of the circle; The weighted fusion of the main path coordinates of the key points and the alternative path coordinates of the segmented skeleton is expressed as follows: In the formula, The fusion weights for the main path of key points, The confidence level of the key point at the tip of the pointer. To ensure the integrity of the segmentation mask, To segment the skeletonized backup path fusion weights, , The final weighted and merged pointer tip coordinates. , The coordinates of the pointer tip output by the main path of the key point. , The pointer tip coordinates are output for the segmented skeletonized alternative path.

7. A method for real-time segmentation and geometric reading of multiple types of industrial dials as described in any one of claims 1-3, characterized in that, In step S1, image data acquisition uses an HDR industrial camera, outputting the raw image stream in Raw Bayer format. The image data preprocessing method is as follows: First, a de-mosaic operation is performed using a bilinear interpolation algorithm to convert the Raw Bayer format into a BGR three-channel color image. Then, CLAHE brightness enhancement processing is performed on the L channel of the LAB color space. The image is divided into a set grid, a Clip Limit constraint factor is set, and a smooth transition between grids is achieved through bicubic interpolation. Finally, the image is scaled proportionally according to a set number of pixels, and the shorter sides are filled with grayscale values ​​to complete the pixel value normalization process.

8. The method for real-time segmentation and geometric reading of multiple types of industrial dials as described in claim 7, characterized in that, The HDR industrial camera has a CMOS image sensor with a dynamic range of no less than 120dB, a resolution of no less than 1920×1200, and a frame rate of no less than 30FPS in conventional monitoring scenarios and no less than 60FPS in high-speed mobile inspection scenarios; and / or, the HDR industrial camera is also equipped with an LED supplementary lighting module with a color rendering index (CRI) > 95 and a wide illumination working range covering 50-50000 lux; and / or, when the HDR industrial camera is installed, the angle between the camera optical axis and the normal to the dial plane is controlled within the range of 0-15°; and / or, the HDR industrial camera synchronously records timestamps and camera number metadata during the acquisition process.

Images