Double-stage pre-training system for reading of industrial inspection instrument

Abstract

The invention discloses a double-stage pre-training system for reading of an industrial inspection instrument, and belongs to the technical field of industrial visual inspection and intelligent inspection. The system comprises a structure interpretable parameterized instrument data synthesis module, a mask auto-encoder pre-training module, a multi-task joint pre-training module and an industrial deployment fine tuning module. Constructing a plurality of pointer type instrument structure templates based on a parametric modeling mode, randomizing parameters to generate high-diversity synthetic instrument data, and automatically generating corresponding structure and semantic annotations; performing unsupervised pre-training by using a mask auto-encoder to learn the relationship between the underlying structure features of the instrument image and the global space; then, multi-task joint pre-training is introduced into the shared backbone network, and multi-task collaborative optimization is carried out; in the industrial deployment stage, through multiple task heads and newly added target detection task heads, small sample fine adjustment is carried out in combination with a small amount of field data, and rapid adaptation to a specific industrial environment is realized. The method is suitable for an automatic instrument reading scene in a complex industrial environment.

Description

A two-stage pre-training system for industrial inspection instrument readings Technical Field

[0001] This invention relates to the fields of computer vision, deep learning and industrial intelligent inspection technology, and specifically to a two-stage pre-training system for industrial inspection instrument reading tasks. Background Technology

[0002] Instrument reading is one of the fundamental and critical tasks in industrial inspection systems, widely used in industries such as power, chemical, petroleum, metallurgy, and rail transportation. Accurate identification and reading of instruments such as pressure gauges, ammeters, and level gauges can promptly detect equipment malfunctions, reduce the risk of production accidents, and significantly reduce the cost of manual inspection and meter reading. Existing computer vision-based industrial instrument reading methods mostly employ the following technical process: collecting a large amount of instrument image data at the industrial site, manually annotating the collected images, supervising training based on convolutional neural networks or object detection networks, correcting the instrument area using image preprocessing algorithms, calculating the reading based on the detection or key point identification results, and outputting and recording the reading results. While these methods can achieve instrument reading in specific scenarios, they still have some prominent problems in practical industrial applications. First, they lack robustness to complex imaging environments. Industrial inspection environments often involve drastic changes in lighting, reflections on instrument surfaces, skewed shooting angles, and equipment vibrations causing image blurring, resulting in unstable image quality. Traditional supervised learning-based models experience a significant drop in recognition accuracy in such scenarios. Second, they are highly dependent on large-scale on-site labeled data. Existing methods are essentially data-driven models, requiring the collection and labeling of large amounts of real-world industrial data. The high cost of data acquisition and labeling severely restricts the rapid deployment of models across different industrial environments. Thirdly, their generalization and transfer capabilities are weak. Industrial instruments are diverse, with significant differences in appearance, scale distribution, and font styles between different manufacturers and specifications. Existing models are poorly adaptable to instrument types outside the training data distribution, typically requiring re-collection of data and targeted training, resulting in high R&D and maintenance costs. Fourthly, model update and iteration cycles are long. When new instrument types are added to the field or shooting conditions change, traditional methods often require retraining, failing to meet the rapid iteration and continuous deployment requirements of industrial inspection systems. Therefore, traditional methods lack sufficient modeling capabilities for the general structural features and visual priors of instruments. The model training process heavily relies on real-world data, lacking low-cost, high-coverage training sample sources; and lacks a unified pre-training framework for multiple tasks and scenarios, leading to fragmented model capabilities. Summary of the Invention

[0003] To address the technical problems of traditional methods, such as weak environmental robustness, heavy reliance on manual data annotation, weak generalization and transfer capabilities, and slow iterative deployment speed, this invention proposes a two-stage pre-training system for industrial inspection instrument readings. The system includes a structurally interpretable parametric instrument data synthesis module, a masked autoencoder pre-training module, a multi-task joint pre-training module, and an industrial deployment fine-tuning module. The structurally interpretable parametric instrument data synthesis module generates synthetic data and automatically acquires data labels, integrating them into a pre-training dataset. The masked autoencoder pre-training module randomly masks the images from the training dataset and inputs them into the module, calculates the loss function, and optimizes it through backpropagation to obtain the first-stage pre-trained model. The multi-task joint pre-training module sets up a multi-task detection head, retains the encoder portion of the model pre-trained by the masked autoencoder pre-training module, and inputs the encoder's output features into the multi-task detection head. Multi-task joint supervised training is performed on the pre-training dataset obtained from the structurally interpretable parametric instrument data synthesis module, calculating a weighted loss function and optimizing it through backpropagation to obtain the final pre-trained model. The industrial deployment fine-tuning module freezes the network parameters of the feature extraction portion of the final pre-trained model and fine-tunes them using actual labeled data from industrial inspection, enabling rapid industrial deployment.

[0004] The structurally interpretable parametric instrument data synthesis module is used to generate instrument composite data. Based on parametric modeling, it constructs structural templates for various pointer-type instruments and modularly decomposes the instrument dial, pointer, and scale structural elements. These elements include randomized combinations of scale style, pointer angle, numeral font, background texture, lighting conditions, and noise interference parameters to generate highly diverse instrument composite images. Simultaneously, it generates structural tags, semantic tags, and task tags that correspond one-to-one with the composite image. The tags include dial center coordinates, pointer angle, scale value range, and segmentation mask information.

[0005] The mask autoencoder pre-training module is used for the first stage pre-training of the model. It uses the pre-training dataset to perform the first stage unsupervised pre-training of the mask autoencoder with the ViT and CNN joint structure to obtain a feature encoding network with general meter structure perception capability.

[0006] The multi-task joint pre-training module is used to perform the second stage of pre-training of the model. Based on the feature encoding network parameters completed in the first stage of pre-training, the encoder part is frozen and retained. Multiple task prediction heads are introduced into the subsequent network structure to perform the second stage of multi-task joint pre-training, so as to further enhance the network's ability to jointly model the multi-dimensional attributes of the instrument.

[0007] The industrial deployment fine-tuning module is used to actually deploy the model that has completed the two-stage pre-training on the industrial inspection site, freeze the feature extraction parameters of the feature encoding network that has completed the two-stage pre-training, retain the feature extraction network structure and part of the network structure of the multiple task prediction heads, add a target detection head, and only fine-tune the detection task head and some high-level features on a small sample of labeled data in the industrial site to achieve rapid deployment of the instrument reading task.

[0008] The structurally interpretable parametric instrument data synthesis module includes a structural template extraction unit, a randomized image generation unit, and a label generation unit. It constructs structural templates for various pointer-type industrial instruments based on parametric modeling and generates highly diverse instrument image data by randomly combining various controllable parameters.

[0009] The structural template extraction unit is used to decompose existing industrial instrument models and extract individual instrument component models, including instrument dials and instrument pointers.

[0010] The randomized image generation unit is used to randomly stitch and process the obtained instrument components to obtain synthetic image data, specifically including the following steps:

[0011] Step S1: Randomly select a dial area of ​​a pointer-type instrument from the instrument component models as the base dial structure of the current synthesized image; the selected dials have differences in appearance features, including size, color texture, and scale distribution, to enhance the diversity of the synthesized data;

[0012] Step S2: Randomly select a pointer model from the pointer component library. The pointer model can be a single-needle or multi-needle structure, and they can be different from each other in length, width, and shape to simulate the actual structure of different types of instruments.

[0013] Step S3: Using the vertical upward direction of the pointer as the zero-degree reference direction, rotate the pointer to cover the main effective reading range of the industrial pointer instrument;

[0014] Step S4: Randomly select a set of scale values ​​from the pre-built set of scale values. The scale values ​​can be evenly or non-evenly distributed, and the value range can be configured according to the instrument type.

[0015] Step S5: Connect the rotated hands to the selected dial at the geometric center, so that the rotation center of the hands coincides with the center of the dial, forming a complete pointer-type instrument structure.

[0016] Step S6: According to the order of the scales on the dial, replace the original scale marking information on the dial with a set of scale values ​​randomly selected in step S4, while keeping the spatial relative position between the scales unchanged, thereby generating a new scale distribution pattern.

[0017] Step S7: Randomly translate the assembled instrument model along the X, Y, and Z axes in the three-dimensional coordinate system to simulate the instrument's different installation positions in the actual industrial environment.

[0018] Step S8: Rotate the assembled instrument model randomly with the instrument center as the rotation center; the rotation operation is used to simulate the tilt and skew posture changes of the instrument during actual shooting.

[0019] Step S9: Randomly set the camera position in the three-dimensional coordinate system so that the camera optical axis is oriented toward the center of the instrument, in order to simulate the imaging form of the instrument under different shooting angles during industrial inspection.

[0020] Step S10: Randomly set one or more light source positions in three-dimensional space, with the light source brightness varying randomly within a preset range, and the light source facing the instrument panel area to simulate the changes in light intensity, direction, and shadow under different industrial site conditions;

[0021] The annotation generation unit is used to synchronously record annotation data information during the processing of the randomized image generation unit. The recorded annotation data information includes dial position, pointer angle, pointer vertex position, scale information, reading information, maximum range value, and minimum range value.

[0022] The mask autoencoder pre-training module includes an image preprocessing unit, an encoder unit, and a decoder unit.

[0023] The image preprocessing unit is used to process the image data in the input pre-training dataset, including:

[0024] Step S1: Perform random masking on the input image with a fixed mask ratio;

[0025] Step S2: Decompose the masked image into several image blocks of the same size;

[0026] The encoder unit employs a network structure combining ViT and CNN as its feature learning network. Specifically: the CNN network extracts local texture and edge structure features from the instrument image; the ViT network performs global modeling of the instrument image to capture the overall structural relationships and spatial layout information of the dial; the multi-scale features output by the CNN network are concatenated in image patch order and input into the ViT network to obtain global image features, with the output features serving as the output of the encoder unit; the decoder unit consists of multiple fully connected layers, which decode the image features obtained by the encoder unit to obtain the final output; the decoder unit performs pixel-level reconstruction of the masked region based on the output features of the encoder unit, and updates the network parameters by calculating the reconstruction loss function between the reconstructed image and the original unmasked image; the mathematical expression of the reconstruction loss function is as follows:

[0027] ;

[0028] in Let be the reconstruction loss function of the mask autoencoder. , Given the height and width of the input image, In pixel coordinates The reconstructed image pixel values ​​at that location, In pixel coordinates The input image pixel values ​​at that location.

[0029] The multi-task joint pre-training module includes a parameter freezing unit, a multi-task head setting unit, and a multi-task joint loss function calculation unit.

[0030] The parameter freezing unit is used to freeze the network parameters obtained by the mask autoencoder pre-training module and retain only the encoder unit part as the feature extraction network for subsequent training.

[0031] The multi-task head setting unit is used to add multiple task heads after the network structure obtained by the parameter freezing unit. These multiple task heads include a spatial regression task head, an angle regression task head, a numerical regression task head, and a mask segmentation task head. By introducing multiple task heads, the structural, geometric, and reading information of the instrument are jointly modeled, thereby obtaining a feature representation with strong generalization ability. By sharing the low-level feature representation and introducing task-specific task heads at higher levels, multi-source supervision signals can be introduced during training, enabling the feature extraction network to simultaneously learn geometric structural features, numerical mapping relationships, and semantic segmentation features, thus significantly enhancing the model's generalization ability and robustness. The spatial regression task head is used to regress coordinate information related to spatial location, and its prediction target is the spatial coordinate information related to the instrument. The spatial regression task head, based on shared features, outputs a two-dimensional vector through a fully connected layer. Each pair of coordinate points This describes the coordinate results of a spatial regression task; the angle regression task header is used to predict angle information related to the geometric direction of the instrument, and its prediction target is the angle information related to the instrument; to avoid the problems of periodicity and non-uniqueness of angles, the angle regression task header uses a sine-cosine joint constraint method for angle prediction, and the specific output is:

[0032] ;

[0033] in Let be the predicted angle value in radians for the i-th angle, and .

[0034] The sine-cosine joint constraint method ensures the uniqueness and continuity of the angle representation, and the angle value is recovered by the arctangent function during the inference stage, which can effectively prevent the ambiguity of the results caused by the periodicity of the angle during model training.

[0035] The numerical regression task header is used to regress information related to instrument values, and its specific output is as follows:

[0036] ;

[0037] in This is the actual value of the predicted i-th value. The task header outputs continuous numerical results using a regression approach.

[0038] The mask segmentation task head is used to perform pixel-level segmentation of the dial area and the pointer area. Its structure adopts the form of a semantic segmentation network. The segmentation results are used to constrain the model's focus on the instrument structure area and provide auxiliary constraints for other regression tasks.

[0039] The multi-task joint loss function calculation unit is used to calculate the multi-task joint weighted loss function, as follows:

[0040] ;

[0041] in Let be the loss function for the spatial regression task header. Let the loss function be the header of the angle regression task. The loss function for the numerical regression task header. The loss function for masking the task header. These are the corresponding weighting coefficients;

[0042] The loss function of the spatial regression task head is as follows:

[0043] ;

[0044] in For the number of training images, The number of keypoints predicted in each image. The predicted coordinates of the k-th keypoint in the n-th image. The true coordinates of the k-th keypoint in the n-th image. Let be the position regression loss function over the entire training dataset.

[0045] The loss function for the angle regression task head is as follows:

[0046] ;

[0047] in For the number of training images, The number of predicted angles in each image. For the first The first image Predicted values ​​for each angle, For the first The first image The actual labeled values ​​from each angle, Let be the angle regression loss function over the entire training dataset.

[0048] The loss function of the numerical regression task head is as follows:

[0049] ;

[0050] in For the number of training images, The number of predicted values ​​in each image. For the first The first image The prediction results for each value, For the first The first image The actual labeled values ​​of each numerical value This is the numerical regression loss function.

[0051] The loss function for the mask segmentation task header is as follows:

[0052] ;

[0053] in For the number of training images, To output the height and width of the mask, For the first The image at the pixel level The real mask label, For the model in pixels The probability of predicting a location as the target category. The loss function for mask segmentation.

[0054] The multi-task joint pre-training module shares the feature extraction network parameters obtained by the parameter freezing unit for all tasks; the multi-task head receives the shared features in parallel and outputs the prediction results; the multi-task joint loss function calculation unit jointly calculates the weighted loss function of the multi-task head to perform end-to-end training on the overall network; during backpropagation, the gradients of each task work together on the shared backbone network, enabling it to learn a general feature representation that meets the needs of multiple tasks.

[0055] The industrial deployment fine-tuning module includes a field data collection unit, a data annotation unit, a data augmentation unit, a network structure adjustment unit, and a small sample fine-tuning unit.

[0056] The field data collection unit is used to collect actual data from industrial instruments in the actual industrial environment of the two-stage pre-training system for industrial inspection instrument readings, and to create a small-sample fine-tuning dataset.

[0057] The data annotation unit is used to annotate the collected data after the application of the field data collection unit. The annotation information includes instrument reading results, pointer position rectangles, and numerical scale position rectangles. The small sample fine-tuning dataset includes images and annotation information.

[0058] The data augmentation unit is used to introduce data augmentation strategies into the small sample fine-tuning dataset to improve the robustness of the model to complex industrial environments. The data augmentation strategies include brightness perturbation, contrast adjustment, random rotation, affine transformation, and local occlusion simulation.

[0059] The network structure adjustment unit is used to freeze the parameters of the shallow network of the backbone feature extraction network obtained by the pre-training after the completion of the multi-task joint pre-training module, so that the general instrument structure features and visual representation capabilities learned in the pre-training stage remain unchanged during the industrial deployment stage. Specifically, the parameters of the shallow network of the ViT and CNN networks are frozen and remain unchanged in the industrial deployment fine-tuning module, and are only used for feature extraction. The spatial regression task head, angle regression task head and mask segmentation task head in the multi-task joint pre-training module are removed, and only the numerical regression task head is retained for the final reading. A new task head for target detection is added, which, together with the retained numerical regression task head, completes fine-tuning on the field data.

[0060] The few-sample fine-tuning unit is used to train the network model after the network structure adjustment unit on the few-sample fine-tuning dataset. It only fine-tunes the target detection task head, the retained numerical regression task head, and the feature layers directly connected to them, without updating the frozen backbone feature extraction network parameters. This allows for rapid adaptation to specific industrial environments while ensuring the model's generalization ability. The trainable parameters are optimized and updated using a learning rate lower than that used in the pre-training stage to avoid overfitting or severe oscillations in the model parameters under small sample conditions. At the same time, the number of training rounds is dynamically adjusted according to the scale of the field data, and training is terminated when the loss function converges or the performance is verified to be stable.

[0061] The joint loss function of the target detection task head and the numerical regression task head is as follows:

[0062]

[0063] in For the overall loss function during the fine-tuning stage, The weights of the target detection loss function are... These are the weights of the numerical regression loss function.

[0064] Let be the loss function for the target detection task header. The loss function for the numerical regression task header includes:

[0065]

[0066] in The weights are the values ​​for the bounding box regression loss. Bounding box regression loss function The weights for the target confidence loss. Let the target confidence loss function be... Weights for predicting loss by category, The loss function is for category prediction. The implementation method is consistent with that in the multi-task joint pre-training module mentioned above.

[0067] The beneficial effects of this invention are:

[0068] This invention establishes a structurally interpretable parametric instrument data synthesis module. During the pre-training dataset creation stage, it modularly decomposes structural elements such as instrument dials, needles, and scales using parametric modeling. It also randomly combines key imaging factors such as lighting conditions, shooting angle, background texture, and noise interference to generate highly diverse synthetic instrument data covering various imaging conditions. This allows the model to fully experience instrument morphology under different lighting variations, viewing angle deviations, and noise interference during the training phase, enhancing the model's adaptability to complex industrial imaging environments from a data perspective and effectively addressing the insufficient robustness of existing methods in complex imaging environments. Simultaneously, this invention utilizes a mask autoencoder pre-training module to enable the model to learn the underlying semantic features of instrument images, demonstrating strong feature extraction and reconstruction capabilities even for instruments in occluded environments, thus addressing the insufficient robustness of existing methods to complex imaging environments from a model perspective.

[0069] During the pre-training phase, the system primarily relies on the large-scale synthetic samples automatically generated by the structure-interpretable parameterized instrument data synthesis module, along with their corresponding structural, semantic, and task labels. This significantly reduces the dependence on manually labeled data from real industrial sites. In the subsequent industrial deployment phase, only a small number of field samples need to be collected and fine-tuned to complete model adaptation. This avoids the high cost problem of collecting and labeling a large amount of field data, which is required in traditional methods, thus effectively reducing the data collection and labeling costs of industrial inspection systems.

[0070] By introducing a joint encoder structure of ViT and CNN as the backbone network during the pre-training stage, where CNN focuses on extracting local texture and edge structure features such as pointers and scales, and ViT is used to model the overall structural relationship and global spatial layout of the instrument, a feature fusion mechanism is used to achieve complementary representation of local features and global structural information. This structure enables the model to focus on learning the general structural priors and geometric relationships of the instrument during the pre-training stage, rather than relying solely on specific appearance styles, thereby significantly improving the model's generalization ability to instruments from different manufacturers, with different specifications and appearances. This effectively solves the problem of weak generalization and transfer capabilities of existing models when faced with instrument types outside the training data distribution. At the same time, this invention introduces a multi-task joint training mechanism during the pre-training stage. Multiple cross-correlated task heads, such as dial type prediction, pointer angle regression, spatial coordinate prediction, numerical information regression, and mask segmentation, are set up simultaneously in the backend of the shared backbone feature extraction network. Through joint loss function for collaborative optimization, the model forms a consistent feature representation at multiple levels of structure, space, and semantics, effectively improving the model's overall understanding of complex instrument reading scenarios and solving the problem of weak model generalization ability in complex scenarios.

[0071] During the industrial deployment phase, the backbone feature extraction network parameters obtained from the two-stage pre-training are frozen, the multi-task heads from the pre-training phase are removed, and a new target detection task head for industrial inspection is added. Only a small number of newly added parameters are fine-tuned using small samples. This fine-tuning strategy enables the model to quickly adapt and deploy under different industrial site conditions without retraining the entire model, significantly shortening the system launch and iteration cycle. This solves the problem of long model update and iteration cycles and meets the actual needs of industrial inspection systems for rapid deployment and stable operation. Attached Figure Description

[0072] Figure 1 is a schematic diagram of the structure of the two-stage pre-training system of the present invention.

[0073] Figure 2 is a schematic diagram of the structure of the structurally interpretable parameterized instrument data synthesis module.

[0074] Figure 3 is a schematic diagram of the structure of the mask autoencoder pre-training module.

[0075] Figure 4 is a schematic diagram of the structure of the multi-task joint pre-training module.

[0076] Figure 5 is a structural diagram of the industrial deployment fine-tuning module. Detailed Implementation

[0077] To make the objectives, technical solutions, and beneficial effects of the present invention clearer, the embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0078] The present invention provides a two-stage pre-training system for industrial inspection instrument readings, as shown in Figure 1, including a structurally interpretable parameterized instrument data synthesis module, a masked autoencoder pre-training module, a multi-task joint pre-training module, and an industrial deployment fine-tuning module. Synthetic data is generated through the structurally interpretable parameterized instrument data synthesis module, and data labels are automatically acquired and integrated into a pre-trained dataset. The images from the training dataset are randomly masked and input into the masked autoencoder pre-training module. The loss function is calculated and optimized through backpropagation to obtain the first-stage pre-trained model. The multi-task joint pre-training module sets up a multi-task detection head. The model pre-trained by the masked autoencoder pre-training module retains the encoder part, and the encoder output features are input to the multi-task detection head. Multi-task joint supervised training is performed on the pre-trained dataset obtained from the structurally interpretable parameterized instrument data synthesis module. The weighted loss function is calculated and optimized through backpropagation to obtain the final pre-trained model. The industrial deployment fine-tuning module freezes the network parameters of the feature extraction part of the final pre-trained model and fine-tunes them using actual labeled data from industrial inspection, enabling rapid industrial deployment.

[0079] Furthermore, a structurally interpretable parametric instrument data synthesis module is used to generate instrument composite data. Based on parametric modeling, it constructs structural templates for various pointer-type instruments and modularly decomposes the instrument dial, pointer, and scale structural elements. These elements specifically include randomized combinations of scale style, pointer angle, numeral font, background texture, lighting conditions, and noise interference parameters to generate highly diverse instrument composite images. Simultaneously, it generates structural labels, semantic labels, and task labels that correspond one-to-one with the composite image. These labels specifically include the dial center coordinates, pointer angle, scale value range, and segmentation mask information.

[0080] Furthermore, the masked autoencoder pre-training module is used for the first stage of pre-training of the model. It utilizes the pre-training dataset to perform the first stage of unsupervised pre-training on the masked autoencoder employing a joint ViT and CNN structure, thereby obtaining a feature encoding network with general meter structure awareness capabilities. Specifically, the CNN network is a ResNet50 network.

[0081] Furthermore, the multi-task joint pre-training module is used to implement the second stage of pre-training of the model. Based on the feature encoding network parameters completed in the first stage of pre-training, the encoder part is frozen and retained. Multiple task prediction heads are introduced into the subsequent network structure to perform the second stage of multi-task joint pre-training, so as to further enhance the network's ability to jointly model the multi-dimensional attributes of the instrument.

[0082] Furthermore, the industrial deployment fine-tuning module is used to actually deploy the model that has completed the two-stage pre-training on the industrial inspection site, freeze the feature extraction parameters of the feature encoding network that has completed the two-stage pre-training, retain the feature extraction network structure and part of the multiple task prediction head network structure, add the target detection head, and only fine-tune the detection task head and some high-level features on a small number of industrial site labeled data to achieve rapid deployment of the instrument reading task.

[0083] In one embodiment, the structurally interpretable parametric instrument data synthesis module, as shown in Figure 2, includes a structural template extraction unit, a randomized image generation unit, and a label generation unit. Based on parametric modeling, structural templates for various pointer-type industrial instruments are constructed, and highly diverse instrument image data is generated by randomly combining various controllable parameters.

[0084] Furthermore, the structural template extraction unit decomposes the existing industrial instrument model and extracts individual instrument component models, including the instrument dial and instrument pointer.

[0085] Furthermore, the randomized image generation unit performs randomized splicing and processing on the obtained instrument components to obtain synthetic image data, including the following steps:

[0086] Step S1: Randomly select a dial area of ​​an analog instrument from the existing dial component library as the base dial structure for the current composite image. Specifically, the dial component library contains four types of dials and four types of hands, with twenty different scale markings.

[0087] Step S2: Randomly select a needle model from the needle component library, wherein the needle model is specifically a single-needle structure.

[0088] Step S3: Using the vertically upward direction of the hands as the zero-degree reference direction, rotate the hands by an angle, the rotation angle being the angle parameter α, where:

[0089] ;

[0090] This angle range setting can cover the main effective reading range of industrial pointer instruments.

[0091] Step S4: Randomly select a set of scale values ​​from the pre-built set of scale values. The scale values ​​can be evenly spaced or non-evenly spaced, specifically evenly spaced.

[0092] Step S5: Connect the rotated hands to the selected dial at the geometric center, so that the rotation center of the hands coincides with the center of the dial, forming a complete pointer-type instrument structure.

[0093] Step S6: According to the order of the scales on the dial, replace the original scale marking information on the dial with a set of scale values ​​randomly selected in step S4, while keeping the spatial relative position between the scales unchanged, thereby generating a new scale distribution.

[0094] Step S7: The assembled instrument model is randomly translated along the X, Y and Z axes in the three-dimensional coordinate system to simulate the instrument's different installation positions in the actual industrial environment.

[0095] Step S8: Using the center of the instrument as the rotation center, randomly rotate the assembled instrument model. The rotation operation uses Euler angles. Specifically:

[0096] Euler angles are the three angles respectively , , ,in:

[0097]

[0098]

[0099]

[0100] The rotation angle is randomly selected within the aforementioned preset range. The rotation operation is used to simulate the tilting, skewing, and other posture changes that occur during actual shooting.

[0101] Step S9: Randomly set the camera position in the three-dimensional coordinate system so that the camera optical axis is oriented towards the center of the instrument, and limit the angle between the camera optical axis and the instrument center normal to be less than 15°, so as to simulate the instrument imaging form under different shooting angles during industrial inspection.

[0102] Step S10: Randomly set one or more light source positions in three-dimensional space. The brightness of the light source varies randomly within a preset range, and the light source is oriented towards the instrument panel area to simulate the changes in light intensity, direction, and shadow under different industrial site conditions.

[0103] Furthermore, the annotation generation unit synchronously records annotation data information during the processing of the randomized image generation unit. Specifically, while generating the composite image, it automatically records and generates the following annotation information: pointer angle annotation; dial center coordinates; minimum scale value and corresponding coordinates and angles; maximum scale value and corresponding coordinates and angles; actual dial reading; pixel-level mask of the dial area and pointer area.

[0104] Repeat the above construction steps 1500 times to form an instrument synthesis training dataset containing 1500 images and corresponding annotation information.

[0105] In one embodiment, the mask autoencoder pre-training module, as shown in Figure 3, includes an image preprocessing unit, an encoder unit, and a decoder unit.

[0106] Furthermore, the image preprocessing unit is used to process the image data within the input pre-training dataset, including:

[0107] Step S1: Apply a random mask with a masking ratio of 0.75 to the input image.

[0108] Step S2: Decompose the masked image into several image blocks. Specifically, the input image size is uniformly adjusted to... The input image is sized to a fixed size. Divided into There are 1 image block, and each image block is represented as:

[0109] ;

[0110] Furthermore, the encoder unit employs a ViT and CNN co-structured encoder as the feature learning network, wherein: the CNN network uses a ResNet50 structure to extract local texture features and edge structure features from the instrument image; the ResNet50 network includes an input convolutional layer, a max-pooling layer, and four residual stages, each residual stage consisting of multiple bottleneck residual units, which achieve cross-layer feature transfer through identity mapping; the ResNet50 network is used to progressively extract low-level texture features, mid-level structural features, and high-level semantic features from the instrument image, and maps these features to a unified feature space for subsequent fusion with ViT features; the ViT network is used to perform global modeling of the instrument image to capture the overall structural relationships and spatial layout information of the dial; the multi-scale features output by the ResNet50 network are concatenated in image patch order and then input into the ViT network. The network acquires global image features, and the output features are used as the encoder output of the mask autoencoder; the decoder unit consists of two fully connected layers, which decode the image features encoded by the encoder unit to obtain a vector with the same output size as the input image size.

[0111] Further, in the image preprocessing unit, the input image is subjected to random masking with a mask ratio of 0.75, and the image is decomposed into image patches. Each image patch is input into the CNN to extract local features, and then the features of all image patches are concatenated into a sequence and input into the ViT. Specifically, the standard ResNet50 is designed with a 224×224 input image as the target. To adapt to image patch input, this embodiment makes the following engineering-feasible adjustments to the front-end structure of ResNet50: retaining the residual block structure of ResNet50; adapting the first layer convolution and pooling structure according to the image patch size; retaining the global average pooling layer for outputting fixed-dimensional feature vectors. For each image patch input, the feature result obtained by ResNet50 is:

[0112] ;

[0113] The features of the stitched image patches are:

[0114] ;

[0115] A linear projection layer maps the CNN features to the embedding dimension of ViT. :

[0116] ;

[0117] in Specifically, D=768.

[0118] To preserve the spatial location information of image patches, positional encoding is added to the features of each image patch:

[0119] ;

[0120] in .

[0121] The decoder performs pixel-level reconstruction of the masked region based on the encoded features, and updates the network parameters by calculating the reconstruction loss function between the reconstructed image and the original unmasked image; the mathematical expression of the reconstruction loss function is as follows:

[0122] ;

[0123] in Let be the reconstruction loss function of the mask autoencoder. , Given the height and width of the input image, In pixel coordinates The reconstructed image pixel values ​​at that location, In pixel coordinates The input image pixel values ​​at that location.

[0124] In one embodiment, the multi-task joint pre-training module, as shown in Figure 4, includes a parameter freezing unit, a multi-task head setting unit, and a multi-task joint loss function calculation unit.

[0125] Furthermore, the parameter freezing unit is used to freeze the network parameters obtained by the masked autoencoder pre-training module, and retain only the encoder unit portion as the feature extraction network for subsequent training. Specifically, the parameters of the ViT and ResNet50 networks are frozen and remain unchanged within the industrial deployment fine-tuning module, only used for feature extraction. Specifically, the Conv1 layer, BatchNorm1 layer, ReLU layer, MaxPool layer, Conv2_x layer, Conv3_x layer, and Conv4_x layer of ResNet50 are frozen, while the Conv5_x layer is retained for fine-tuning to enhance adaptability to specific industrial scenarios; the ViT linear mapping layer, position encoding, multi-head self-attention layer, feedforward network portion, and LayerNorm layer are frozen.

[0126] Furthermore, the multi-task head setting unit is used to add multiple task heads after the network structure obtained by the parameter freezing unit. These multiple task heads include a spatial regression task head, an angle regression task head, a numerical regression task head, and a mask segmentation task head. By introducing multiple task heads, the structural, geometric, and reading information of the instrument are jointly modeled, thereby obtaining a feature representation with strong generalization ability. By sharing the low-level feature representation and introducing task-specific task heads at higher levels, multi-source supervision signals can be introduced during training, enabling the encoder to simultaneously learn geometric structural features, numerical mapping relationships, and semantic segmentation features, thereby significantly enhancing the model's generalization ability and robustness. The multiple task heads are used to complete different instrument recognition sub-tasks, specifically:

[0127] The spatial regression task head is used to regress coordinate information related to spatial location. Its prediction targets include: the coordinates of the instrument pointer vertex, the coordinates corresponding to the minimum value of the instrument range, the coordinates corresponding to the maximum value of the instrument range, and the coordinates of the dial center. The spatial regression task head, based on shared features, outputs a two-dimensional vector through a fully connected layer. Each pair of coordinate points This is used to describe the coordinate results of a spatial regression task. Specifically, it outputs a spatial coordinate vector of n=4.

[0128] The angle regression task head is used to predict angle information related to the instrument's geometric direction. Its prediction targets include: the current angle of the instrument pointer, the angle corresponding to the minimum scale point of the instrument's range, and the angle corresponding to the maximum scale point of the instrument's range. To avoid the problems of periodicity and non-uniqueness of angles, the angle regression task head uses a sine-cosine joint constraint method for angle prediction. The specific output is as follows:

[0129] ;

[0130] in Let be the predicted angle value in radians for the i-th angle, and .

[0131] The sine-cosine joint constraint method ensures the uniqueness and continuity of the angle representation, and the arctangent function is used to recover the angle value during the inference phase, effectively preventing ambiguity in the model training results caused by the periodicity of the angle. Specifically, the output is an angle prediction vector of n=3.

[0132] The numerical regression task header is used to regress information related to instrument values, and its specific output is as follows:

[0133] ;

[0134] in This is the actual value of the predicted i-th value. The prediction targets include: the minimum value of the instrument's range, the maximum value of the instrument's range, and the current instrument reading. This task header outputs continuous numerical results through regression. Specifically, it outputs a numerical prediction vector of n=3.

[0135] The mask segmentation task head is used to perform pixel-level segmentation of the dial area and the pointer area. Its structure adopts the form of a semantic segmentation network. The segmentation results are used to constrain the model's focus on the instrument structure area and provide auxiliary constraints for other regression tasks.

[0136] The multi-task joint weighted loss function is as follows:

[0137] ;

[0138] in Let be the loss function for the spatial regression task header. Let the loss function be the header of the angle regression task. The loss function for the numerical regression task header. The loss function for masking the task header. For the corresponding weighting coefficients, specifically:

[0139] ;

[0140] The loss function of the spatial regression task head is as follows:

[0141] ;

[0142] in For the number of training images, The number of keypoints predicted in each image. The predicted coordinates of the k-th keypoint in the n-th image. The true coordinates of the k-th keypoint in the n-th image. Let be the position regression loss function over the entire training dataset.

[0143] The loss function of the angle regression task head is as follows:

[0144] ;

[0145] in For the number of training images, The number of predicted angles in each image. For the first The first image Predicted values ​​for each angle, For the first The first image The actual labeled values ​​from each angle, Let be the angle regression loss function over the entire training dataset.

[0146] The loss function of the numerical regression task head is as follows:

[0147] ;

[0148] in For the number of training images, The number of predicted values ​​in each image. For the first The first image The prediction results for each value, For the first The first image The actual labeled values ​​of each numerical value This is the numerical regression loss function.

[0149] The loss function for the mask segmentation task header is as follows:

[0150] ;

[0151] in For the number of training images, To output the height and width of the mask, For the first The image at the pixel level The real mask label, For the model in pixels The probability of predicting a location as the target category. The loss function for mask segmentation.

[0152] The multi-task joint training method involved is as follows: all tasks share the parameters of the ViT+ResNet50 joint encoder; each task head receives the shared features in parallel and outputs the prediction results; the overall network is trained end-to-end through a joint loss function; during backpropagation, the gradients of each task work together on the shared backbone network, enabling it to learn a general feature representation that meets the needs of multiple tasks.

[0153] In one embodiment, the industrial deployment fine-tuning module, as shown in Figure 5, includes a field data collection unit, a data annotation unit, a data augmentation unit, a network structure adjustment unit, and a small sample fine-tuning unit.

[0154] Furthermore, the field data collection unit is used to collect actual data from industrial instruments in a real industrial environment for the application of the two-stage pre-training system for industrial inspection instrument readings, in order to create a small-sample fine-tuning dataset. Specifically, the field data collection unit collects 100 real-world data points.

[0155] Furthermore, the data annotation unit is used to annotate the collected data after applying the field data collection unit. The annotation information specifically includes instrument reading results, pointer position rectangles, and numerical scale position rectangles. The image and annotation information together constitute the small sample fine-tuning dataset.

[0156] Furthermore, the data augmentation unit is used to introduce data augmentation strategies into the small sample fine-tuning dataset to improve the robustness of the model to complex industrial environments. The data augmentation strategies specifically include brightness perturbation, contrast adjustment, random rotation, affine transformation, and local occlusion simulation.

[0157] Furthermore, the network structure adjustment unit is used to freeze the parameters of the shallow network of the backbone feature extraction network obtained from the pre-training after the completion of the multi-task joint pre-training module, so that the general instrument structure features and visual representation capabilities learned in the pre-training stage remain unchanged during the industrial deployment stage. Specifically, the parameters of the shallow networks of the ViT and CNN networks are frozen and remain unchanged within the industrial deployment fine-tuning module, and are only used for feature extraction. Specifically, the Conv1 layer, BatchNorm1 layer, ReLU layer, MaxPool layer, Conv2_x layer, and Conv3_x layer of ResNet50 are frozen. Layer x and Conv4_x are retained, while Conv5_x is used for fine-tuning to enhance adaptability to specific industrial scenarios. The ViT linear mapping layer, position encoding, multi-head self-attention layer, feedforward network part, and LayerNorm layer are frozen to achieve rapid adaptation based on small sample field data. The spatial regression task head, angle regression task head, and mask segmentation task head in the multi-task joint pre-training module are removed, and only the numerical regression task head is retained for final reading. A new task head for object detection is added, which, together with the retained numerical regression task head, completes fine-tuning on field data. Specifically, the YOLO11 object detection head is used.

[0158] Furthermore, the few-sample fine-tuning unit trains the network model, after passing through the network structure adjustment unit, on the few-sample fine-tuning dataset. It only fine-tunes the target detection task head, the retained numerical regression task head, and the feature layers directly connected to them, without updating the frozen backbone feature extraction network parameters. This ensures rapid adaptation to specific industrial environments while maintaining the model's generalization ability. A learning rate lower than that used in the pre-training phase is employed to optimize and update the trainable parameters, avoiding overfitting or severe oscillations in model parameters under small-sample conditions; specifically, a learning rate of 0.005 is used. Simultaneously, the number of training epochs is dynamically adjusted based on the scale of the field data, and training terminates when the loss function converges or performance is verified to be stable.

[0159] The joint loss function of the target detection task head and the numerical regression task head is as follows:

[0160] ;

[0161] in For the overall loss function during the fine-tuning stage, The weights of the target detection loss function are... The weights for the numerical regression loss function are as follows:

[0162] ;

[0163] Let be the loss function for the target detection task header. The loss function for the numerical regression task header includes:

[0164] ;

[0165] in The weights are the values ​​for the bounding box regression loss. Bounding box regression loss function The weights for the target confidence loss. Let the target confidence loss function be... Weights for predicting loss by category, The loss function is used for category prediction.

[0166] Specifically:

[0167] ;

[0168] Specifically, CIoU loss is used to regress the bounding box, and it is defined as follows:

[0169] ;

[0170] in, For the target quantity, Let be the predicted value for the i-th target. Let be the true value of the i-th target. CIoU is defined as:

[0171] ;

[0172] in, It is the diagonal length of the smallest bounding rectangle containing the predicted bounding box and the ground truth bounding box. For aspect ratio consistency constraints, This is the balance coefficient. The squared Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box is given. For intersection-union ratio, the specific definition is as follows:

[0173] ;

[0174] in, Indicates the area of ​​the region. These represent the intersection and union, respectively.

[0175] The above aspect ratio consistency constraints The definition is as follows:

[0176] ;

[0177] in, To predict the width and height of the bounding box, These are the width and height of the ground truth bounding box. This parameter is used to constrain the consistency of the predicted bounding box with the ground truth bounding box in terms of shape (aspect ratio).

[0178] The above balance coefficient The definition is as follows:

[0179] ;

[0180] When the overlap between the predicted bounding box and the ground truth bounding box is small A smaller aspect ratio avoids excessive penalties; when the IoU is large, it strengthens the constraints on shape consistency.

[0181] CIoU, based on traditional IoU, introduces center point distance constraints and aspect ratio consistency constraints, so that the bounding box regression process considers the target position, target scale and target shape at the same time, thus providing stable gradient information even when the targets do not overlap or have weak overlap.

[0182] The target confidence loss function The definition is as follows:

[0183] ;

[0184] Among them, positive samples negative samples .

[0185] The category prediction loss function The definition is as follows:

[0186] ;

[0187] in For the number of objects, Let be the true label of the i-th object for category c. If the object belongs to category c, then... ,otherwise . Let be the probability that the i-th object belongs to category c.

[0188] The loss function of the numerical regression task head The implementation method is consistent with the above-mentioned parts.

Claims

1. A two-stage pre-training system for industrial inspection instrument readings, characterized in that: It includes a structurally interpretable parametric instrument data synthesis module, a masked autoencoder pre-training module, and a multi-task joint pre-training module. The structurally interpretable parametric instrument data synthesis module generates synthetic data and automatically acquires data labels, integrating them into a pre-training dataset. The masked autoencoder pre-training module randomly masks the images in the training dataset and inputs them into the module, calculates the loss function, and optimizes it through backpropagation to obtain the first-stage pre-trained model. The multi-task joint pre-training module sets up a multi-task detection head, retains the encoder portion of the model pre-trained by the masked autoencoder pre-training module, and inputs the encoder output features into the multi-task detection head. Multi-task joint supervised training is performed on the pre-training dataset obtained by the structurally interpretable parametric instrument data synthesis module, and the weighted loss function is calculated and optimized through backpropagation to obtain the final pre-trained model.

2. The two-stage pre-training system for industrial inspection instrument readings according to claim 1, characterized in that: The structurally interpretable parametric instrument data synthesis module is used to generate instrument composite data. Based on parametric modeling, it constructs structural templates for various pointer-type instruments, modularly decomposes the instrument dial, pointer, and scale structural elements, and randomly combines them to generate highly diverse instrument composite images. While generating composite images, it simultaneously generates structural labels, semantic labels, and task labels that correspond one-to-one with the composite images.

3. The two-stage pre-training system for industrial inspection instrument readings according to claim 1, characterized in that: The mask autoencoder pre-training module is used for the first stage pre-training of the model. The first stage unsupervised pre-training of the mask autoencoder with the ViT and CNN joint structure is performed using the pre-training dataset to obtain a feature encoding network with general meter structure perception capability.

4. A two-stage pre-training system for industrial inspection instrument readings according to claim 1 or 3, characterized in that: The multi-task joint pre-training module is used to implement the second stage of pre-training of the model. Based on the feature encoding network parameters completed in the first stage of pre-training, the encoder part is frozen and retained. Multiple task prediction heads are introduced into the subsequent network structure to perform the second stage of multi-task joint pre-training, so as to further enhance the network's ability to jointly model the multi-dimensional attributes of the instrument.

5. A two-stage pre-training system for industrial inspection instrument readings according to claim 1, characterized in that: It also includes an industrial deployment fine-tuning module, which is used to actually deploy the model that has completed the two-stage pre-training on the industrial inspection site, freeze the feature extraction parameters of the feature encoding network that has completed the two-stage pre-training, retain the feature extraction network structure and part of the network structure of the multiple task prediction heads, add a target detection head, and only fine-tune the detection task head and some high-level features on a small number of industrial site labeled data to achieve rapid deployment of the instrument reading task.

6. A two-stage pre-training system for industrial inspection instrument readings according to claim 2, characterized in that: The structure-interpretable parameterized instrument data synthesis module includes a structure template extraction unit, a randomized image generation unit, and a label generation unit. Based on parametric modeling, structural templates for various pointer-type industrial instruments are constructed, and highly diverse instrument image data is generated by randomly combining various controllable parameters. The structural template extraction unit is used to decompose existing industrial instrument models and extract individual instrument component models, including instrument dials and instrument pointers. The randomized image generation unit is used to randomly stitch and process the obtained instrument components to obtain synthetic image data, including the following steps: Step S1: Randomly select a dial area of ​​a pointer-type instrument from the instrument component models as the base dial structure of the current synthetic image; the selected dials differ in appearance features, including size, color texture, and scale distribution, to enhance the diversity of the synthetic data; Step S2: Randomly select a pointer model from the pointer component library, the pointer model can be a single-needle or multi-needle structure. Furthermore, they differ in length, width, and shape to simulate the actual structure of different types of instruments; Step S3: Using the vertical upward direction of the pointer as the zero-degree reference direction, rotate the pointer to cover the main effective reading range of the industrial pointer instrument; Step S4: Randomly select a set of scale values ​​from the pre-built set of scale values. The scale values ​​can be evenly or non-evenly spaced, and the value range can be configured according to the instrument type; Step S5: Connect the pointer after angle rotation to the selected dial at the geometric center position, so that the rotation center of the pointer coincides with the center position of the dial, forming a complete pointer instrument structure; Step S6: According to the order of the scales on the dial, replace the original scale marking information on the dial with a set of scale values ​​randomly selected in Step S4, while keeping the spatial relative position between the scales unchanged, thereby generating a new scale distribution form; Step S7: Randomly translate the assembled instrument model along the X, Y, and Z axes in a three-dimensional coordinate system to simulate different installation positions of the instrument in an actual industrial environment; Step S8: Randomly rotate the assembled instrument model with the instrument center as the rotation center; the rotation operation is used to simulate the posture changes of the instrument during actual shooting. Step S9: Randomly set the camera position in the three-dimensional coordinate system so that the camera optical axis is oriented towards the center of the instrument to simulate the imaging form of the instrument under different shooting angles during industrial inspection; Step S10: Randomly set one or more light source positions in the three-dimensional space, the brightness of the light source changes randomly within a preset range, and the light source is oriented towards the instrument dial area to simulate the changes in light intensity, direction and shadow under different industrial site conditions. The annotation generation unit is used to synchronously record annotation data information during the processing of the randomized image generation unit.

7. A two-stage pre-training system for industrial inspection instrument readings according to claim 3, characterized in that: The mask autoencoder pre-training module includes an image preprocessing unit, an encoder unit, and a decoder unit. The image preprocessing unit processes the image data in the input pre-training dataset, including: Step S1: performing random masking processing on the input image with a fixed mask ratio; Step S2: decomposing the masked image into several image blocks of the same size. The encoder unit uses a network structure composed of ViT and CNN as the feature learning network, wherein: the CNN network is used to extract local texture features and edge structure features in the instrument image; the ViT network is used to perform global modeling of the instrument image to capture the overall structural relationship and spatial layout information of the dial; the multi-scale features output by the CNN network are concatenated in the order of image blocks and then input into the ViT network. The network acquires global image features, and the output features are used as the output of the encoder unit. The decoder unit consists of multiple fully connected layers, which decode the image features obtained by the encoder unit to obtain the final output result. The decoder unit performs pixel-level reconstruction of the masked region based on the output features of the encoder unit, and updates the network parameters by calculating the reconstruction loss function between the reconstructed image and the original unmasked image. The mathematical expression of the reconstruction loss function is as follows: ;in Let be the reconstruction loss function of the mask autoencoder. 、 Given the height and width of the input image, In pixel coordinates The reconstructed image pixel values ​​at that location, In pixel coordinates The input image pixel values ​​at that location.

8. A two-stage pre-training system for industrial inspection instrument readings according to claim 4, characterized in that: The multi-task joint pre-training module includes a parameter freezing unit, a multi-task head setting unit, and a multi-task joint loss function calculation unit. The parameter freezing unit is used to freeze the network parameters obtained by the mask autoencoder pre-training module and retain only the encoder unit part as a feature extraction network for subsequent training. The multi-task head setting unit is used to add multiple task heads after the network structure obtained by the parameter freezing unit. The multiple task heads include a spatial regression task head, an angular regression task head, a numerical regression task head, and a mask segmentation task head. By introducing multiple task heads to jointly model the structural, geometric, and reading information of the instrument, a feature representation with strong generalization ability is obtained. By sharing the low-level feature representation and introducing task-specific task heads at the high level, multi-source supervision signals can be introduced during training, enabling the feature extraction network to learn geometric structural features, numerical mapping relationships, and semantic segmentation features simultaneously, thereby significantly enhancing the model's generalization ability and robustness. The spatial regression task head is used to regress coordinate information related to spatial location, and its prediction target is the spatial coordinate information related to the instrument. The spatial regression task head is based on shared features and outputs a two-dimensional vector through a fully connected layer. Each pair of coordinate points This describes the coordinate results of a spatial regression task; the angle regression task header is used to predict angle information related to the geometric direction of the instrument, and its prediction target is the angle information related to the instrument; to avoid the problems of periodicity and non-uniqueness of angles, the angle regression task header uses a sine-cosine joint constraint method for angle prediction, and the specific output is: ;in Let be the predicted angle value in radians for the i-th angle, and The sine-cosine joint constraint method ensures the uniqueness and continuity of the angle representation, and the arctangent function is used to recover the angle value during the inference phase, effectively preventing ambiguity in the model training results caused by the periodicity of the angle; the numerical regression task head is used to regress information related to the instrument values, and its specific output is: ;in To predict the true value of the i-th numerical value, this task head outputs continuous numerical results through regression. The mask segmentation task head is used to perform pixel-level segmentation of the dial area and the hand area. Its structure adopts a semantic segmentation network form. The segmentation results are used to constrain the model's focus range on the instrument structure area and provide auxiliary constraints for other regression tasks. The multi-task joint loss function calculation unit is used to calculate the multi-task joint weighted loss function, as follows: ;in Let be the loss function for the spatial regression task header. Let the loss function be the header of the angle regression task. The loss function for the numerical regression task header. The loss function for masking the task header. These are the corresponding weight coefficients; the loss function of the spatial regression task head is as follows: ;in For the number of training images, The number of keypoints predicted in each image. The predicted coordinates of the k-th keypoint in the n-th image. The true coordinates of the k-th keypoint in the n-th image. The position regression loss function is defined for the entire training dataset; the loss function for the angle regression task head is as follows: in For the number of training images, The number of predicted angles in each image. For the first The first image Predicted values ​​for each angle, For the first The first image The actual labeled values ​​from each angle, The angle regression loss function is defined for the entire training dataset; the specific loss function for the numerical regression task head is as follows: ;in For the number of training images, The number of predicted values ​​in each image. For the first The first image The prediction results for each value, For the first The first image The actual labeled values ​​of each numerical value The loss function is a numerical regression function; the specific loss function for the mask segmentation task header is as follows: ;in For the number of training images, To output the height and width of the mask, For the first The image at the pixel level The real mask label, For the model in pixels The probability of predicting a location as the target category. The loss function is a mask segmentation function; the multi-task joint pre-training module is used for all tasks to share the feature extraction network parameters obtained by the parameter freezing unit; the multi-task head receives the shared features in parallel and outputs the prediction results; the weighted loss function of the multi-task head is jointly calculated by the multi-task joint loss function calculation unit to perform end-to-end training on the overall network; during backpropagation, the gradients of each task work together on the shared backbone network, enabling it to learn a general feature representation that meets the needs of multiple tasks.

9. A two-stage pre-training system for industrial inspection instrument readings according to claim 5, characterized in that: The industrial deployment fine-tuning module includes a field data collection unit, a data annotation unit, a data augmentation unit, a network structure adjustment unit, and a small-sample fine-tuning unit. The field data collection unit is used to collect actual data from industrial instruments in the actual industrial environment where the two-stage pre-training system for industrial inspection instrument readings is applied, and to create a small-sample fine-tuning dataset. The data annotation unit is used to annotate the collected data after applying the field data collection unit. The annotation information includes instrument reading results, pointer position rectangles, and numerical scale position rectangles. The data augmentation unit is used to introduce data augmentation strategies into the small sample fine-tuning dataset to improve the robustness of the model to complex industrial environments. The data augmentation strategies include brightness perturbation, contrast adjustment, random rotation, affine transformation, and local occlusion simulation. The network structure adjustment unit is used to freeze the parameters of the shallow network of the backbone feature extraction network obtained by pre-training after completing the multi-task joint pre-training module, so that the general instrument structure features and visual representation capabilities learned in the pre-training stage remain unchanged during the industrial field deployment stage. Specifically, the shallow network parameters of the ViT and CNN networks are frozen and remain unchanged within the industrial deployment fine-tuning module, and are only used for feature extraction; Remove the spatial regression task head, angle regression task head, and mask segmentation task head from the multi-task joint pre-training module, and retain only the numerical regression task head for final readings; add a new task head for object detection, which, together with the retained numerical regression task head, is fine-tuned on the field data. The few-sample fine-tuning unit is used to train the network model after the network structure adjustment unit on the few-sample fine-tuning dataset. It only fine-tunes the target detection task head, the retained numerical regression task head, and the feature layers directly connected to them, without updating the frozen backbone feature extraction network parameters. This ensures rapid adaptation to specific industrial environments while maintaining the model's generalization ability. A learning rate lower than that used in the pre-training stage is used to optimize and update the trainable parameters to avoid overfitting or severe oscillations in the model parameters under small-sample conditions. Simultaneously, the number of training epochs is dynamically adjusted according to the scale of the field data, and training terminates when the loss function converges or the performance is verified to be stable. The joint loss function of the target detection task head and the numerical regression task head is as follows: ;in For the overall loss function during the fine-tuning stage, The weights of the target detection loss function are... These are the weights of the numerical regression loss function; Let be the loss function for the target detection task header. The loss function for the numerical regression task header includes: ;in The weights are the values ​​for the bounding box regression loss. Bounding box regression loss function The weights for the target confidence loss. Let the target confidence loss function be... Weights for predicting loss by category, For the category prediction loss function, the The implementation method is consistent with that in the multi-task joint pre-training module mentioned above.

Images