A digital multimeter reading recognition method

Abstract

The application relates to the technical field of industrial visual inspection, and discloses a digital multimeter reading recognition method, which comprises the following steps: fixing a to-be-recognized digital multimeter at an industrial monitoring station, so that the display screen of the digital multimeter faces an RGB camera, a depth camera and a six-axis IMU sensor coordinated by a hardware synchronous triggering unit; synchronously collecting and constructing a multi-modal image set aligned in time and space; performing three-level adaptive environment compensation processing on the multi-modal image set to output compensated multi-modal data; calling a lightweight multi-modal time sequence fusion network embedded with a channel and space double attention mechanism, inputting the compensated data for end-to-end recognition, and outputting a positioning frame, recognized reading and confidence; performing post-processing on a heterogeneous computing pipeline accelerated by a CPU multi-thread and a GPU, dynamically adjusting a frame rate, controlling an end-to-end time delay to be less than or equal to 250 milliseconds, and transmitting the final reading to an industrial monitoring terminal. The application can continuously output a reading result with high reliability.

Description

Technical Field

[0001] This application relates to the technical field of industrial visual inspection, and in particular to a method for recognizing digital multimeter readings. Background Technology

[0002] In industrial automation monitoring scenarios, automatic reading recognition of digital multimeters is a crucial step in achieving intelligent perception of equipment status. Current mainstream solutions largely rely on a single RGB vision channel for image acquisition and character recognition, using traditional image processing or basic deep learning models to interpret the readings.

[0003] However, industrial environments are characterized by strong light fluctuations, continuous mechanical vibrations, and dynamic changes in ambient temperature. These characteristics can lead to quality issues in acquired images, such as contrast imbalance, blurred character edges, and coordinate system drift, which significantly affect the stability and reliability of the recognition system.

[0004] As can be seen from the above, the stability of existing identification methods decreases significantly under complex working conditions, making it difficult to continuously output highly reliable reading results. How to achieve continuous output of highly reliable reading results remains to be solved. Summary of the Invention

[0005] In order to continuously output highly reliable reading results, this application provides a digital multimeter reading recognition method.

[0006] Firstly, this application provides a method for recognizing the readings of a digital multimeter, employing the following technical solution: A method for recognizing the readings of a digital multimeter includes: The digital multimeter to be identified is securely fixed to the industrial monitoring station, with its display area facing the RGB camera, depth camera and six-axis IMU sensor coordinated by the hardware synchronization trigger unit. The multimodal image set is synchronously acquired and constructed with spatiotemporal alignment. The multimodal image set includes RGB images, edge gradient images generated by adaptive threshold Canny edge detection, depth images and IMU pose data. The multimodal image set is subjected to a three-level adaptive environment compensation process, and the compensated multimodal data is output. The three-level adaptive environment compensation process includes illumination compensation based on the improved MSRCR algorithm with the introduction of a local contrast enhancement factor, sub-pixel level vibration correction based on IMU attitude data, and thermal drift compensation based on the temperature-pixel offset nonlinear mapping model. A lightweight multimodal temporal fusion network with a pre-trained embedded channel and spatial dual attention mechanism is invoked. The compensated multimodal data is input into the lightweight multimodal temporal fusion network for end-to-end recognition, and the recognition result is output. The recognition result includes the location box of the multimeter digital characters, the recognition reading, and the confidence level. The recognition results are post-processed in real time by a heterogeneous computing parallel pipeline consisting of CPU multi-threading and GPU acceleration, and the final reading is output. The post-processing includes digital sequence verification, decimal point logic determination, unit sign matching and smoothing filtering. Dynamic frame rate adjustment ensures that the end-to-end processing latency is ≤250 milliseconds. The final reading is transmitted in real time to the industrial monitoring terminal for on-site data recording and equipment status monitoring.

[0007] Optionally, the thermal drift compensation further includes: Multiple discrete temperature gradient points are set in a temperature-controlled calibration environment, and sequential image acquisition is performed on the display screen of a multimeter in a fixed state. Extract stable geometric feature points of the display screen from the sequence of images, and calculate the pixel coordinate offset sequence of the geometric feature points under each temperature gradient; A nonlinear mapping model is constructed based on the pixel coordinate offset sequence and the corresponding temperature parameters, and the model parameters are optimized by a time-series smoothing filter algorithm to generate a thermal drift correction parameter set.

[0008] Optionally, the dynamic frame rate adjustment mechanism may further include: The system monitors the confidence sequence of continuous frame recognition results in real time. When the confidence level is continuously higher than the first preset threshold, it automatically switches to a low-power frame rate mode to optimize system resources. When the confidence level fluctuation is lower than the second preset threshold, the spectrum analysis module of the inertial measurement unit sensor data is activated to determine whether there is a vibration interference signal exceeding the preset intensity. If significant vibration interference is detected, the system instantly switches to high-precision frame rate mode, performs time-domain weighted fusion verification of multi-frame recognition results, outputs a fused stable reading, and generates an interference status identifier for system traceability.

[0009] Optionally, in the digital sequence verification step, a logical verification mechanism based on physical constraints is added, including: Obtain the current identification reading and historical time-series reading sequence; By combining prior knowledge of the range boundary of a multimeter and the continuity of changes in physical quantities, a model for judging the rationality of reading evolution is constructed. When the judgment result indicates an abnormal jump, the semantic information of the linkage unit symbol and the relationship between the numerical magnitude are cross-validated. Generate verification confidence markers and feed them back to the recognition process to suppress misidentification outputs caused by non-physical laws.

[0010] Optionally, in the cross-validation process of the semantic information of the unit symbol and the relationship between the numerical magnitude, the method further includes: Extracting 3D spatial structural features of unit symbol regions from depth images; Align the three-dimensional spatial structural features with the symbol candidate regions in the RGB image across modal spaces; Construct a symbol-numerical semantic association rule base, verify the consistency of the matching between unit symbols and recognized numerical values ​​in terms of dimensional logic, dynamically adjust the confidence of the recognition results based on the verification results, and generate semantic verification logs for system traceability.

[0011] Optionally, the deployment process of the lightweight multimodal temporal fusion network integrates an online incremental optimization mechanism, and the method further includes: When a new multimeter interface or environmental interference mode is continuously detected, the incremental sample screening and labeling process is automatically triggered. Implement cross-domain adversarial enhancement and hard-case focused sampling for incremental samples; Incremental knowledge is transferred to existing network core modules to generate an adaptive update model.

[0012] Optionally, the online incremental optimization mechanism and the system health diagnosis module operate in conjunction, and the joint operation also includes: Real-time analysis of multimodal data quality indicators and model output stability parameters; When sensor performance degradation or model generalization ability decline is diagnosed, the incremental optimization process is automatically activated and maintenance recommendations are generated. Synchronously record abnormal contexts and optimization trajectories to construct a joint health record for the device and model; feed the health record back to the incremental sample screening process to form a closed-loop enhancement system of perception, diagnosis and optimization.

[0013] Secondly, this application provides a digital multimeter reading recognition system, which adopts the following technical solution: A digital multimeter reading recognition system includes: The multimodal synchronous acquisition module securely fixes the digital multimeter to be identified to the industrial monitoring station and makes its display screen face the RGB camera, depth camera and six-axis IMU sensor coordinated by the hardware synchronization trigger unit. It synchronously acquires and constructs a spatiotemporally aligned multimodal image set, which includes RGB images, edge gradient images generated by adaptive threshold Canny edge detection, depth images and IMU attitude data. The adaptive environment compensation module performs three-level adaptive environment compensation processing on the multimodal image set and outputs compensated multimodal data. The three-level adaptive environment compensation processing includes illumination compensation based on the improved MSRCR algorithm with the introduction of a local contrast enhancement factor, sub-pixel level vibration correction based on IMU attitude data, and thermal drift compensation based on the temperature-pixel offset nonlinear mapping model. The attention fusion recognition module calls a pre-trained lightweight multimodal temporal fusion network with embedded channel and spatial dual attention mechanisms, inputs the compensated multimodal data into the lightweight multimodal temporal fusion network for end-to-end recognition, and outputs recognition results, including the positioning box of the multimeter digital characters, the recognition reading, and the confidence level. The heterogeneous pipeline processing module performs real-time post-processing on the recognition results through a heterogeneous computing parallel pipeline composed of CPU multi-threading and GPU acceleration, and outputs the final reading. The post-processing includes digital sequence verification, decimal point logic determination, unit sign matching and smoothing filtering, and ensures that the end-to-end processing latency is ≤250 milliseconds through dynamic frame rate adjustment. The final reading is transmitted to the industrial monitoring terminal in real time for on-site data recording and equipment status monitoring.

[0014] Thirdly, this application provides an electronic device that adopts the following technical solution: An electronic device includes a processor, wherein the processor runs a program for the digital multimeter reading identification method described in any one of the preceding claims.

[0015] Fourthly, this application provides a storage medium, which adopts the following technical solution: A storage medium storing a program for the digital multimeter reading recognition method described in any one of the above.

[0016] In summary, this application includes at least one of the following beneficial technical effects: By constructing a spatiotemporally aligned multimodal perception system (integrating RGB vision, depth geometry, and inertial motion information) and introducing edge gradient enhancement features, a clear and interference-resistant input foundation is provided for the recognition process. A three-level adaptive environmental compensation mechanism specifically suppresses image quality degradation caused by sudden changes in illumination, mechanical vibration, and thermal drift, significantly improving the stability of input data in dynamic industrial environments. A lightweight multimodal temporal fusion network, relying on a dual attention mechanism of channels and space, strengthens character region feature focusing and cross-modal temporal correlation, effectively improving recognition robustness in complex backgrounds. Furthermore, a heterogeneous computing pipeline integrates digital logic verification and dynamic resource scheduling strategies, ensuring real-time performance while performing multi-dimensional verification of recognition results, thereby systematically eliminating environmental interference and algorithm misjudgment risks, and achieving continuous high-reliability output of reading results during long-term operation.

[0017] Furthermore, through enhancement mechanisms such as thermal drift calibration models, confidence-linked adaptive frame rate adjustment, logical verification based on physical constraints, semantic verification of unit symbols, online incremental optimization, and joint health diagnosis of devices and models, a full-link reliability assurance system covering data acquisition, identification and processing, and system operation and maintenance has been constructed. This system can not only dynamically adapt to the evolution of new instrument interfaces and environmental interference, but also suppress non-physical jump misidentification through semantic logic verification, and achieve collaborative optimization of models and hardware based on health status feedback, enabling the system to maintain high accuracy, high robustness, and autonomous maintenance capabilities in long-term industrial deployments. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a method for recognizing the readings of a digital multimeter according to an exemplary embodiment.

[0019] Figure 2 This is a flowchart illustrating a DC power supply circuit monitoring scenario in an automotive parts assembly workshop, according to an exemplary embodiment.

[0020] Figure 3 This is a diagram illustrating a comparison of 72-hour multimeter voltage monitoring in an automotive parts assembly workshop, according to an exemplary embodiment.

[0021] Figure 4 This is a diagram illustrating a comparison of 72-hour multimeter voltage monitoring in an automotive parts assembly workshop, according to an exemplary embodiment.

[0022] Figure 5 This is a diagram illustrating the 72-hour maximum meter voltage monitoring in an automotive parts assembly workshop according to an exemplary embodiment.

[0023] Figure 6 This is a structural block diagram of a digital multimeter reading recognition system according to an exemplary embodiment. Detailed Implementation

[0024] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0025] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0026] This application discloses a method for recognizing the readings of a digital multimeter, referring to... Figure 1 ,include: The S100 securely fixes the digital multimeter to be identified to the industrial monitoring station, and makes its display area face the RGB camera, depth camera and six-axis IMU sensor coordinated by the hardware synchronization trigger unit. It synchronously acquires and constructs a spatiotemporally aligned multimodal image set, which includes RGB images, edge gradient images generated by adaptive threshold Canny edge detection, depth images and IMU pose data.

[0027] The detailed execution process of S100 is as follows: S101, Equipment fixing and sensor alignment debugging: First, based on the workstation layout, space dimensions, and multimeter model specifications at the industrial monitoring site, select a suitable dedicated adjustable clamp and place the digital multimeter to be identified stably into the clamp's matching slot. Adjust the clamp's horizontal adjustment knob and vertical limit bracket to ensure the multimeter is level, without tilting or loosening. Simultaneously, clean the multimeter's display screen of dust, stains, and obstructions to prevent glare, blurring, or other issues that could affect data acquisition. This ensures the multimeter will not shift or deviate in posture due to workstation vibrations, equipment operation interference, or other factors throughout the entire industrial monitoring cycle.

[0028] Next, the RGB camera, depth camera, and six-axis IMU sensor are fixed on a pre-set bracket directly in front of the multimeter display. The bracket's height and angle are adjustable to ensure that the field of view of all three sensors completely covers the multimeter display area. The hardware synchronization trigger unit is then activated to collaboratively calibrate the installation position and acquisition angle of the three sensors. The lens orientation of the sensors is adjusted so that the acquisition center points of the three sensors are precisely aligned with the center of the display screen, ensuring that the display screen area is completely facing the lenses of the three sensors. This ensures that the sensors can capture the complete image of the display screen and the device's posture information, avoiding data loss or distortion due to alignment deviations.

[0029] Furthermore, the trigger frequency and synchronization accuracy of the hardware synchronization trigger unit were debugged to ensure that it could accurately coordinate the working timing of the RGB camera, depth camera and six-axis IMU sensor, eliminate the acquisition delay and timing misalignment problems that may occur when each sensor works independently, provide hardware-level protection for the spatiotemporal alignment of subsequent multimodal data, and ensure that the three sensors can start and acquire data synchronously under the same trigger signal.

[0030] S102, Multimodal data synchronous acquisition and spatiotemporal alignment processing: First, after the equipment is fixed and the sensors are aligned, the hardware synchronization trigger unit sends a unified synchronization acquisition trigger signal. After receiving the trigger signal, the RGB camera, depth camera and six-axis IMU sensor start the acquisition work at the same time and enter the continuous acquisition state. The acquisition frequency matches the real-time requirements of industrial monitoring to ensure that the dynamic changes of the multimeter display and the changes of equipment posture can be captured in real time.

[0031] Then, multiple types of data are collected, specifically including: RGB Image Acquisition: The RGB camera captures color images of the multimeter display screen in real time. During the acquisition process, the lens focal length is kept fixed and focused on the digital character area of ​​the display screen to obtain basic visual data such as visual texture information of the display screen, color contrast of digital characters, and character outlines, providing visual support for subsequent character positioning and recognition. Edge gradient image generation: The original RGB image acquired in real time by the RGB camera is immediately executed with the adaptive threshold Canny edge detection algorithm. By dynamically adjusting the high and low thresholds of the Canny algorithm, it automatically adapts to the changes in lighting in the industrial environment, accurately detects the edge contours of the display screen area, the boundary lines between digital characters and the background, generates an edge gradient image, enhances the edge features of digital characters, weakens background interference, and facilitates the subsequent extraction of key features of digital characters. Depth Image Acquisition: The depth camera synchronously acquires depth images of the multimeter display area, captures the spatial depth information of each pixel on the display, clarifies the distance between the display and the sensor, the spatial orientation of the display plane, distinguishes the foreground of the display from background clutter in the industrial site, and provides spatial dimension data support for subsequent background interference elimination and acquisition deviation correction. IMU Attitude Data Acquisition: The six-axis IMU sensor synchronously acquires the device's attitude data, including the three-axis acceleration data and three-axis angular velocity data from the multimeter and the sensor itself. It captures dynamic changes such as minute vibrations and attitude shifts of the device in real time during industrial monitoring, providing attitude reference data for subsequent vibration correction and attitude compensation.

[0032] Furthermore, the spatiotemporal alignment processing and multimodal image set construction involve a hardware synchronization trigger unit that verifies the acquisition timing of each sensor in real time, eliminating acquisition time differences between different sensors and ensuring that RGB images, edge gradient images, depth images, and IMU attitude data acquired at the same time correspond to the same device state. Simultaneously, by combining the spatial depth information of the depth image and the IMU attitude data, the spatial coordinates of the RGB images and edge gradient images are calibrated to correct spatial misalignment caused by sensor installation deviations and minor device displacements, thus achieving spatiotemporal alignment of multimodal data. Finally, all acquired data are integrated to construct a spatiotemporally aligned multimodal image set, which fully includes four core raw data types: RGB images, edge gradient images, depth images, and IMU attitude data.

[0033] By precisely fixing the multimeter, collaboratively calibrating multiple sensors, and synchronously collecting data, spatiotemporal alignment of multimodal data was achieved. This effectively avoided data distortion or loss caused by interference such as vibration, alignment deviation, and timing misalignment in the industrial field. It also enhanced the edge features of digital characters and captured the dynamic posture of the equipment, ensuring the accuracy and real-time performance of subsequent recognition processes.

[0034] S200 performs three-level adaptive environment compensation processing on the multimodal image set and outputs the compensated multimodal data. The three-level adaptive environment compensation processing includes illumination compensation based on the improved MSRCR algorithm with the introduction of a local contrast enhancement factor, sub-pixel level vibration correction based on IMU attitude data, and thermal drift compensation based on the temperature-pixel offset nonlinear mapping model.

[0035] The detailed execution process of S200 is as follows: S201 first preprocesses the spatiotemporally aligned multimodal image set constructed in S100, eliminating invalid data and correcting data format deviations to provide standardized input for the three-level adaptive environment compensation. Specific operations include: Select clear RGB images, edge gradient images, and depth images with no missing frames, and remove blurry or incomplete data caused by sensor momentary failure or sudden occlusion on site; The IMU attitude data (three-axis acceleration and three-axis angular velocity) are converted into standardized attitude parameters by unifying the units. All image data are size-normalized to ensure that the image pixel size and channel format are consistent, while preserving the core features of the numeric characters and avoiding feature distortion during the normalization process. Finally, all preprocessed data are subjected to correlation verification to ensure that the multimodal data in the same time series correspond to each other, and to avoid time series misalignment and data mismatch. After preprocessing, a standardized multimodal data set is output.

[0036] S202, the first-level adaptive environment compensation processing is illumination compensation based on an improved MSRCR algorithm. Addressing issues such as reduced image contrast, blurred digital characters, and weakened edge features caused by unstable lighting in industrial environments (e.g., direct sunlight, low light, local shadows), it executes an improved MSRCR algorithm that incorporates a local contrast enhancement factor to complete illumination compensation. The specific operation is as follows: S2021 performs multi-scale Retinex decomposition on the preprocessed RGB image and edge gradient image. By setting three Gaussian filters of different scales, the low-frequency component (illumination component) and high-frequency component (reflection component) of the image are extracted respectively. The low-frequency component corresponds to the illumination change of the image, and the high-frequency component corresponds to the core features such as digital characters and edge contours.

[0037] S2022 introduces a local contrast enhancement factor on the basis of the traditional MSRCR algorithm. This factor dynamically adjusts the enhancement weight of each pixel by calculating the gray difference and gray mean between each pixel and its neighboring pixels. It focuses on enhancing areas with uneven lighting (such as shadow-covered character areas and areas with strong light reflection) and moderately enhances areas with uniform lighting, avoiding noise amplification caused by excessive enhancement.

[0038] S2023 adjusts the gain of the decomposed high-frequency component (reflection component), and combines it with the local contrast enhancement factor to enhance the grayscale contrast between the digital characters and the background, highlight the edge features of the characters, and suppress image noise at the same time; it smooths the low-frequency component (illuminance component) to eliminate illumination deviations caused by sudden changes in illumination and local strong / weak light, so that the overall illumination intensity of the image tends to be uniform.

[0039] S2024 reconstructs the adjusted high-frequency and low-frequency components to generate an RGB image and edge gradient image after illumination compensation. At the same time, it combines the spatial depth information of the depth image to locally correct the compensated image, ensuring uniform illumination intensity in the display area and further weakening the interference of complex lighting in industrial environments on digital character recognition. Finally, it outputs multimodal image data after illumination compensation.

[0040] S203, the second-level adaptive environment compensation processing is sub-pixel vibration correction based on IMU attitude data. Industrial monitoring equipment operation generates minute vibrations, causing slight attitude shifts in the multimeter body and sensors, resulting in slight blurring and pixel shifts in the acquired images, affecting the positioning and recognition accuracy of digital characters. Therefore, sub-pixel vibration correction based on IMU attitude data is performed. The specific operation is as follows: S2031 retrieves the IMU attitude data (three-axis acceleration and three-axis angular velocity) synchronously acquired by S100, combines it with the preprocessed standardized attitude parameters, and smooths the IMU attitude data using the Kalman filter algorithm to remove abnormal data points caused by vibration noise, and accurately calculates the real-time attitude offset of the multimeter and sensor (including translation offset and rotation offset).

[0041] S2032 maps the calculated attitude offset to the image pixel offset. Through a sub-pixel level interpolation algorithm (bilinear interpolation), pixel-level correction is performed on the RGB image, edge gradient image, and depth image after illumination compensation. This corrects the image offset and blurring caused by vibration, ensuring that the pixel position of the digital characters is accurate and the edge contours are clearly distinguishable.

[0042] S2033 During the calibration process, the spatial depth information of the depth image is used as a reference to ensure that the calibrated image will not have spatial distortion, while preserving the sub-pixel-level features of the digital characters to avoid character deformation caused by over-calibration. After the calibration is completed, the image sharpness is checked. If the sharpness of the calibrated image does not reach the preset threshold, the IMU attitude calculation parameters and interpolation coefficients are readjusted, and the calibration operation is performed again until the requirements are met, and the vibration-calibrated multimodal image data is output.

[0043] S204, the third-level adaptive environment compensation processing is thermal drift compensation based on a temperature-pixel offset nonlinear mapping model. Industrial environments experience significant temperature fluctuations, which can cause thermal drift in sensors (RGB cameras, depth cameras), leading to image pixel shifts and grayscale distortion, affecting the accuracy of digital character recognition. Therefore, thermal drift compensation based on the temperature-pixel offset nonlinear mapping model is performed. The specific operation is as follows: S2041 acquires the sensor's operating temperature data in real time (synchronously acquired with multimodal data), and combines it with the preprocessed multimodal image data to extract the image's pixel offset and grayscale distortion value, establishing the correlation between temperature and pixel offset and grayscale distortion value.

[0044] S2042 calls a pre-trained temperature-pixel offset nonlinear mapping model, inputs the real-time sensor operating temperature into the model, and the model outputs the pixel offset compensation amount and grayscale correction coefficient at the corresponding temperature. This model is trained on a large number of samples under different temperature environments and can accurately adapt to the temperature fluctuation range of industrial sites.

[0045] S2043, based on the compensation parameters output by the model, performs thermal drift correction on the vibration-corrected RGB image, edge gradient image, and depth image, correcting pixel shift and grayscale distortion caused by temperature changes, ensuring stable grayscale values ​​and accurate pixel positions for digital characters.

[0046] S2044 After the correction is completed, the image is checked for grayscale consistency to ensure that the grayscale features and edge features of the digital characters remain consistent under different temperature environments, so as to avoid the interference of temperature fluctuations on subsequent character recognition, and output multimodal image data after thermal drift compensation.

[0047] After completing three levels of adaptive environment compensation, the S205 integrates and verifies the multimodal data after illumination compensation, vibration correction, and thermal drift compensation to ensure that the compensation effect meets the subsequent recognition requirements, and outputs the final compensated multimodal data. Specific operations include: The RGB image, edge gradient image, depth image, and IMU pose data (after synchronous correction) after three levels of compensation are reintegrated to ensure the spatiotemporal alignment of the multimodal data. The integrated data undergoes quality verification, with a focus on verifying the clarity of digital characters, the integrity of edge features, and the accuracy of pixel positions. Invalid data that still does not meet the requirements after compensation is removed. Simultaneously, the quality evaluation indicators of the compensated data (such as image clarity, character contrast, and pixel offset error) are calculated. If the evaluation indicators do not reach the preset threshold, the corresponding compensation step is returned (e.g., if the illumination compensation is not up to standard, return to S202), the compensation parameters are adjusted, and the compensation operation is re-executed. If all data meet the quality requirements, the compensated multimodal data is output for use in the subsequent end-to-end recognition process of S300.

[0048] Through three-level adaptive environmental compensation and full-process operation, it solves the three major interferences in industrial field: unstable lighting, equipment vibration, and temperature fluctuation. It optimizes the quality and stability of multimodal data collected by S100, provides accurate and reliable input for S300 end-to-end identification and S400 real-time post-processing, ensures the high accuracy and real-time performance of the entire identification method, and adapts to the needs of complex industrial monitoring environments.

[0049] S300 invokes a pre-trained lightweight multimodal temporal fusion network with embedded channel and spatial dual attention mechanisms. The compensated multimodal data is input into the lightweight multimodal temporal fusion network for end-to-end recognition, and the recognition results are output. The recognition results include the location box of the multimeter digit characters, the recognition reading, and the confidence level.

[0050] The detailed execution process of S300 is as follows: S301 first initiates the network call program, retrieving a lightweight multimodal temporal fusion network with embedded channel and spatial dual attention mechanisms, pre-trained on an industrial multimeter dataset, to complete network initialization. Specifically, this includes: Load the pre-trained weight parameters and network structure configuration file of the network to ensure that the number of channels, input size, fusion layer parameters and the format and dimension of the compensated multimodal data are completely matched. Initialize the network's forward inference environment, disable the network training mode, enable the inference acceleration mode, reduce redundant computation, and adapt to the real-time recognition needs of industry. The parameters of the core modules of the network (channel attention module, spatial attention module, and temporal fusion module) are verified to ensure that there are no missing or biased parameters in each module, so as to avoid recognition distortion caused by abnormal parameters. Simultaneously, the optimal inference parameters (such as activation function thresholds and feature fusion weights) determined during network training are loaded to ensure that the network can perform the recognition task in the best state. After initialization, the output is a lightweight multimodal temporal fusion network that can be directly used for inference.

[0051] S302 adapts and adjusts the compensated multimodal data output from S205 to ensure it fully matches the input requirements of the lightweight multimodal temporal fusion network, avoiding recognition failures or accuracy degradation due to data format or dimensional mismatches. The specific steps are as follows: The compensated RGB image, edge gradient image, and depth image are channel integrated. According to the channel order preset by the network, the three types of image data are stitched together into multi-channel input data, preserving the core features of each modality. Feature encoding is performed on IMU pose data to transform standardized pose parameters into network-recognizable feature vectors, which are then associated and bound with multi-channel image data to ensure temporal consistency. The integrated multimodal data is adjusted to the network's preset input size, and a bilinear interpolation algorithm is used for size scaling to avoid distortion of numeric character features during the scaling process; Finally, the adapted multimodal data is normalized to adjust the data value range to within the range of network adaptation, suppressing the interference caused by differences in data units. After adaptation, standardized network input data is output.

[0052] S303: The adapted standardized network input data is input into the initialized lightweight multimodal temporal fusion network to start the end-to-end inference process. Through dual attention mechanism and multimodal temporal fusion, the digital character features are accurately extracted and recognition is completed. The specific process is as follows: S3031, Preliminary Multimodal Feature Extraction: The network performs preliminary feature extraction on the input multi-channel image data and IMU pose feature vector through convolutional layers and pooling layers. It extracts the visual texture features of RGB images, the character edge features of edge gradient images, the spatial position features of depth images, and the dynamic features of IMU pose, respectively, to obtain the primary feature maps and feature vectors of each modality.

[0053] S3032, Channel Attention Feature Filtering: The channel attention module filters and enhances the primary features of each modality. This module calculates the importance weight of each feature channel, focuses on strengthening feature channels related to digital character recognition (such as character edge feature channels and visual texture feature channels), weakens irrelevant background feature channels (such as industrial site environment background channels), reduces redundant feature interference, and improves feature recognition efficiency.

[0054] S3033, Spatial Attention Feature Focusing: The spatial attention module performs spatial focusing on the feature map after channel filtering. This module calculates the attention of each pixel in the feature map to accurately locate the spatial area where the digital characters on the multimeter display screen are located, focuses on the pixel area of ​​the digital characters, weakens the interference of the display screen background, border and industrial site background, strengthens the spatial position features and contour features of the digital characters, and provides accurate spatial positioning support for subsequent character recognition.

[0055] S3034, Multimodal Temporal Fusion Processing: The temporal fusion module fuses the features of each modality after dual attention filtering. Combining multimodal data from multiple frames, the features of different modalities and time series are fused according to preset weights to make up for the limitations of single modality and single time series features, improve the integrity and reliability of features, and at the same time reduce the amount of computation and ensure real-time performance through a lightweight fusion algorithm. S3035, End-to-end recognition and inference: The fused comprehensive feature map is input to the fully connected layer and classification layer of the network. The softmax classifier classifies and recognizes the digit characters, while the regression layer predicts the spatial location of the digit characters to complete the end-to-end recognition. No manual intervention is required during the inference process to ensure the automation and efficiency of recognition.

[0056] S304, After the network inference is completed, the preliminary identification results are output. These results are then parsed and filtered to remove outliers and retain valid identification information. Specific operations include: Analyze the network output to extract the coordinates of the multimeter's digital characters (precisely marking the spatial position of each digital character), the digital character recognition result (i.e., the preliminary recognition reading), and the recognition confidence level (reflecting the reliability of the recognition result). Set a confidence threshold (determined by network training to judge the validity of the recognition results) and remove recognition results with confidence below the threshold. These results are usually misrecognitions caused by background interference or feature blurring. Perform a reasonableness check on the positioning box and remove recognition results with positioning boxes that exceed the display area or with abnormal positioning box size (too large or too small) to ensure that the positioning box can accurately surround the digital characters. The initial readings are format-checked, and results that do not conform to the multimeter reading specifications (such as non-numeric characters or abnormal decimal point positions) are removed. After parsing and filtering, a set of valid recognition results is output.

[0057] S305 integrates the filtered valid recognition results, sorts the positioning frames and recognition readings of each digit according to the actual arrangement of the digits on the multimeter display, ensuring that the order of the recognition readings is consistent with the actual arrangement of the digits on the display; it then associates and binds the sorted positioning frames, recognition readings, and corresponding confidence scores to form a complete single-frame recognition result; if there are multiple frames of time-series data, it performs preliminary fusion of the recognition results from multiple frames, taking the recognition reading with the highest confidence score and its corresponding positioning frame as the core recognition result for the current moment, thus improving the stability of the recognition result; finally, it encodes and outputs the recognition result (including positioning frames, recognition readings, and confidence scores) according to the network's preset output format, and transmits it to the subsequent S400 real-time post-processing stage, ensuring that the output result has a standardized format and complete data, facilitating subsequent post-processing operations.

[0058] By employing a dedicated lightweight network and a dual attention mechanism, end-to-end recognition is performed on the compensated multimodal data, accurately outputting the digit character location box, the recognition reading, and the confidence level. At the same time, through lightweight network and inference optimization, real-time recognition is ensured, balancing recognition accuracy and efficiency, providing core recognition support for the automated and high-precision operation of the entire reading recognition method.

[0059] The S400 uses a heterogeneous computing parallel pipeline consisting of CPU multi-threading and GPU acceleration to perform real-time post-processing on the recognition results and output the final reading. The post-processing includes digital sequence verification, decimal point logic determination, unit sign matching and smoothing filtering. Dynamic frame rate adjustment ensures that the end-to-end processing latency is ≤250 milliseconds. The final reading is transmitted in real time to the industrial monitoring terminal for on-site data recording and equipment status monitoring.

[0060] The detailed execution process of S400 is as follows: S401 first initiates a heterogeneous computing parallel pipeline that coordinates CPU multi-threading and GPU acceleration, completing initial configuration to adapt to real-time post-processing requirements. Specific operations include: Configure CPU multi-threading parameters and allocate independent threads according to the computational load of each post-processing stage (verification, judgment, matching, filtering). Lightweight threads are allocated for number sequence verification and decimal point logic judgment, while high-performance threads are allocated for smoothing filtering, so that each stage can be executed in parallel without interference. The GPU acceleration module is activated, and the GPU acceleration kernel is loaded. Computationally intensive tasks such as smoothing filtering and multi-frame data processing are assigned to the GPU for execution. The CPU is responsible for overall scheduling, lightweight verification, and data transmission, forming a collaborative division of labor between the CPU and GPU to reduce redundant calculations. The transmission bandwidth of the pipeline is adjusted to ensure that the recognition results output by the S305 can be quickly transmitted to the pipeline and that the post-processed data can be efficiently output. At the same time, the latency monitoring module is initialized, and the end-to-end processing latency threshold (≤250 milliseconds) is preset for subsequent real-time latency monitoring and adjustment.

[0061] S402 preprocesses the recognition results (location box, recognition reading, confidence level) transmitted by S305, eliminating format deviations and associating time-series data to provide standardized input for subsequent post-processing. Specific operations include: The encoding format of the recognition results is analyzed, and core data (recognition reading string, confidence score value, and location box coordinates) is extracted, and redundant encoding information is removed; the format of the recognition reading string is standardized, the case of characters is unified, and invalid spaces are removed to ensure that the reading format is consistent; The recognition results are associated with multiple frames in time sequence. The readings and confidence scores are sorted according to the acquisition time sequence, and the reading change trend of consecutive frames is marked to provide time sequence support for subsequent smoothing filtering and logical judgment. The preprocessed recognition results are validated a second time to remove suspicious results with confidence levels close to the threshold or slight deviations in the bounding boxes, ensuring that the data input to the post-processing stage is accurate and reliable. After preprocessing, standardized post-processing input data is output.

[0062] S403 addresses potential issues during the S300 recognition process, such as misrecognition and omission of digit characters (e.g., recognizing "8" as "3" or omitting the last digit). It performs a digit sequence verification to correct recognition errors. The specific operation is as follows: S4031, presets the digital sequence rules for multimeter readings (in conjunction with the multimeter model, it clarifies the range of digits for the reading, the reasonable interval between integer and decimal places, such as a maximum of 5 digits for voltage range readings, with 1-2 decimal places).

[0063] S4032 compares the preprocessed recognition reading string with the preset rules to verify the rationality of the number sequence. If the number of digits in the reading exceeds the preset range or non-digit characters appear (except for legal decimal points), it is determined to be an abnormal reading.

[0064] S4033, for abnormal readings, combine the positioning frame coordinates and confidence level under the same time sequence, as well as the identified readings of adjacent frames, to make corrections and supplements (e.g., if adjacent frames are all identified as "12.3V", the current frame is identified as "1.3V", and the confidence level is low, then correct it to "12.3V"); the readings that pass the verification are marked as valid readings, and the readings that fail and cannot be corrected are discarded, and the set of valid readings after digital sequence verification is output.

[0065] S404, based on the actual working principle and reading specifications of a multimeter, performs decimal point logic determination on the valid reading after digital sequence verification to ensure accurate decimal point positioning and avoid reading errors caused by S300 recognition deviation (such as recognizing "12.3" as "123" and "1.23" as "12.3"). Specific operations include: S4041 retrieves the preset decimal point rules for each range of the multimeter (e.g., for the mA range, the decimal point is fixed at the second to last place; for the V range, the decimal point position is either 1 or 2 places depending on the range).

[0066] S4042, combined with the positioning frame coordinates output by S305, determines the spacing between the numbers and characters. If the spacing between adjacent characters exceeds the normal range, it is determined that there is a decimal point (the spacing between the decimal point and the spacing between the numbers and characters is significantly different).

[0067] S4043 integrates information from three sources: the length of the digit sequence, the gear position rule, and the spacing between the positioning frames. It performs logical judgment and correction on the decimal point position. If the recognized reading does not have a decimal point but conforms to the decimal point rule of a certain gear position, a decimal point is added. If the decimal point position does not conform to the gear position rule, the decimal point position is adjusted. After the judgment is completed, the reading with the accurate decimal point position is output.

[0068] The S405, combining the multimeter's operating range and recognition results, performs unit sign matching on the decimal point-determined readings to ensure consistency between the reading and the unit, providing complete and usable reading information for industrial monitoring terminals. The specific operation is as follows: S4051, establish a multimeter unit symbol library, including units corresponding to common ranges (such as V, mV, A, mA, Ω, kΩ, etc.), and associate each unit with its corresponding reading range (e.g., when the unit is mV, the reading range is usually 0-1000).

[0069] S4052 retrieves the multimeter range information associated with the IMU attitude data in S100, or matches the corresponding unit symbol by identifying the numerical range of the reading; the matched unit symbol is associated and bound with the reading. If the identification result already contains the unit symbol, the consistency between the unit symbol and the reading and range is checked. If they are inconsistent, the unit symbol is corrected.

[0070] S4053, if the recognition result does not contain a unit symbol, the corresponding unit symbol will be directly matched and supplemented; at the same time, abnormal readings that are seriously inconsistent with the reading range will be removed (e.g., if the reading is 1200, the matching unit is mV, which is unreasonable and should be corrected to V), and the complete reading with the accurate unit symbol will be output.

[0071] S406 performs smoothing filtering to address reading fluctuations caused by minor disturbances in industrial environments (such as adjacent frame readings being "12.3V", "12.5V", "12.2V"), ​​eliminating reading jitter and improving the stability of the final reading. The specific operation is as follows: The S4061 employs a moving average filtering algorithm, setting the filter window size (based on the acquisition frequency, typically 3-5 frames) to perform a moving average calculation on complete readings with units across multiple time frames. During the filtering process, abnormal fluctuation readings within the filter window (such as readings whose difference from the average reading within the window exceeds a preset threshold) are removed to prevent abnormal readings from affecting the filtering results.

[0072] S4062 performs precision correction on the filtered reading, retaining the same number of decimal places as the multimeter reading, ensuring that the filtered reading is both stable and accurate without precision distortion; after filtering, it outputs a smooth and stable filtered reading.

[0073] The S407 monitors the end-to-end processing latency of the entire identification process in real time, ensuring a latency of ≤250 milliseconds through dynamic frame rate adjustment to meet the real-time requirements of industrial monitoring. Specific operations include: S4071 uses a latency monitoring module to collect the entire processing time from S100 data acquisition to S406 smoothing filtering in real time, and compares it with the preset 250 millisecond threshold. S4072 If the detected latency is close to the threshold (e.g., ≥220 milliseconds), it will automatically reduce the frame rate of data acquisition and post-processing, reduce the amount of data processed per frame, prioritize the efficiency of the core post-processing links (verification, filtering), and avoid untimely reading updates caused by excessively low frame rate. S4073 If the latency is much lower than the threshold (e.g., ≤180 milliseconds), the frame rate will be appropriately increased to improve the reading update frequency and ensure that the dynamic changes of the multimeter reading can be captured in real time. During the dynamic adjustment process, the accuracy and stability of the reading are checked in real time to avoid the decrease in accuracy caused by the frame rate adjustment, and the end-to-end processing latency is always maintained within the range of ≤250 milliseconds.

[0074] S408 performs a final verification on the stable readings after smoothing and filtering. After confirming that the readings are accurate, the units are correct, and there are no abnormal fluctuations, it encodes the final readings according to the adaptation format of the industrial monitoring terminal and unifies the data transmission format (such as JSON format), including core information such as reading value, unit symbol, acquisition time, and confidence level. It then starts the data transmission module and uses industrial Ethernet or wireless transmission protocols to transmit the encoded final readings to the industrial monitoring terminal in real time.

[0075] During transmission, data transmission verification is performed. If transmission interruption or data loss occurs, a retransmission mechanism is immediately initiated to ensure the reliability of data transmission. At the same time, the final reading is synchronously stored in the local cache for subsequent data traceability and anomaly investigation. After receiving the data, the industrial monitoring terminal completes on-site data recording and real-time monitoring of equipment status, and the entire S400 process is completed.

[0076] Real-time post-processing of recognition results is achieved through a heterogeneous computing parallel pipeline, correcting misidentification and deviation issues in the S300 recognition process. Multi-stage verification and filtering ensure the accuracy and stability of the final reading. At the same time, dynamic frame rate adjustment strictly controls the end-to-end latency to ≤250 milliseconds, transmitting the final reading to the industrial monitoring terminal in real time. This connects the front-end recognition process with the back-end industrial applications, enabling automated, real-time, and accurate output of multimeter readings. It provides reliable data support for on-site data recording and equipment status monitoring, ensuring the practicality and industrial adaptability of the entire recognition method.

[0077] In the embodiments of this application, reference is made to Figure 2 The scenario of monitoring the DC power supply circuit in an automotive parts assembly workshop is selected. This workshop has multiple stamping machines and assembly robots operating synchronously, and there are typical industrial interferences such as machine tool vibration, direct sunlight at noon, weak light in the evening, and workshop temperature fluctuations of 18-32℃. The technical solution in this application requires continuous monitoring of the digital multimeter at the monitoring station for 72 hours, continuously outputting the power supply circuit voltage reading (range 0-30V, accuracy ±0.01V) to ensure stable monitoring of the equipment power supply status.

[0078] During implementation, preparation work was first carried out according to the S100 steps. A special adjustable fixture was selected according to the workshop workstation layout. The multimeter was fixed on a bracket away from the vibration transmission of the machine tool. After cleaning the oil stains on the surface of the display screen, the RGB camera, depth camera and six-axis IMU sensor were fixed 30cm in front of the display screen. The alignment and calibration of the three were completed through the hardware synchronous trigger unit to ensure that the acquisition center point coincides with the center of the display screen. The trigger frequency was adjusted to 10 frames / second. RGB images, adaptive threshold Canny edge gradient images, depth images and IMU attitude data were acquired synchronously. Multimodal image sets were constructed through spatiotemporal alignment processing, which effectively avoided problems such as sensor offset and timing misalignment caused by machine tool vibration, and provided high-quality raw data for subsequent processing.

[0079] For complex interference in the workshop, data optimization is achieved through S200 three-level adaptive environmental compensation: When the screen is exposed to strong sunlight at noon, local reflections occur, causing the characters to become blurry. Based on the improved MSRCR algorithm that introduces a local contrast enhancement factor, illumination compensation is performed on the RGB image and edge gradient image to dynamically enhance the character contrast in the reflective area and weaken the light interference. The slight vibrations generated by the machine tool operation cause a slight image shift. The IMU attitude data is retrieved through step S203. After removing vibration noise through Kalman filtering, the attitude shift is calculated and corrected by sub-pixel bilinear interpolation to ensure accurate character pixel position. When the workshop temperature fluctuates, the sensor experiences slight thermal drift. The temperature-pixel offset nonlinear mapping model is invoked. Based on the real-time sensor temperature, compensation parameters are output to correct pixel offset and grayscale distortion. After integration and verification by S205, the compensated multimodal data that meets the requirements is output. After ensuring that the data quality meets the standards, it is passed to step S300.

[0080] In step S300, a lightweight multimodal temporal fusion network with an embedded dual attention mechanism is initialized. The compensated data is adapted to the network input format. Character-related features are filtered through channel attention, and spatial attention is used to focus on the character area of ​​the display screen. Multi-frame temporal fusion is combined to improve feature integrity. After end-to-end inference, recognition results with a confidence level ≥ 0.95 are selected, which effectively avoids problems such as misidentifying "8.52V" as "3.52V" and missing the last "0". The system initially outputs accurate recognition readings and positioning box information.

[0081] Finally, the S400 process completes the post-processing and real-time transmission, ensuring continuous and reliable readings. Computational resource scheduling: Start CPU multi-threading and GPU accelerated heterogeneous computing pipeline, allocate lightweight threads for digital sequence verification and decimal point logic determination, and allocate high-performance GPU threads for smoothing filtering to ensure parallel processing efficiency; Digital sequence verification: After preprocessing, the recognition results are corrected by digital sequence verification to correct the deviation of "8.5V" being misidentified as "8.50V". Combined with the multimeter voltage range rules and the positioning frame spacing, the decimal point logic judgment is completed to avoid "85.2V" being misjudged as "8.52V". The "V" unit is bound by unit symbol matching to eliminate abnormal data where the unit does not match the reading range. Smoothing Filtering and Fluctuation Suppression: A moving average filtering algorithm with a window size of 5 frames is used to eliminate reading fluctuations caused by machine tool vibration (such as fluctuations of 8.51V, 8.49V, and 8.50V), and output smooth and stable readings.

[0082] Reference Figure 3 ,in, Figure 3 A comparison of 72-hour multimeter voltage monitoring in an automotive parts assembly workshop. Figure 3 This diagram shows a comparison between the original fluctuating readings (blue curve) and the stable readings (red curve) after smoothing and filtering under workshop environmental interference. It can be seen that the original readings exhibit high-frequency, small-amplitude fluctuations due to vibration, temperature, and other disturbances. However, after moving average filtering, the reading curves become smooth and stable, effectively suppressing noise interference and providing a reliable basis for subsequent condition assessment.

[0083] By dynamically adjusting the frame rate, the end-to-end processing latency is monitored in real time to ensure that the latency is stable within the range of 220-240 milliseconds. Finally, the encoded readings (including numerical value, unit, acquisition time, and confidence level) are transmitted to the workshop monitoring terminal via industrial Ethernet and simultaneously stored in the local cache.

[0084] Reference Figure 4 , Figure 4 The 72-hour multimeter voltage monitoring comparison in the automotive parts assembly workshop (Figure 4) demonstrates the monitoring effect under strong periodic interference (such as periodic vibration of the stamping machine). The original fluctuating reading (blue curve) showed periodic large spikes, while the stable reading after smoothing and filtering (orange curve) effectively filtered out these spike interferences, maintaining small fluctuations around the reference value, verifying the filtering algorithm's ability to suppress periodic interference.

[0085] During this 72-hour continuous monitoring, a total of 34,560 readings were output, with an effective data rate of 99.8%. The reading errors were all controlled within ±0.01V, with no missed or false alarms. This fully demonstrates that the technical solution proposed in this application can address various interferences in the industrial field through coordinated efforts across the entire process, continuously outputting highly reliable multimeter readings to meet the real-time monitoring needs of industry.

[0086] Reference Figure 5 , Figure 5 For 72-hour voltage monitoring in automotive parts assembly workshops, Figure 5This demonstrates the consistency between the original fluctuating readings (blue curve) and the stable readings after smoothing and filtering (orange curve) in a long-term monitoring scenario where voltage rises slowly. It can be seen that the filtered readings accurately follow the overall upward trend of the original readings while eliminating high-frequency noise. This ensures both the accuracy of the trend and the stability of the readings, meeting the needs of long-term trend monitoring.

[0087] In this embodiment of the application, the specific execution process of thermal drift compensation is as follows: Step 1: Set up a temperature-controlled calibration environment and acquire sequence images: An industrial-grade temperature-controlled calibration chamber was selected to build a closed, vibration-free, and light-stable calibration environment to avoid external interference. In combination with the actual temperature fluctuation range of 18-32℃ in industrial sites, multiple discrete temperature gradient points were set within this range to ensure coverage of all working conditions.

[0088] The multimeter is fixed in the calibration box with a special clamp to keep its posture and position consistent with the on-site monitoring and without displacement; after the temperature stabilizes at each temperature gradient point, the RGB camera and depth camera are controlled to synchronously acquire a sequence of images, acquiring 20-30 clear images for each gradient, with the acquisition frequency consistent with the on-site (10 frames / second) to ensure that the calibration data is consistent with reality.

[0089] Step 2: Extract stable geometric feature points and calculate the pixel coordinate offset sequence: Invalid images were removed from the sequence images of each temperature gradient, and clear samples were selected. Using an image feature extraction algorithm, the four corner points of the display screen and the internal fixed marker points were selected as stable geometric feature points. Based on the pixel coordinates of the feature points corresponding to 25℃ (standard temperature), the pixel coordinate offsets of the same feature points under other temperature gradients were calculated and integrated to form a pixel coordinate offset sequence corresponding to each temperature, providing basic data for model construction.

[0090] Step 3: Construct a nonlinear mapping model, optimize parameters, and generate a set of correction parameters. Each temperature parameter and its corresponding pixel coordinate offset sequence are imported into the data processing unit. A nonlinear mapping model between temperature and pixel offset is constructed using a nonlinear fitting algorithm (such as polynomial fitting) to characterize the thermal drift law of the sensor.

[0091] The model parameters are optimized by using a time-series smoothing filtering algorithm to remove abnormal data, smooth fluctuations, and improve model accuracy. Based on the temperature correction requirements, a thermal drift correction parameter set including pixel offset compensation and grayscale correction coefficients is generated and pre-stored in the system for real-time use in step S204 on site.

[0092] By performing pre-temperature calibration and model building, an accurate set of thermal drift correction parameters is generated, providing reliable support for real-time thermal drift compensation on site. This ensures that the correction parameters are adapted to temperature fluctuations, improves compensation accuracy, avoids reading distortion, and guarantees the reliability of multimeter reading recognition.

[0093] In this embodiment of the application, the method in the dynamic frame rate adjustment mechanism specifically includes: Step 1: Switch to low-power frame rate mode based on confidence sequence. Monitor the confidence sequence of consecutive frame recognition results in real time. A first preset threshold (set to 0.95 based on the recognition standard mentioned earlier) is used. When the confidence level of 5-8 consecutive frames remains above this threshold, it indicates that the current recognition environment is stable and the readings are reliable. The system automatically switches to low-power frame rate mode, appropriately reducing the acquisition and inference frame rate (e.g., from 10 frames / second to 5 frames / second). This optimizes system computing power and energy consumption without affecting reading accuracy.

[0094] Step 2: Determine vibration interference based on confidence level fluctuations. A second preset threshold (e.g., 0.05) is set. When the confidence level fluctuation amplitude of consecutive frames is lower than this threshold, it indicates that the readings may be affected by latent vibration interference. At this time, the spectrum analysis module of the six-axis IMU sensor data is activated to perform spectrum analysis on the acceleration and angular velocity data collected by the IMU to determine whether there is a vibration interference signal exceeding the preset intensity.

[0095] Step 3: Switch to high-precision frame rate and perform fusion verification under vibration interference. If the spectrum analysis detects a significant vibration interference signal, the system instantly switches to high-precision frame rate mode (e.g., from 5 frames / second to 15 frames / second), and simultaneously performs time-domain weighted fusion verification of the multi-frame recognition results. The system calculates the stable reading after fusion according to the confidence level and generates an interference status identifier for subsequent traceability and investigation.

[0096] By dynamically adjusting the frame rate in conjunction with confidence level and vibration interference, the system balances resource optimization and reading stability. This reduces power consumption when identification is stable and improves accuracy when vibration interference occurs, ensuring continuous and reliable readings in complex industrial scenarios.

[0097] In this embodiment of the application, a logical verification mechanism based on physical law constraints is added to the digital sequence verification step, specifically including: In actual execution, the current digital sequence verification reading is first synchronously acquired, along with the historical time-series reading sequence of recent consecutive frames (such as the last 10 valid readings) to ensure the integrity of the time-series data. Combining the actual range boundaries of the multimeter (such as the voltage range of 0-30V in this example) and the prior knowledge of the continuity of physical quantity changes in the industrial field (such as the voltage and current of the power supply circuit will not experience instantaneous large jumps), a reading evolution rationality judgment model is constructed, and a reasonable reading jump threshold is preset (such as the voltage range jump not exceeding 5V).

[0098] When the discrimination model detects an abnormal jump in the current reading relative to the historical time series (e.g., a sudden change from 8.5V to 28V, exceeding the jump threshold), it immediately performs cross-validation by linking the semantic information of the unit symbol with the magnitude of the numerical value to determine whether the abnormal jump is caused by unit misjudgment (e.g., misjudging mV as V) or numerical misidentification. After verification, a verification confidence flag (qualified / abnormal) is generated and fed back to the entire recognition process. If the flag is abnormal, the output of the misidentified reading is suppressed, and a re-verification is triggered simultaneously.

[0099] By constraining with physical laws and cross-validating, the limitations of simple numerical sequence rule verification are overcome, further filtering out misidentified readings caused by non-physical laws and improving the accuracy of numerical sequence verification.

[0100] In this embodiment of the application, the specific execution process of the method in the cross-validation of the relationship between the semantic information of the unit symbol and the numerical magnitude is as follows: In actual execution, the current digital sequence verification reading is first synchronously acquired, along with the historical time-series reading sequence of recent consecutive frames (such as the last 10 valid readings) to ensure the integrity of the time-series data. Combining the actual range boundaries of the multimeter (such as the voltage range of 0-30V in the previous example) and the prior knowledge of the continuity of physical quantity changes in the industrial field (such as the voltage and current of the power supply circuit will not experience instantaneous large jumps), a reading evolution rationality judgment model is constructed, and a reasonable reading jump threshold is preset (such as the voltage range jump not exceeding 5V).

[0101] When the discriminant model detects an abnormal jump in the current reading relative to the historical time series (such as a sudden change from 8.5V to 28V, exceeding the jump threshold), it immediately performs cross-validation by linking the semantic information of the unit sign with the magnitude of the numerical value. This cross-validation supplementary process was not explained in the previous text and is executed as follows: Retrieve the depth image acquired by S100, focus on the region where the unit symbol is located, and extract its three-dimensional spatial structure features such as three-dimensional spatial contour and pixel depth distribution; perform cross-modal space alignment between the three-dimensional spatial structure features and the symbol candidate regions identified in the RGB image to ensure that the two correspond to the same unit symbol and avoid mismatch of symbol regions; A symbol-numerical semantic association rule library is pre-built to store the numerical magnitude range corresponding to each unit symbol (such as V, mV, A). The rule library is used to verify the consistency of the current unit symbol and the identified numerical value in terms of dimensional logic (e.g., the numerical value 1200 does not match the unit mV, but matches V). The system dynamically adjusts the confidence level of the recognition result based on the verification results (increasing confidence level if there is a match, decreasing confidence level and marking it as abnormal if there is no match). At the same time, a semantic verification log is generated to record the verification process and the reasons for the abnormality for subsequent traceability and investigation by the system. After verification is completed, a verification confidence flag (pass / abnormal) is generated and fed back to the entire recognition process. If it is an abnormal flag, the output of the misidentified reading is suppressed and a re-verification is triggered simultaneously.

[0102] In this embodiment of the application, cross-modal feature alignment and semantic rule verification are used to avoid misjudgment of units and misidentification caused by mismatch between symbols and values. At the same time, it provides a basis for system anomaly tracing and further improves the reliability of digital sequence verification.

[0103] In this embodiment, the deployment process of the lightweight multimodal temporal fusion network integrates an online incremental optimization mechanism. The specific execution process of the method is as follows: In actual deployment, the system continuously monitors the confidence fluctuations, character positioning deviations and environmental parameter changes during the network recognition process. When a new type of multimeter interface (such as different display layouts and character styles) or an unprecedented environmental interference mode (such as special lighting or compound vibration) is detected, and such situations occur continuously for 3-5 frames, the incremental sample screening and labeling process is automatically triggered.

[0104] During the screening process, the system automatically retains samples with low recognition confidence and abnormal feature matching, and removes fuzzy and invalid samples. At the same time, it quickly completes the annotation of key features of the samples (such as character regions, unit symbols, and interference types). Subsequently, cross-domain adversarial enhancement processing is implemented on the incremental samples after screening and annotation to simulate feature changes under different industrial environments. At the same time, a hard case focusing sampling strategy is adopted to focus on sampling samples that are easy to misidentify and have indistinct features, thereby improving the representativeness of the incremental samples.

[0105] Finally, the new knowledge contained in the incremental samples is transferred to the existing network's channel attention, spatial attention, and temporal fusion core modules. By using a lightweight parameter update algorithm to adjust the module weights, an adaptive update model is generated without retraining the entire network, ensuring update efficiency.

[0106] Based on the above steps, online adaptive optimization of the network can be achieved, adapting to new multimeter interfaces and unknown environmental interference without manual intervention, reducing network performance degradation, and ensuring the continuous stability of network identification accuracy in long-term industrial monitoring, thereby improving the scenario adaptability and practicality of the entire technical solution.

[0107] In this embodiment, the online incremental optimization mechanism and the system health diagnosis module operate in concert, and the specific concerted execution process is as follows: During actual operation, the system health diagnosis module works in tandem with the online incremental optimization mechanism to analyze in real time the quality indicators of the multimodal data collected by the S100 (such as image clarity and feature integrity) and the output stability parameters of the lightweight multimodal temporal fusion network (such as confidence fluctuation and recognition deviation rate), thereby achieving synchronous monitoring of sensor status and model performance. When the diagnosis module detects sensor performance degradation (such as blurred images or data distortion) or a decrease in model generalization ability (such as increased false recognition rate or reduced adaptability), it automatically activates the online incremental optimization process without triggering manual intervention. At the same time, it generates targeted maintenance suggestions (such as sensor cleaning and parameter fine-tuning prompts) based on the diagnosed anomaly type.

[0108] It should be noted that during this process, the system synchronously records the context information (such as environmental parameters and abnormal behavior) and the entire optimization trajectory (such as sample screening and parameter update details) when the anomaly occurs, constructs a device-model joint health record, and clearly retains the operational status data of the sensors and the model; finally, the health record is fed back to the incremental sample screening stage, focusing on screening samples related to health anomalies, forming a closed-loop enhancement system of "perception-diagnosis-optimization" to achieve two-way linkage improvement.

[0109] By linking health diagnosis and incremental optimization, performance anomalies of sensors and models can be detected in advance, proactively triggering optimization and providing maintenance guidance. At the same time, the optimization effect is continuously strengthened through a closed-loop system, further ensuring the long-term stable operation of the system, reducing manual maintenance costs, and improving the reliability and long-term effectiveness of the entire recognition solution.

[0110] This application discloses a method for recognizing the readings of a digital multimeter, referring to... Figure 1 ,include: The multimodal synchronous acquisition module 001 securely fixes the digital multimeter to be identified to the industrial monitoring station and makes its display area face the RGB camera, depth camera and six-axis IMU sensor coordinated by the hardware synchronization trigger unit. It synchronously acquires and constructs a spatiotemporally aligned multimodal image set, which includes RGB images, edge gradient images generated by adaptive threshold Canny edge detection, depth images and IMU pose data. The adaptive environment compensation module 002 performs three-level adaptive environment compensation processing on the multimodal image set and outputs the compensated multimodal data. The three-level adaptive environment compensation processing includes illumination compensation based on the improved MSRCR algorithm with the introduction of a local contrast enhancement factor, sub-pixel level vibration correction based on IMU attitude data, and thermal drift compensation based on the temperature-pixel offset nonlinear mapping model. The attention fusion recognition module 003 calls a pre-trained lightweight multimodal temporal fusion network with embedded channel and spatial dual attention mechanisms, inputs the compensated multimodal data into the lightweight multimodal temporal fusion network for end-to-end recognition, and outputs the recognition results, including the positioning box of the multimeter digit characters, the recognition reading, and the confidence level. The heterogeneous pipeline processing module 004 performs real-time post-processing of the recognition results through a heterogeneous computing parallel pipeline composed of CPU multi-threading and GPU acceleration, and outputs the final reading. The post-processing includes digital sequence verification, decimal point logic determination, unit sign matching and smoothing filtering. Dynamic frame rate adjustment ensures that the end-to-end processing latency is ≤250 milliseconds. The final reading is transmitted to the industrial monitoring terminal in real time for on-site data recording and equipment status monitoring.

[0111] This application also discloses an electronic device, including a processor, wherein the processor runs a program for the digital multimeter reading recognition method described in any one of the above embodiments.

[0112] This application also discloses a storage medium storing a program for the digital multimeter reading recognition method described in any one of the above embodiments.

[0113] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for recognizing the readings of a digital multimeter, characterized in that, include: The digital multimeter to be identified is securely fixed to the industrial monitoring station, with its display area facing the RGB camera, depth camera and six-axis IMU sensor coordinated by the hardware synchronization trigger unit. The multimodal image set is synchronously acquired and constructed with spatiotemporal alignment. The multimodal image set includes RGB images, edge gradient images generated by adaptive threshold Canny edge detection, depth images and IMU pose data. The multimodal image set is subjected to a three-level adaptive environment compensation process, and the compensated multimodal data is output. The three-level adaptive environment compensation process includes illumination compensation based on the improved MSRCR algorithm with the introduction of a local contrast enhancement factor, sub-pixel level vibration correction based on IMU attitude data, and thermal drift compensation based on the temperature-pixel offset nonlinear mapping model. A lightweight multimodal temporal fusion network with a pre-trained embedded channel and spatial dual attention mechanism is invoked. The compensated multimodal data is input into the lightweight multimodal temporal fusion network for end-to-end recognition, and the recognition result is output. The recognition result includes the location box of the multimeter digital characters, the recognition reading, and the confidence level. The recognition results are post-processed in real time by a heterogeneous computing parallel pipeline consisting of CPU multi-threading and GPU acceleration, and the final reading is output. The post-processing includes digital sequence verification, decimal point logic determination, unit sign matching and smoothing filtering. Dynamic frame rate adjustment ensures that the end-to-end processing latency is ≤250 milliseconds. The final reading is transmitted in real time to the industrial monitoring terminal for on-site data recording and equipment status monitoring.

2. The digital multimeter reading recognition method according to claim 1, characterized in that, Thermal drift compensation also includes: Multiple discrete temperature gradient points are set in a temperature-controlled calibration environment, and sequential image acquisition is performed on the display screen of a multimeter in a fixed state. Extract stable geometric feature points of the display screen from the sequence of images, and calculate the pixel coordinate offset sequence of the geometric feature points under each temperature gradient; A nonlinear mapping model is constructed based on the pixel coordinate offset sequence and the corresponding temperature parameters, and the model parameters are optimized by a time-series smoothing filter algorithm to generate a thermal drift correction parameter set.

3. The digital multimeter reading recognition method according to claim 1, characterized in that, In dynamic frame rate adjustment mechanisms, the methods also include: The system monitors the confidence sequence of continuous frame recognition results in real time. When the confidence level is continuously higher than the first preset threshold, it automatically switches to a low-power frame rate mode to optimize system resources. When the confidence level fluctuation is lower than the second preset threshold, the spectrum analysis module of the inertial measurement unit sensor data is activated to determine whether there is a vibration interference signal exceeding the preset intensity. If significant vibration interference is detected, the system instantly switches to high-precision frame rate mode, performs time-domain weighted fusion verification of multi-frame recognition results, outputs a fused stable reading, and generates an interference status identifier for system traceability.

4. The digital multimeter reading recognition method according to claim 1, characterized in that, In the digital sequence verification step, a logical verification mechanism based on physical law constraints is added, including: Obtain the current identification reading and historical time-series reading sequence; By combining prior knowledge of the range boundary of a multimeter and the continuity of changes in physical quantities, a model for judging the rationality of reading evolution is constructed. When the judgment result indicates an abnormal jump, the semantic information of the linkage unit symbol and the relationship between the numerical magnitude are cross-validated. Generate verification confidence markers and feed them back to the recognition process to suppress misidentification outputs caused by non-physical laws.

5. The digital multimeter reading recognition method according to claim 4, characterized in that, In the cross-validation process of the relationship between the semantic information of unit symbols and numerical magnitudes, the method also includes: Extracting 3D spatial structural features of unit symbol regions from depth images; Align the three-dimensional spatial structural features with the symbol candidate regions in the RGB image across modal spaces; Construct a symbol-numerical semantic association rule base, verify the consistency of the matching between unit symbols and recognized numerical values ​​in terms of dimensional logic, dynamically adjust the confidence of the recognition results based on the verification results, and generate semantic verification logs for system traceability.

6. The digital multimeter reading recognition method according to claim 1, characterized in that, The deployment process of lightweight multimodal temporal fusion networks integrates an online incremental optimization mechanism, and the method also includes: When a new multimeter interface or environmental interference mode is continuously detected, the incremental sample screening and labeling process is automatically triggered. Implement cross-domain adversarial enhancement and hard-case focused sampling for incremental samples; Incremental knowledge is transferred to existing network core modules to generate an adaptive update model.

7. The digital multimeter reading recognition method according to claim 6, characterized in that, The online incremental optimization mechanism and the system health diagnosis module work together, and this collaboration also includes: Real-time analysis of multimodal data quality indicators and model output stability parameters; When sensor performance degradation or model generalization ability decline is diagnosed, the incremental optimization process is automatically activated and maintenance recommendations are generated. Synchronously record abnormal contexts and optimization trajectories to construct a joint health record for the device and model; feed the health record back to the incremental sample screening process to form a closed-loop enhancement system of perception, diagnosis and optimization.

8. A digital multimeter reading recognition system, characterized in that, include: The multimodal synchronous acquisition module securely fixes the digital multimeter to be identified to the industrial monitoring station and makes its display screen face the RGB camera, depth camera and six-axis IMU sensor coordinated by the hardware synchronization trigger unit. It synchronously acquires and constructs a spatiotemporally aligned multimodal image set, which includes RGB images, edge gradient images generated by adaptive threshold Canny edge detection, depth images and IMU attitude data. The adaptive environment compensation module performs three-level adaptive environment compensation processing on the multimodal image set and outputs compensated multimodal data. The three-level adaptive environment compensation processing includes illumination compensation based on the improved MSRCR algorithm with the introduction of a local contrast enhancement factor, sub-pixel level vibration correction based on IMU attitude data, and thermal drift compensation based on the temperature-pixel offset nonlinear mapping model. The attention fusion recognition module calls a pre-trained lightweight multimodal temporal fusion network with embedded channel and spatial dual attention mechanisms, inputs the compensated multimodal data into the lightweight multimodal temporal fusion network for end-to-end recognition, and outputs recognition results, including the positioning box of the multimeter digital characters, the recognition reading, and the confidence level. The heterogeneous pipeline processing module performs real-time post-processing on the recognition results through a heterogeneous computing parallel pipeline composed of CPU multi-threading and GPU acceleration, and outputs the final reading. The post-processing includes digital sequence verification, decimal point logic determination, unit sign matching and smoothing filtering, and ensures that the end-to-end processing latency is ≤250 milliseconds through dynamic frame rate adjustment. The final reading is transmitted to the industrial monitoring terminal in real time for on-site data recording and equipment status monitoring.

9. An electronic device, characterized in that, Includes a processor, wherein the processor runs a program for the digital multimeter reading identification method as described in any one of claims 1-7.

10. A storage medium, characterized in that, The program stores the digital multimeter reading identification method as described in any one of claims 1-7.

Images