Instrument identification method for industrial robot inspection

Abstract

The invention belongs to the technical field of industrial robot inspection, and discloses an instrument identification method for industrial robot inspection, an instrument identification system comprises a pose adjustment module, a lightweight detection module, an image double enhancement processing module and a double-branch instrument reading identification module, and the instrument identification method comprises the following steps: obtaining an instrument image, inputting the instrument image into an image dual-enhancement processing module to obtain a preprocessed instrument image; the preprocessed instrument image is input into a lightweight detection module, an instrument target detection result is output, the instrument type is judged according to the result, and an instrument reading result is output and recorded. According to the method, the improved lightweight target detection model and the adaptive image optimization processing algorithm are combined, the real-time performance and the high efficiency of instrument reading in an industrial scene are effectively improved, more intelligent and accurate instrument reading and industrial inspection can be provided, and powerful support is provided for information acquisition in a complex industrial environment.

Description

Technical Field

[0001] This invention belongs to the field of industrial robot inspection technology, specifically relating to an instrument identification method for industrial robot inspection. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing technologies, industrial inspection has gradually transformed from traditional manual inspection to intelligent and unmanned operation, and is widely used in key fields such as power systems, petrochemicals, and energy metallurgy, undertaking core tasks such as instrument status monitoring and equipment fault early warning. However, in the complex environment of industrial sites, instrument detection and reading recognition face many technical challenges, such as perspective distortion caused by the shooting angle of industrial cameras, image quality fluctuations caused by uneven lighting in workshops, missed detections and false detections of small targets such as instrument pointers and scales in complex backgrounds, and the excessive computational load of traditional recognition algorithms, which cannot meet the real-time requirements of edge devices. These problems seriously restrict the automation level and reliability of industrial inspection.

[0003] Chinese patent application ZL2024118423199 discloses an information processing method, device, equipment, and storage medium for an inspection robot. This patent collects equipment images during the robot's mobile inspection process, corrects and identifies the instrument area images, and performs equipment self-inspection based on the instrument information. However, this technology lacks adaptive processing capabilities for complex uneven lighting and perspective distortion in industrial scenes during image preprocessing. It relies solely on conventional region segmentation and correction strategies, making it difficult to cope with severe geometric distortion caused by tilted shooting or image quality degradation under strong or backlight conditions in workshops. Furthermore, it does not design a dedicated recognition branch for the differentiated characteristics of pointer and digital instruments. When using a single network architecture to process different types of instruments, it is prone to problems such as large pointer angle regression errors and low recognition rates of digital characters in blurred scenes.

[0004] Chinese patent application ZL2024116225437 discloses a method and system for detecting and recognizing pointer instrument readings based on an improved YOLOv8n. This patent acquires pointer instrument image data and sequentially performs data augmentation and dial annotation, improving the recognition efficiency and accuracy of pointer instrument readings. However, this patent only designs detection and recognition schemes for pointer instruments and cannot be compatible with the reading analysis of digital instruments, resulting in significant limitations in industrial inspection scenarios with mixed instruments. Although its improved YOLOv8n network optimizes the dial detection process, it does not introduce lightweight convolutional units and hybrid attention mechanisms, resulting in high model computational complexity. This makes it difficult to directly deploy on edge devices such as industrial inspection robots, and it lacks targeted optimization for the deformation characteristics of small targets such as pointers, making it prone to missed detections in complex backgrounds.

[0005] In summary, existing industrial instrument detection and inspection technologies face multiple technical bottlenecks in complex industrial scenarios: traditional detection models have high computational complexity, making them difficult to adapt to the real-time requirements of edge devices, resulting in high rates of missed and false detections for small targets such as pointers and scales; image preprocessing uses fixed parameter strategies, failing to adaptively address issues such as perspective distortion and uneven lighting; single network architectures lack sufficient accuracy for mixed recognition of pointer and digital instruments, leading to significant reading errors in blurry or low-contrast scenarios; inspection systems lack multi-module collaborative control mechanisms, exhibiting weak temporal analysis capabilities for continuous frame data, resulting in delayed abnormal reading warnings. Furthermore, data synchronization efficiency is low during multi-robot collaborative inspections, and the edge device model update mechanism is imperfect, making it difficult to adapt to dynamic changes in the industrial environment. Summary of the Invention

[0006] To address the key issues of low instrument detection accuracy, poor real-time performance, and weak adaptability in complex industrial environments, this application provides an instrument identification method for industrial robot inspection. This method integrates lightweight detection, adaptive preprocessing, multi-type instrument identification, and intelligent inspection control, effectively improving the reliability and efficiency of industrial inspection in complex environments.

[0007] To achieve the above objectives, this application employs the following technical solution:

[0008] This application discloses an instrument recognition method for industrial robot inspection. The instrument recognition method is implemented through an instrument recognition system, which includes a pose adjustment module, a lightweight detection module, an image dual enhancement processing module, and a dual-branch instrument reading recognition module. Specifically, the instrument recognition method includes the following steps:

[0009] Step 1: Acquire instrument images: The camera on the industrial inspection robot acquires instrument images in the industrial field. During the acquisition process, the camera's shooting position and angle are adjusted in real time according to the feedback from the pose adjustment module to obtain instrument images.

[0010] Step 2, Spatial Correction and Spectral Enhancement Processing: The instrument image acquired in Step 1 is input into the image dual enhancement processing module, and the pre-processed instrument image after preprocessing by the image dual enhancement processing module is output.

[0011] Step 3, Target Detection and Localization: The preprocessed instrument image obtained in Step 2 is input into the lightweight detection module. The lightweight detection module detects and localizes the preprocessed instrument image obtained in Step 2 and outputs the instrument target detection result.

[0012] Step 4: The dual-branch instrument reading recognition module realizes dual-branch reading recognition: Based on the instrument target detection results output in Step 3, the instrument type is determined. For pointer instruments, the pointer angle is calculated and converted into an instrument reading. For digital instruments, the CNN network performs character recognition to obtain the instrument reading.

[0013] Step 5, Result Output and Recording: Output and record the identified instrument readings. If any abnormality is detected during the output process, an early warning mechanism will be triggered. The instrument identification system will record the abnormality and send it to the manual monitoring terminal to notify relevant personnel for handling.

[0014] A further improvement in this application is that the pose adjustment module is used to control the pose of the inspection robot to ensure clear acquisition and accurate detection of instrument images. Specifically, step 1 involves adjusting the camera's position and angle in real time according to the pose adjustment module, including the following steps:

[0015] Step 1.1: Acquire instrument images using the camera of the industrial inspection robot, and obtain instrument position and attitude information using the YOLOv8 target detection model in the lightweight detection module;

[0016] Step 1.2: Based on the instrument position and attitude information obtained in Step 1.1, i.e., the position in the instrument feature point image, determine the relative pose between the camera and the instrument, i.e., the translation vector and rotation angle.

[0017] Step 1.3: Adjust the camera's shooting position and angle based on the relative pose obtained in Step 1.2, i.e., the translation vector and rotation angle, so that the instrument image is in the center of the camera's field of view and is captured clearly.

[0018] A further improvement of this application lies in the fact that the image dual enhancement processing module uses a perspective correction network and a spectral enhancement strategy to process the input instrument image, thereby achieving multi-dimensional optimization of the instrument image quality. The perspective correction network maps the tilted instrument image from a distorted viewpoint to a normal viewpoint. The spectral enhancement strategy uses the SSR algorithm to separate the illumination component and reflection component of the image, and the Adaptive Illumination Compensation (ALIC) algorithm to optimize the illumination component, reconstruct the instrument image, and achieve illumination equalization to obtain a preprocessed instrument image. Specifically, it includes the following steps:

[0019] Step 2.1: Use the U-Net structure as the perspective correction network to train the geometric transformation features of the instrument images acquired in Step 1, and generate the perspective transformation matrix. By mapping the tilted instrument image from the distorted viewpoint to the normal viewpoint, the restored instrument image is obtained. The transformation process is expressed as follows:

[0020]

[0021] in, The coordinates of the instrument image captured at an angle. For the repaired instrument image Coordinates, through perspective transformation matrix To achieve perspective distortion repair of instrument images, For transpose;

[0022] Step 2.2: Spectral Enhancement Strategy. First, the image repaired in Step 2.1 is separated using the SSR algorithm. Illumination and reflection components: using a Gaussian blur filter For the image repaired in step 2.1 Perform convolution operations to obtain the illumination components Capture global illumination trends and separate the intrinsic reflection components of objects. , is represented as:

[0023]

[0024]

[0025]

[0026] in, This is a scale parameter used to control the fuzzy scale, and it is adjusted according to specific circumstances. For convolution operations, and These are the horizontal and vertical coordinates of the instrument image, used for positioning on the image. is the base of the natural logarithm;

[0027] Step 2.3: Analyze the illumination components using the Adaptive Illumination Compensation (ALIC) algorithm. The local distribution of the image brightness is determined using the YCbCr color space: Overexposed areas are reduced by compression transform to decrease strong light. The dark areas are enhanced and transformed using the image dual enhancement processing module to generate optimized illumination components. Finally, the image is reconstructed using the following formula to achieve illumination balance:

[0028] ;

[0029] in, To reconstruct the image;

[0030] Step 2.4: Use the CLAHE algorithm to reconstruct the image obtained in Step 2.3. Divide into several sub-blocks and perform histogram equalization on each, in local areas Within, the reconstructed image obtained in step 2.3 pixels grayscale value The following transformations are performed to obtain the preprocessed instrument image:

[0031]

[0032] in, This is a contrast limiting parameter. For local areas The total number of pixels within, To exclude pixels Another pixel outside, For indicator functions, the indicator function is defined when the condition is true. Otherwise, the indicator function .

[0033] A further improvement of this application is that the lightweight detection module achieves instrument positioning based on the improved target detection model YOLOv8. The target detection model YOLOv8 includes a backbone network, a neck network, and a detection head, specifically:

[0034] The backbone network includes GSConv lightweight convolutional units, C2f_MSBlock modules, and SPPF modules;

[0035] The GSConv lightweight convolutional unit includes a standard convolutional layer and a depthwise separable convolutional layer. The standard convolutional layer extracts features from the input preprocessed instrument image and reduces the number of input channels from... Mapped to That is, half the number of output channels, where The number of input image channels, To output the number of image channels, the depthwise separable convolutional layer includes depthwise convolution and pointwise convolution, and uses a 5x5 convolutional kernel to extract image features of meters, pointers and scales, and outputs feature maps;

[0036] The C2f_MSBlock module includes several 1x1 convolutions and a convolution kernel size of [missing value]. The depth of the separable convolutional layer;

[0037] The neck network includes a Faster_CSPPC module and a Hybrid Local Channel Attention (MLCA) module. The Faster_CSPPC module includes convolutional layers, several CSPPC_Bottlenecks, and 1x1 convolutions. The CSPPC_Bottleneck consists of two partial convolutions. The feature map output by the GSConv lightweight convolutional unit after extracting image features of the instrument, pointer, and scale is used as the input feature map of the CSPPC_Bottleneck. After passing through several CSPPC_Bottlenecks, the information is re-aggregated through an inverse residual block composed of two 1x1 convolutions, and a feature vector is output. The Hybrid Local Channel Attention (MLCA) module includes local pooling, global pooling, a one-dimensional convolutional layer, and unpooling. The output feature vector of the CSPPC_Bottleneck is used as the input feature vector of the MLCA module. After local pooling, the branch: the side branch directly processes local features through global pooling and 1x1 convolution in sequence, and the main branch extracts global features through global pooling and unpooling. Finally, the results are added with the side branch results to fuse the feature information.

[0038] The detection head uses an adaptive anchor frame generation algorithm (AAG) to dynamically adjust the anchor frame parameters by analyzing the distribution of the instrument, pointer, and scale, and outputs the instrument target detection results.

[0039] The dual-branch instrument reading recognition module includes a pointer-type instrument recognition section and a digital instrument recognition section. Different methods are used for reading recognition of pointer-type instruments and digital instruments, respectively.

[0040] A further improvement in this application is that step 3 specifically includes the following steps:

[0041] Step 3.1: The feature map of the preprocessed instrument image input to a certain GSConv lightweight convolutional unit is first passed through a standard convolutional layer to generate an intermediate feature map. This intermediate feature map is then input into a depthwise separable convolutional layer. Finally, the results from the standard convolutional layer and the depthwise separable convolutional layer are concatenated along the channel dimension, and the channels are rearranged through a dimensionality transformation operation to output the feature map. , is represented as:

[0042]

[0043] in, For input features, , For the standard convolution process, For depthwise separable convolution processes, This is the channel rearrangement process;

[0044] Step 3.2: The C2f_MSBlock module processes the output feature map obtained from the GSConv lightweight convolutional unit. Divided into Each branch has a feature transformation performed within it using a 1x1 convolution, and then an application is made. The depthwise separable convolutional layer is followed by a 1x1 convolution, and finally, all branches are fused together by another 1x1 convolution to integrate the features of each branch, resulting in multiple sets of branch convolutional results. , is represented as:

[0045]

[0046] in, To invert the bottleneck block, after transformation using a 1×1 convolution, The channel dimension has been increased to Then input features Divided into Different groups ,in Finally, the resulting multi-branch convolution results are... The images of the instrument, pointer, and scale are fused and added together to obtain the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit.

[0047] Step 3.3: The output of the C2f_MSBlock module, namely the image feature maps of the instrument, pointer and scale, is processed by the backbone network and then input into the Faster_CSPPC module in the neck. Information is then aggregated in the channel dimension of the Faster_CSPPC module through two 1×1 pointwise convolutions. The first pointwise convolution is used to expand the number of channels, and the second is used to reduce the number of channels to form an inverse residual block structure. Finally, residual connections are performed to obtain the output feature vector of CSPPC_Bottleneck.

[0048] Step 3.4: The output feature vector of CSPPC_Bottleneck is used as the input feature vector of the Hybrid Local Channel Attention Module (MLCA). There will be two pooling steps to transform the input feature vector into... vectors ( The feature map is divided into blocks in width W and height H. Local features are first processed directly using Local Pooling (LAP) and 1x1 convolution. Building upon this initial stage, two branches transform the input feature vector into a one-dimensional vector, each containing global and local information respectively. After one-dimensional convolution, unpooling restores the original resolution of the two vectors, followed by information fusion to achieve mixed interest. This process enhances the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit—the image feature maps of the meter, pointer, and scale—and inputs them into the detection head to generate anchor boxes.

[0049] Step 3.5: The Adaptive Anchor Box Generation Algorithm (AAG) is based on the geometric prior obtained from the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit, i.e., the image feature maps of the meter, pointer, and scale. Initial anchor box sizes are generated through K-means++ clustering. The set of data is used to dynamically adjust the anchor frame size of the instrument, pointer, and scale using the following formula, and the instrument target detection results are output:

[0050]

[0051] in, The adjusted anchor frame dimensions. This is the initial anchor frame size. The preset reference scale threshold, The statistical scale is the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit, which is the input image, i.e., the image feature map of the meter, pointer, and scale. is the base of the natural logarithm.

[0052] A further improvement of this application lies in step 4, where the pointer-type instrument converts the pointer angle into an instrument reading, specifically including the following steps:

[0053] Step 4.1: Identify the pointer, lowest tick, and highest tick using the GSConv lightweight convolutional unit, and extract the center point of the pointer from the pointer region image. and end point The input is fed into an Angle Regression Network (ARN) to compute the pointer direction vector. Based on the scale distribution pattern, the pointer direction vector is... Mapped to angle values:

[0054]

[0055] Step 4.2: Calculate the minimum angle between the lowest and highest scale divisions. and maximum angle value Establish the correspondence between angle values ​​and readings.

[0056]

[0057] in, For the final reading, For the minimum range, For the maximum range;

[0058] Step 4.3: During the training of the Angle Regression Network (ARN), the labeled real pointer angle is used as the supervision signal, and the mean squared error (MSE) loss function is used for optimization.

[0059]

[0060] in, The number of training samples. For Angle Regression Network (ARN) to predict angles, This represents the actual pointer angle.

[0061] The beneficial effects of this application are:

[0062] This application, by deploying on industrial inspection robots and combining an improved lightweight target detection model with adaptive image optimization processing algorithms, overcomes the limitations of traditional geometric feature-based instrument readings, effectively improving the real-time performance and efficiency of instrument readings in industrial scenarios. Through integration with industrial inspection robots, this application provides more intelligent and accurate instrument readings and industrial inspections, offering strong support for information acquisition in complex industrial environments.

[0063] The lightweight detection model in this application significantly reduces computational complexity while maintaining detection accuracy by improving the feature fusion path and introducing a depthwise separable convolution mechanism. This enables the model to be efficiently deployed on edge devices such as industrial inspection robots, effectively solving the problem of insufficient real-time performance caused by excessive computation in traditional algorithms. The small target detection enhancement strategy enhances the ability to capture subtle features such as pointers and scales through a lightweight attention mechanism and geometric prior anchor box optimization, thereby improving the reliability of detection in complex backgrounds.

[0064] The image preprocessing stage of this application employs a dual approach of spatial correction and spectral enhancement. Spatial correction utilizes geometric transformation learning to repair perspective distortion, while spectral enhancement combines adaptive illumination compensation and contrast enhancement algorithms to dynamically address issues such as uneven lighting and strong light reflection in industrial settings. This provides high-quality image input for subsequent inspections, fundamentally improving image quality degradation caused by shooting angle and ambient lighting. The dual-branch instrument recognition module designs dedicated algorithms for the differentiated characteristics of pointer and digital instruments. Through the synergy of an angle regression network and a simplified convolutional neural network, it achieves high-precision reading analysis for different types of instruments, overcoming the limitations of a single network architecture in mixed instrument scenarios.

[0065] In terms of inspection and shooting control, the pose adjustment mechanism based on visual feedback can optimize the relative position of the camera and the instrument in real time to ensure clear image acquisition; the edge dynamic optimization mechanism enables the model to adaptively update parameters according to changes in the industrial site environment, continuously maintain the stability of detection performance, and comprehensively improve the automation level and environmental adaptability of the industrial inspection system. Attached Figure Description

[0066] Figure 1 This is a flowchart of this application.

[0067] Figure 2 This is a flowchart illustrating the lightweight testing module of this application.

[0068] Figure 3 This is a flowchart illustrating the pose adjustment module of this application.

[0069] Figure 4 This is a flowchart illustrating the image dual enhancement processing module of this application.

[0070] Figure 5 This is a flowchart illustrating the dual-branch instrument reading recognition module of this application.

[0071] Figure 6 This is a schematic diagram of the spatial correction in this application.

[0072] Figure 7 This is a schematic diagram comparing the spectral enhancement before and after the application.

[0073] Figure 8 This is the architecture diagram of the YOLOv8 object detection model in this application.

[0074] Figure 9 This is the architecture diagram of the C2f_MSBlock module in this application.

[0075] Figure 10 This is the architecture diagram of the Faster_CSPPC module in this application.

[0076] Figure 11 This is an architecture diagram of the hybrid local channel attention module of this application. Detailed Implementation

[0077] The embodiments of this application will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of this application, these practical details are not essential. In addition, for the sake of simplicity, some conventional structures and components will be shown in the drawings in a simple schematic manner.

[0078] Imagine a smart factory within a large petrochemical enterprise, where multiple industrial inspection robots perform real-time monitoring of instruments on equipment such as reactors, pipelines, and valves, following pre-set inspection routes. The factory environment includes high temperatures, dust, and complex lighting conditions. The instruments range from pointer-type pressure gauges to digital flow sensors. The system must meet the requirements for high-precision readings and real-time anomaly warnings.

[0079] This application discloses an instrument recognition method for industrial robot inspection. The instrument recognition method is implemented through an instrument recognition system, which includes a pose adjustment module, a lightweight detection module, an image dual enhancement processing module, and a dual-branch instrument reading recognition module.

[0080] like Figure 2 As shown, the lightweight detection module achieves instrument positioning based on the improved target detection model YOLOv8, such as... Figure 8 As shown, the target detection model YOLOv8 includes a backbone network, a neck network, and a detection head, specifically:

[0081] The backbone network includes GSConv lightweight convolutional units, a C2f_MSBlock module, and an SPPF module. The GSConv lightweight convolutional units include standard convolutional layers and depthwise separable convolutional layers. By combining standard convolution with depthwise separable convolution, the number of model parameters is reduced while maintaining strong feature extraction capabilities. The standard convolutional layers extract features from the preprocessed instrument image input, reducing the number of input channels from... Mapped to That is, half the number of output channels, where The number of input image channels, To output the number of image channels, the depthwise separable convolutional layer includes depthwise convolution and pointwise convolution, using a 5x5 convolutional kernel to extract image features of meters, pointers, and scales, thereby significantly reducing the computational load and outputting feature maps.

[0082] like Figure 9 As shown, the C2f_MSBlock module includes several 1x1 convolutions and a convolution kernel size of [missing value]. The depth of the separable convolutional layer.

[0083] like Figure 10 As shown, the neck network includes a Faster_CSPPC module and a Hybrid Local Channel Attention (MLCA) module. The Faster_CSPPC module includes convolutional layers, several CSPPC_Bottlenecks, and 1x1 convolutions. Each CSPPC_Bottleneck consists of two partial convolutions. The feature map output by the GSConv lightweight convolutional unit after extracting image features of the meter, pointer, and scale is used as the input feature map of the CSPPC_Bottleneck. After passing through several CSPPC_Bottlenecks, the information is re-aggregated via an inverse residual block composed of two 1x1 convolutions, and a feature vector is output. Figure 11 As shown, the Hybrid Local Channel Attention Module (MLCA) includes local pooling, global pooling, a one-dimensional convolutional layer, and unpooling. The output feature vector of CSPPC_Bottleneck is used as the input feature vector of the Hybrid Local Channel Attention Module. After local pooling, the following branches are formed: the side branches directly process local features through global pooling and 1x1 convolution, while the main branch extracts global features through global pooling and unpooling. Finally, the results of the side branches are added together to fuse the feature information.

[0084] The detection head uses an adaptive anchor frame generation algorithm (AAG) to dynamically adjust the anchor frame parameters by analyzing the distribution of the instrument, pointer, and scale, and outputs the instrument target detection results.

[0085] The dual-branch instrument reading recognition module includes a pointer instrument recognition section and a digital instrument recognition section, and different methods are used to recognize the readings of pointer instruments and digital instruments respectively.

[0086] like Figure 4 As shown, the image dual enhancement processing module uses a perspective correction network and a spectral enhancement strategy to process the input instrument image, achieving multi-dimensional optimization of the instrument image quality. The perspective correction network maps the tilted instrument image from a distorted viewpoint to a normal viewpoint. The spectral enhancement strategy uses the SSR algorithm to separate the illumination and reflection components of the image and the Adaptive Illumination Compensation (ALIC) algorithm to optimize the illumination component, reconstructing the instrument image and achieving illumination equalization, thus obtaining the preprocessed instrument image.

[0087] like Figure 1 As shown, this application also provides an instrument identification method for industrial robot inspection, which includes the following steps:

[0088] Step 1: Acquire Instrument Images: The camera on the industrial inspection robot acquires instrument images in the industrial field. During the acquisition process, the camera's shooting position and angle are adjusted in real time based on feedback from the pose adjustment module to obtain clear and complete instrument images.

[0089] like Figure 3 As shown, the pose adjustment module is used to control the pose of the inspection robot to ensure clear acquisition and accurate detection of instrument images. Step 1 involves adjusting the camera's position and angle in real time according to the pose adjustment module, specifically including the following steps:

[0090] Step 1.1: Acquire instrument images using the camera of the industrial inspection robot, and obtain instrument position and attitude information using the YOLOv8 target detection model in the lightweight detection module;

[0091] Step 1.2: Based on the instrument position and attitude information obtained in Step 1.1, i.e., the position in the instrument feature point image, determine the relative pose between the camera and the instrument, i.e., the translation vector and rotation angle.

[0092] Step 1.3: Based on the relative pose obtained in Step 1.2 (i.e., translation vector and rotation angle), adjust the camera's shooting position and angle. For example, if the instrument display is incomplete, translate the lens; if the instrument and lens plane are not parallel, rotate the lens to ensure the instrument image is centered within the camera's field of view and clearly captured. Maintain a suitable shooting angle and distance. In practice, a feature point matching-based method can be used to calculate the pose, such as ORB feature matching combined with the PnP (Perspective-n-Point) algorithm. This method uses the known positions of instrument feature points in the image to solve for the camera's pose parameters.

[0093] Step 2, Spatial Correction and Spectral Enhancement Processing: The instrument image acquired in Step 1 is input into the image dual enhancement processing module, which outputs a high-quality pre-processed instrument image after preprocessing by the image dual enhancement processing module, such as... Figure 6 As shown. Specifically, it includes the following steps:

[0094] Step 2.1: Use the U-Net structure as the perspective correction network to train the geometric transformation features of the instrument images acquired in Step 1, and generate the perspective transformation matrix. By mapping the tilted instrument image from the distorted viewpoint to the normal viewpoint, the restored instrument image is obtained. The transformation process is expressed as follows:

[0095]

[0096] in, The coordinates of the instrument image captured at an angle. For the repaired instrument image Coordinates, through perspective transformation matrix To achieve perspective distortion repair of instrument images, For transpose;

[0097] Step 2.2: Spectral Enhancement Strategy. First, the image repaired in Step 2.1 is separated using the SSR algorithm. Illumination and reflection components: using a Gaussian blur filter For the image repaired in step 2.1 Perform convolution operations to obtain the illumination components Capture global illumination trends and separate the intrinsic reflection components of objects. , is represented as:

[0098]

[0099]

[0100]

[0101] in, This is a scale parameter used to control the fuzzy scale, and it is adjusted according to specific circumstances. For convolution operations, and These are the horizontal and vertical coordinates of the instrument image, used for positioning on the image. is the base of the natural logarithm;

[0102] Step 2.3: Analyze the illumination components using the Adaptive Illumination Compensation (ALIC) algorithm. The local distribution of the image brightness is determined using the YCbCr color space: Overexposed areas are almost pure white, resulting in loss of detail. Compression transformation is used to reduce strong light. The dark areas are almost pure black and dominated by noise. The image dual enhancement processing module is used to enhance the transformation and generate optimized illumination components. Finally, the image is reconstructed using the following formula to achieve illumination balance:

[0103] .

[0104] in, To reconstruct the image;

[0105] Step 2.4: Use the CLAHE algorithm to reconstruct the image obtained in Step 2.3. The image is divided into several sub-blocks, and histogram equalization is performed on each block to enhance image detail and contrast, and improve visibility in low-light areas. Specifically, this is done in local areas. Within, the reconstructed image obtained in step 2.3 pixels grayscale value The following transformations are performed to obtain the preprocessed instrument image:

[0106]

[0107] in, This is a contrast limiting parameter. For local areas The total number of pixels within, To exclude pixels Another pixel outside, For indicator functions, the indicator function is defined when the condition is true. Otherwise, the indicator function .like Figure 7 As shown, spectral enhancement methods can effectively optimize images with low light levels and blurred details, facilitating subsequent target detection.

[0108] Step 3, Target Detection and Localization: The preprocessed instrument image obtained in Step 2 is input into the lightweight detection module. The lightweight detection module detects and locates the preprocessed instrument image obtained in Step 2 and outputs the instrument target detection result.

[0109] Specifically, the steps include the following:

[0110] Step 3.1: The feature map of the preprocessed instrument image input to a certain GSConv lightweight convolutional unit is first passed through a standard convolutional layer to generate an intermediate feature map. This intermediate feature map is then input into a depthwise separable convolutional layer. Finally, the results from the standard convolutional layer and the depthwise separable convolutional layer are concatenated along the channel dimension, and the channels are rearranged through a dimensionality transformation operation to output the feature map. , is represented as:

[0111]

[0112] in, For input features, , For the standard convolution process, For depthwise separable convolution processes, This is the channel rearrangement process;

[0113] The size of the input feature map is Output feature map The size is For standard convolution, the computational cost of the YOLOv8 object detection model is:

[0114]

[0115] in, The kernel size is [value], and in the GSConv lightweight convolutional unit, it is first [value]... The group undergoes depthwise convolution, followed by pointwise convolution with 1×1 convolutions. The total computational cost of the YOLOv8 object detection model is:

[0116]

[0117] Compared to standard convolution, it significantly reduces computational cost, enabling the model to run efficiently on embedded devices and meet the real-time detection needs of industrial sites.

[0118] Step 3.2: The C2f_MSBlock module processes the output feature map obtained from the GSConv lightweight convolutional unit. Divided into There are several branches, each processing a different subset of features. Within each branch, feature transformation is performed using a 1x1 convolution, and then a... The depthwise separable convolutional layer is followed by a 1x1 convolution to enhance features and reduce the number of parameters. Finally, all branches are fused together by a 1x1 convolution to integrate the features of each branch, resulting in multiple sets of branch convolution results. , is represented as:

[0119]

[0120] in, To invert the bottleneck block, after transformation using a 1×1 convolution, The channel dimension has been increased to Then input features Divided into Different groups ,in To reduce computational costs and adapt to the actual industrial instrument testing needs, the embodiments of this application select... The result is 3, and finally the multiple branches of the convolution result are obtained. The images of the instrument, pointer, and scale are fused and added together to obtain the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit.

[0121] Step 3.3: The output of the C2f_MSBlock module, namely the image feature maps of the instrument, pointer, and scale, is processed by the backbone network and then input into the Faster_CSPPC module in the neck. Partial convolution (PConv) is introduced in the Bottleneck of this module to reduce channel sparsity. Then, information is aggregated through two 1×1 pointwise convolutions in the channel dimension of the Faster_CSPPC module. The first pointwise convolution is used to expand the number of channels, and the second is used to reduce the number of channels, forming an inverse residual block structure. Finally, residual connections are performed to obtain the output feature vector of CSPPC_Bottleneck.

[0122] Step 3.4: The output feature vector of CSPPC_Bottleneck is used as the input feature vector of the Hybrid Local Channel Attention Module (MLCA). There will be two pooling steps to transform the input feature vector into... vectors ( The feature map is divided into blocks in width W and height H. Local features are first processed directly using Local Pooling (LAP) and 1x1 convolution. Building upon this initial stage, two branches transform the input feature vector into a one-dimensional vector, each containing global and local information respectively. After one-dimensional convolution, unpooling restores the original resolution of the two vectors, followed by information fusion to achieve mixed interest. This process enhances the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit—the image feature maps of the meter, pointer, and scale—and inputs them into the detection head to generate anchor boxes.

[0123] Step 3.5: The Adaptive Anchor Box Generation Algorithm (AAG) is based on the geometric prior obtained from the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit, i.e., the image feature maps of the meter, pointer, and scale. Initial anchor box sizes are generated through K-means++ clustering. The set of data is used to dynamically adjust the anchor frame size of the instrument, pointer, and scale using the following formula, and the instrument target detection results are output:

[0124]

[0125] in, The adjusted anchor frame dimensions. This is the initial anchor frame size. The preset reference scale threshold, The statistical scale is the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit, which is the input image, i.e., the image feature map of the meter, pointer, and scale. is the base of the natural logarithm.

[0126] Step 4: The dual-branch instrument reading recognition module realizes dual-branch reading recognition: Based on the instrument target detection results output in Step 3, the instrument type is determined. For pointer instruments, the pointer angle is calculated to convert the instrument reading into an instrument reading. For digital instruments, the CNN network performs character recognition to obtain the instrument reading.

[0127] In this step, the pointer-type instrument converts the pointer angle into an instrument reading, specifically including the following steps:

[0128] Step 4.1: Identify the pointer, lowest tick, and highest tick using the GSConv lightweight convolutional unit, and extract the center point of the pointer from the pointer region image. and end point The input is fed into an Angle Regression Network (ARN) to compute the pointer direction vector. Based on the scale distribution pattern, the pointer direction vector is... Mapped to angle values:

[0129]

[0130] Step 4.2: Calculate the minimum angle between the lowest and highest scale divisions. and maximum angle value Establish the correspondence between angle values ​​and readings.

[0131]

[0132] in, For the final reading, For the minimum range, For the maximum range;

[0133] Step 4.3: During the training of the Angle Regression Network (ARN), the labeled real pointer angle is used as the supervision signal, and the mean squared error (MSE) loss function is used for optimization.

[0134]

[0135] in, The number of training samples. For Angle Regression Network (ARN) to predict angles, This represents the actual pointer angle.

[0136] Step 5, Result Output and Recording: Output and record the identified instrument readings for subsequent analysis and decision-making. At the same time, if any abnormalities are found during the output process, such as abnormal data changes or instrument damage, an early warning mechanism will be triggered. The instrument identification system will record the abnormality and send it to the manual monitoring terminal to notify relevant personnel for handling.

[0137] like Figure 5As shown, the dual-branch instrument reading recognition module employs a simple convolutional neural network (CNN) architecture to recognize characters on the display area of ​​digital instruments. First, the digital display area is located using a target detection model, and then the image of this area is input into the CNN network. The network is processed through convolutional layers, pooling layers, and fully connected layers to output the recognition result. During training, a large number of sample images of digital instruments are used for annotation and training, enabling the network to learn the feature patterns of digital characters, thereby accurately recognizing the readings.

[0138] Simulation Experiment

[0139] Table 1 shows a comparison of the feature extraction capabilities and time complexity of common methods with those of this application, based on the PASCAL VOC 2007+12 dataset.

[0140] Table 1

[0141]

[0142] As can be seen from Table 1, the lightweight improvement method adopted in this application performs excellently at a more stringent detection accuracy of mAP@0.95, reflecting strong feature extraction capabilities. At the same time, the computational complexity is lower than that of methods with the same accuracy level, proving that it has achieved a better balance between "high-precision detection" and "computational efficiency".

[0143] Table 2

[0144]

[0145] As shown in Table 2, the Faster_CSPPC module used in this application significantly reduces computational latency while ensuring the accuracy of small target detection and maintaining sufficient computational speed.

[0146] This application provides an integrated solution for automated testing, real-time reading, and intelligent inspection of industrial field instruments through the deep integration of multiple technology modules, effectively improving the reliability and efficiency of industrial inspection in complex environments.

[0147] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. An instrument identification method for industrial robot inspection, characterized in that: The instrument recognition method is implemented through an instrument recognition system, which includes a pose adjustment module, a lightweight detection module, an image dual enhancement processing module, and a dual-branch instrument reading recognition module. Specifically, the instrument recognition method includes the following steps: Step 1: Acquire instrument images: The camera on the industrial inspection robot acquires instrument images in the industrial field. During the acquisition process, the camera's shooting position and angle are adjusted in real time according to the feedback from the pose adjustment module to obtain instrument images. Step 2, Spatial Correction and Spectral Enhancement Processing: The instrument image acquired in Step 1 is input into the image dual enhancement processing module, and the pre-processed instrument image after preprocessing by the image dual enhancement processing module is output. Step 3, Target Detection and Localization: The preprocessed instrument image obtained in Step 2 is input into the lightweight detection module. The lightweight detection module detects and localizes the preprocessed instrument image obtained in Step 2 and outputs the instrument target detection result. Step 4: The dual-branch instrument reading recognition module realizes dual-branch reading recognition: Based on the instrument target detection results output in Step 3, the instrument type is determined. For pointer instruments, the pointer angle is calculated and converted into an instrument reading. For digital instruments, the CNN network performs character recognition to obtain the instrument reading. Step 5, Result Output and Recording: Output and record the identified instrument readings. If an abnormality is detected during the output process, an early warning mechanism will be triggered. The instrument identification system will record the abnormality and send it to the manual monitoring terminal.

2. The instrument identification method for industrial robot inspection according to claim 1, characterized in that: The lightweight detection module achieves instrument positioning based on the improved target detection model YOLOv8. The improved target detection model YOLOv8 includes a backbone network, a neck network, and a detection head, specifically: The backbone network includes GSConv lightweight convolutional units, C2f_MSBlock modules, and SPPF modules; The GSConv lightweight convolutional unit includes a standard convolutional layer and a depthwise separable convolutional layer. The standard convolutional layer extracts features from the input preprocessed instrument image and reduces the number of input channels from... Mapped to That is, half the number of output channels, where The number of input image channels, To output the number of image channels, the depthwise separable convolutional layer includes depthwise convolution and pointwise convolution, and uses a 5x5 convolutional kernel to extract image features of meters, pointers and scales, and outputs feature maps; The C2f_MSBlock module includes several 1x1 convolutions and a convolution kernel size of [missing value]. The depth of the separable convolutional layer; The neck network includes a Faster_CSPPC module and a hybrid local channel attention module. The Faster_CSPPC module includes convolutional layers, several CSPPC_Bottlenecks, and 1x1 convolutions. Each CSPPC_Bottleneck consists of two partial convolutions. The output feature map of the GSConv lightweight convolutional unit serves as the input feature map of the CSPPC_Bottleneck. After passing through several CSPPC_Bottlenecks, the information is re-aggregated and output as a feature vector via an inverted residual block composed of two 1x1 convolutions. The hybrid local channel attention module includes local pooling, global pooling, a one-dimensional convolutional layer, and unpooling. The output feature vector of the CSPPC_Bottleneck serves as the input feature vector of the hybrid local channel attention module. After local pooling, the branch: the side branch directly processes local features through global pooling and 1x1 convolutions in sequence, while the main branch extracts global features through global pooling and unpooling. Finally, the results are added to the side branch results to fuse the feature information. The detection head uses an adaptive anchor frame generation algorithm to dynamically adjust the anchor frame parameters by analyzing the distribution of the instrument, pointer, and scale, and outputs the instrument target detection results. The dual-branch instrument reading recognition module includes a pointer-type instrument recognition section and a digital instrument recognition section.

3. The instrument identification method for industrial robot inspection according to claim 2, characterized in that: Step 3 specifically includes the following steps: Step 3.1: The feature map of the preprocessed instrument image input to a certain GSConv lightweight convolutional unit is first passed through a standard convolutional layer to generate an intermediate feature map. This intermediate feature map is then input into a depthwise separable convolutional layer. Finally, the results from the standard convolutional layer and the depthwise separable convolutional layer are concatenated along the channel dimension, and the channels are rearranged through a dimensionality transformation operation to output the feature map. , is represented as: ， in, The input features are the features of the preprocessed instrument image. , For the standard convolution process, For depthwise separable convolution processes, This is the channel rearrangement process; Step 3.2: The C2f_MSBlock module outputs the feature map obtained in step 3.

1. Divided into There are several branches. Within each branch, feature transformation is performed using 1x1 convolutions, and then a convolution kernel of size is applied. The depthwise separable convolutional layer is followed by a 1x1 convolution, and finally, all branches are fused together by another 1x1 convolution to integrate the features of each branch, resulting in multiple sets of branch convolution results. , is represented as: ， in, To invert the bottleneck block, the input features are transformed through a 1×1 convolution. The channel dimension has been increased to Then input features Divided into Different groups ,in Finally, the resulting multi-branch convolution results The images of the instrument, pointer, and scale are fused and added together to obtain the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit. Step 3.3: The output of the C2f_MSBlock module, namely the image feature map of the instrument, pointer and scale, is input into the Faster_CSPPC module of the neck. Then, information is aggregated through two 1×1 pointwise convolutions in the channel dimension of the Faster_CSPPC module to form an inverted residual block structure. Finally, residual connections are performed to obtain the output feature vector of CSPPC_Bottleneck. Step 3.4: The output feature vector of CSPPC_Bottleneck is used as the input feature vector of the hybrid local channel attention module. The input feature vector is transformed into a one-dimensional vector using two branches, and local and global features are extracted respectively. Then, feature fusion is performed to obtain multiple sets of branch convolution results. Finally, the multiple sets of branch convolution results are added to enhance the channel information extraction capability. The output results of the C2f_MSBlock module in the GSConv lightweight convolution unit, namely the image feature maps of the instrument, pointer and scale, are enhanced with information and input into the detection head to generate anchor boxes. Step 3.5: The adaptive anchor box generation algorithm is based on the output of the C2f_MSBlock module in the GSConv lightweight convolutional unit, i.e., the image feature maps of the meter, pointer, and scale. Initial anchor box sizes are generated through K-means++ clustering. The set of data is used to dynamically adjust the anchor frame size of the instrument, pointer, and scale using the following formula, and the instrument target detection results are output: ， in, The adjusted anchor frame dimensions. This is the initial anchor frame size. The preset reference scale threshold, The statistical scale of the image feature maps of the meters, pointers, and scales is the input image, i.e., the output of the C2f_MSBlock module. is the base of the natural logarithm.

4. The instrument identification method for industrial robot inspection according to claim 3, characterized in that: In step 4, the pointer-type instrument converts the pointer angle into an instrument reading, specifically including the following steps: Step 4.1: Identify the pointer, lowest tick, and highest tick using the GSConv lightweight convolutional unit, and extract the center point of the pointer from the pointer region image. and end point The input is fed into an angle regression network to calculate the pointer direction vector. , pointer direction vector Mapped to angle values: ； Step 4.2: Calculate the minimum angle between the lowest and highest scale divisions. and maximum angle value Establish the correspondence between angle values ​​and readings: ， in, For the final reading, For the minimum range, For the maximum range; Step 4.3: During the training of the angle regression network, the labeled real pointer angle is used as the supervision signal, and the mean squared error loss function is used for optimization. ， in, The number of training samples. For angle regression networks to predict angles, This represents the actual pointer angle.

5. A method for instrument identification in industrial robot inspection according to any one of claims 1-4, characterized in that: The pose adjustment module is used to control the pose of the inspection robot to ensure the acquisition and detection of instrument images. Step 1 involves adjusting the camera's position and angle in real time according to the pose adjustment module, specifically including the following steps: Step 1.1: Collect instrument images using the camera of the industrial inspection robot, and obtain the instrument's position and attitude information using the lightweight detection module; Step 1.2: Based on the instrument position and attitude information obtained in Step 1.1, determine the relative pose between the camera and the instrument, i.e., the translation vector and rotation angle. Step 1.3: Adjust the camera's shooting position and angle based on the relative pose obtained in Step 1.2, i.e., the translation vector and rotation angle, so that the instrument image is in the center of the camera's field of view and is captured clearly.

6. A method for instrument identification in industrial robot inspection according to any one of claims 1-4, characterized in that: The image dual enhancement processing module uses a perspective correction network and a spectral enhancement strategy to process the input instrument image. The perspective correction network maps the tilted instrument image from a distorted viewpoint to a normal viewpoint. The spectral enhancement strategy uses the SSR algorithm to separate the illumination and reflection components of the instrument image, and an adaptive illumination compensation algorithm to optimize the illumination component and reconstruct the instrument image to obtain a preprocessed instrument image. The specific steps include: Step 2.1: Use the U-Net structure as the perspective correction network to train the geometric transformation features of the instrument images acquired in Step 1, and generate the perspective transformation matrix. The instrument images captured at an angle are mapped from the distorted viewpoint to the normal viewpoint to obtain the restored instrument images. The transformation process is expressed as follows: ， in, The coordinates of the instrument image captured at an angle. For the repaired instrument image Coordinates, through perspective transformation matrix To achieve perspective distortion repair of instrument images, For transpose; Step 2.2, Spectral Enhancement Strategy: First, the instrument image repaired in Step 2.1 is separated using the SSR algorithm. Illumination and reflection components: using a Gaussian blur filter For the instrument image repaired in step 2.1 Perform convolution operations to obtain the illumination components Separate the reflection component , is represented as: ， ， ， in, For scale parameters, For convolution operations, and The horizontal and vertical axes of the instrument image, is the base of the natural logarithm; Step 2.3: Analyze the illumination components using an adaptive illumination compensation algorithm. The local distribution of the image brightness is determined using the YCbCr color space: Overexposed areas are reduced by compression transform to decrease strong light. The dark areas are enhanced and transformed using the image dual enhancement processing module to generate optimized illumination components. The image is finally reconstructed using the following formula: ， in, To reconstruct the image; Step 2.4: Use the CLAHE algorithm to reconstruct the image obtained in Step 2.

3. Divide into several sub-blocks and perform histogram equalization on each, in local areas Within, the reconstructed image obtained in step 2.3 pixels grayscale value The following transformations are performed to obtain the preprocessed instrument image: ， in, This is a contrast limiting parameter. For local areas The total number of pixels within, To exclude pixels Another pixel outside, For indicator functions, the indicator function is defined when the condition is true. Otherwise, the indicator function .

Images