Insulator RTV spraying robot spraying effect online evaluation method and system

Abstract

The application belongs to the technical field of power equipment operation and maintenance and deep learning, and provides an insulator RTV spraying robot spraying effect online evaluation method and system. The scheme is to perform standardized enhancement processing on multi-view images, obtain a spraying state probability matrix and an uncertainty region probability matrix based on the enhanced four-channel images and a trained lightweight spraying state segmentation network, establish an insulator curved surface development coordinate system, respectively map the spraying state probability matrix and the uncertainty region probability matrix under multiple views to a unified development space, perform confidence weighted fusion on the multi-view mapping results according to a view angle weight, an image quality weight and a model uncertainty weight, calculate an overall sprayed coverage rate, an overall missed spraying rate, an abnormal spraying proportion and an uncertainty region proportion based on the fused spraying state probability graph and the fused uncertainty graph, and output a spraying effect evaluation result and a spraying grade. The accuracy of insulator RTV spraying effect evaluation is significantly improved.

Description

Technical Field

[0001] This invention belongs to the fields of power equipment operation and maintenance and deep learning technology, and in particular relates to an online evaluation method and system for the spraying effect of an RTV spraying robot for insulators. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The quality of RTV coating on insulators directly affects the anti-flashover performance of transmission line insulators. Traditional evaluation of coating effectiveness mainly relies on manual visual inspection or post-coating sampling verification, which suffers from low efficiency, strong subjectivity, and difficulty in providing real-time feedback. With the development of intelligent operation and maintenance of transmission lines and coating robot technology, using robots equipped with vision devices for online evaluation of RTV coating effectiveness has become an important direction for improving the quality and automation level of coating operations.

[0004] Traditional visual evaluation methods for RTV coating quality of insulators mostly remain at the stage of "image detection and offline scoring," which has many problems: they rely on single-angle images or fixed shooting results, lacking the ability to develop and model the curved surface structure of insulators and fuse multiple perspectives; the judgment of coating quality mainly depends on defect category identification and area ratio statistics, making it difficult to distinguish the risk difference between scattered small-area defects and continuous large-area missed spraying; the evaluation results usually only output scores or grades, lacking a unified expression of the spatial distribution of coating coverage state, making it difficult to meet the requirements of real-time performance, stability, and interpretability in the online operation of coating robots.

[0005] To address the aforementioned issues, existing technologies can achieve automatic identification and quantitative evaluation of various types of spraying defects. However, their essence remains grid classification evaluation based on a single spraying area image. They lack a fusion evaluation mechanism for real-time transmission of multi-angle images from the spraying robot, and fail to adequately solve the problems of multi-angle occlusion, curved surface coverage fusion, and missing spraying location of key parts during robot on-site operations. Furthermore, they do not further convert the detection results into robot-executable respraying areas, respraying paths, and re-inspection strategies, making it difficult to meet the online evaluation and stability rating requirements during robot RTV spraying operations. Summary of the Invention

[0006] To address at least one of the technical problems mentioned above, this invention provides a method and system for online evaluation of the spraying effect of an insulator RTV spraying robot.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a method for online evaluation of the spraying effect of an RTV spraying robot for insulators, comprising the following steps: The obtained multi-view images are subjected to standardization enhancement processing to obtain four-channel images after enhancement; Based on the four-channel images after enhancement and the trained lightweight spraying state segmentation network, a spraying state probability matrix and an uncertainty region probability matrix are obtained; wherein the construction process of the lightweight spraying state segmentation network comprises: Based on the four-channel images after enhancement, a detail feature matrix and a semantic context feature matrix are extracted, and a boundary prior matrix is generated; the detail feature matrix, the semantic context feature matrix and the boundary prior matrix are adaptively fused to obtain a fused feature matrix; the fused feature matrix, the boundary prior matrix and an output head are combined to obtain a spraying state probability matrix, a boundary probability matrix and an uncertainty region probability matrix; A curved surface development coordinate system of an insulator is established, and the spraying state probability matrices and the uncertainty region probability matrices under multiple views are respectively mapped to a unified development space; according to view angle weights, image quality weights and model uncertainty weights, the multi-view mapping results are subjected to confidence weighted fusion to obtain a fused spraying state probability graph and a fused uncertainty graph in the unified development space; Based on the fused spraying state probability graph and the fused uncertainty graph, an overall sprayed coverage rate, an overall missed spraying rate, an abnormal spraying proportion and an uncertainty region proportion are calculated, and a spraying effect evaluation result and a spraying grade are output.

[0008] Further, the standardization enhancement processing of the obtained multi-view images comprises: The local mean filter is used to estimate the local illumination component of each single-view RGB image ; The original image is subjected to illumination normalization in combination with the local illumination component to obtain a normalized image ; The normalized image is subjected to per-channel color correction to output a color corrected image ; The color corrected image is subjected to local contrast enhancement to obtain a local enhanced image ; Based on the color corrected image , a highlight region prior estimation is performed to obtain a highlight region prior graph ; The color corrected image , the local enhanced image and the highlight prior graph are fused to obtain a standardization enhanced image ; The highlight prior graph is fused with the standardization enhanced image The images are stitched together to obtain an enhanced four-channel image. The first three channels are the standardized enhanced RGB images, and the fourth channel is the prior image of the reflective area.

[0009] Furthermore, the extraction of the detail feature matrix based on the enhanced four-channel image includes: A shallow convolution mapping is performed on the four-channel input, and the image resolution is downsampled to half of the original image to obtain the initial detail feature matrix. ; Based on the initial detail feature matrix, multi-scale detail features of small missed spray edges and spray texture variations are extracted simultaneously. The extracted multi-scale detail features are concatenated through channels to obtain a multi-scale fused detail feature matrix. ; The prior matrix of the reflective region is downsampled to the scale of detailed features to calculate the reliability matrix. ; Multi-scale fusion of detail feature matrices With reliability matrix Concatenate the data to generate a detail attention matrix. ; Combining detail attention matrix For the initial detail feature matrix Multi-scale fusion detail feature matrix and reliability matrix Fusion into the final detail feature matrix .

[0010] Furthermore, a semantic context feature matrix is ​​extracted based on the enhanced four-channel image, including: A shallow convolution mapping is performed on the four-channel input, and the image resolution is downsampled to half of the original image to obtain the initial semantic feature matrix. ; Based on the inverse residual module, the spatial resolution is gradually reduced to obtain the downsampled semantic feature matrix. ; Multi-scale context features are obtained by extracting context features from the downsampled semantic feature matrix using depthwise convolutions with different dilation rates. Contextual features at different scales are concatenated, and channel compression and fusion are performed using 1×1 convolution to obtain a multi-scale up-down fused feature matrix. ; Global average pooling is performed on the multi-scale context feature matrix, and semantic attention vectors are generated through lightweight channel mapping. ; Based on semantic attention vectors Multi-scale up-down fusion feature matrix and the downsampled semantic feature matrix The final semantic context feature matrix is ​​obtained. .

[0011] Furthermore, the adaptive fusion of the detail feature matrix, semantic context feature matrix, and reflective region prior matrix includes: semantic context feature matrix Upsampling is performed, and the upsampled semantic features are channel-compressed using 1×1 convolutions to obtain a semantic feature matrix whose scale and number of channels are consistent with the detailed features. ; Depth convolutions with different dilation rates were used to extract semantic feature matrices. Multi-scale semantic features are extracted and then concatenated and fused to obtain a multi-scale semantic enhancement feature matrix. Multi-scale semantic enhancement feature matrix Perform global average pooling to obtain the channel description vector. Generate channel attention vectors based on channel description vectors ; right Average pooling and max pooling are performed along the channel direction to obtain the average response matrix. and maximum response matrix ; Average response matrix Maximum response matrix Concatenate the boundary prior matrix with the boundary prior matrix to generate a boundary-guided spatial attention matrix. ; Based on channel attention vector Spatial attention matrix Multi-scale semantic enhancement feature matrix Relabeling is performed to obtain effective semantic features. ; Based on the adaptive selection of detailed features or semantic features in different regions, a gated fusion matrix is ​​constructed. ; Combined with gated fusion matrix Effective semantic features and detailed feature matrix Obtain the fusion feature matrix .

[0012] Furthermore, during the training of the lightweight spraying state segmentation network, its loss function... for: , , , , , in, Loss due to segmentation during spraying process. For category weights, For the c-th type of real label, This represents the probability that pixel position (x, y) belongs to the c-th spraying state. Indicates the number of spraying status categories. For boundary loss, Represents the boundary probability matrix. Indicates the true label of the boundary. Losses in uncertain areas, Represents the probability matrix of the region of uncertainty. Indicates the true label of the uncertain area For Dice's loss, This indicates the predicted probability of unpainted areas. This indicates the actual label in the unpainted area. To prevent division by zero of extremely small constants, , , The corresponding loss weights.

[0013] Furthermore, the fusion spraying state probability diagram and fusion uncertainty diagram are represented as follows: , , in, Indicates the unfolding point after the three perspectives are merged. The probability of being in the c-th type of spraying state. Indicates the first i Each perspective at the unfolding point The fusion weight at the point, z Indicates the axial position of the insulator. Represents the circumferential angle. Indicates the first i The probability of the c-th type of spraying state after surface unfolding and mapping from a given perspective. To prevent division by zero of extremely small constants, This represents the probability of uncertainty after fusion. Indicates the first i The probability of the uncertain region after mapping from each perspective.

[0014] A second aspect of the present invention provides an online evaluation system for the spraying effect of an insulator RTV spraying robot, comprising: The multi-view image enhancement module is used to perform normalization enhancement processing on the acquired multi-view images to obtain enhanced four-channel images. A lightweight image segmentation module is used to obtain the spraying state probability matrix and the uncertainty region probability matrix based on the enhanced four-channel image and the trained lightweight spraying state segmentation network. The construction process of the lightweight spraying state segmentation network includes: Based on the enhanced four-channel image, detail feature matrix and semantic context feature matrix are extracted, and boundary prior matrix is ​​generated. The detail feature matrix, semantic context feature matrix and boundary prior matrix are adaptively fused to obtain fused feature matrix. The fused feature matrix, boundary prior matrix and output head are combined to obtain spraying state probability matrix, boundary probability matrix and uncertainty region probability matrix. The multi-view mapping weighting module is used to establish the insulator surface unfolding coordinate system and map the spraying state probability matrix and the uncertainty region probability matrix under multiple views to a unified unfolding space. Based on the view angle weight, image quality weight and model uncertainty weight, the multi-view mapping results are fused with confidence weight to obtain the fused spraying state probability map and fused uncertainty map under the unified unfolding space. The spraying effect evaluation module is used to calculate the overall sprayed coverage, overall missed spraying rate, abnormal spraying ratio, and uncertain area ratio based on the fused spraying state probability map and fused uncertainty map, and output the spraying effect evaluation results and spraying grade.

[0015] A third aspect of the present invention provides a computer-readable storage medium.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the online evaluation method for the spraying effect of an insulator RTV spraying robot as described above.

[0017] A fourth aspect of the present invention provides a computer device.

[0018] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it performs the steps in the online evaluation method for the coating effect of an insulator RTV coating robot as described above.

[0019] Compared with the prior art, the beneficial effects of the present invention are: This invention creatively proposes an online evaluation method for the spraying effect of an RTV spraying robot for insulators and develops an online evaluation system for the spraying effect of an RTV spraying robot for insulators. By integrating multi-view image acquisition, image standardization enhancement, lightweight spraying state segmentation, insulator surface unfolding mapping, multi-view confidence fusion, and quantitative evaluation of spraying effect, the online evaluation method realizes real-time identification and fusion evaluation of sprayed areas, unsprayed areas, abnormal sprayed areas, and uncertain areas of insulators. This significantly improves the accuracy, stability, and real-time performance of RTV spraying effect evaluation and reduces the subjective error and verification cost of manual visual inspection.

[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of the online evaluation method for the spraying effect of the RTV spraying robot for insulators provided in this embodiment of the invention; Figure 2 This is a schematic diagram of three-view acquisition provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the unfolded surface of an insulator provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the fusion result provided in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] Example 1 likeFigure 1 As shown, this embodiment provides an online evaluation method for the spraying effect of an insulator RTV spraying robot, including the following steps: S1. Acquire multi-angle image data covering the insulator from different orientations, and as follows: Figure 2 As shown in this embodiment, insulator coating images are acquired from three different perspectives. Each perspective is rotated around the insulator's central axis, and the three visual acquisition directions are evenly arranged along the circumference, with an angle of 120° between adjacent perspectives. By employing a three-view image acquisition and surface unfolding fusion technology, a unified representation of the coating state on the insulator's outer surface is achieved. This solves the problems of incomplete coating evaluation caused by traditional single-view images failing to cover the entire curved surface, skirt obstruction, and the absence of visibility on the back side, significantly improving the accuracy of overall coating coverage assessment.

[0027] S2. Perform normalization enhancement processing on the image from each angle to obtain the enhanced four-channel image; Images from each angle undergo standardization and enhancement processing. This is achieved through local illumination normalization, color correction, local contrast enhancement, and prior estimation of reflective areas. This reduces the impact of uneven lighting, edge reflections, shadows, and camera color drift on spraying status recognition. This improves the robustness of the subsequent spraying status segmentation model under complex lighting, reflection, and shadow environments.

[0028] Specifically, the steps include the following: S201. Use local mean filtering to estimate the local illumination components of each single-view RGB image; Local illumination component Represented as: , in, This represents the normalized original RGB image, where H and W represent the image height and width, respectively. Indicates the window size is Local mean filtering; S202. Perform illumination normalization on the original image by combining the local illumination component to obtain the normalized image. , represented as: , , in, Indicates average light intensity. To prevent division by zero of extremely small constants, This means limiting the pixel value to the range of [0,1]. Its function is to reduce local shadows and uneven lighting, making the color of the sprayed area more stable under different lighting conditions.

[0029] S203. Perform channel-by-channel color correction on the illumination-normalized image and output the color-corrected image. : , in, These are the learnable gain parameters for the RGB three channels. These are the learnable bias parameters for the three RGB channels, and ⊙ indicates multiplication channel by channel. , The initial training values ​​are 1 and 0, respectively. Their purpose is to correct color shifts caused by different cameras, light sources, and shooting conditions.

[0030] S204. Perform local contrast enhancement on the color-corrected image to obtain the enhanced image; First, calculate the local mean of the color-corrected image. : , Then calculate the local standard deviation : , Then, local normalization and contrast enhancement are performed to obtain the locally enhanced image. ; , in, Indicates the window size is k Local mean filtering of 2, This is a local contrast enhancement factor, which enhances the texture and color differences between painted areas, unpainted areas, and abnormally painted areas.

[0031] S205. Prior estimation of reflective areas based on the color-corrected image; Highly reflective areas in RTV spray-painted images typically have the characteristics of "high brightness and low saturation".

[0032] First, calculate the brightness map. : , Recalculate the saturation approximation diagram : , in, c This represents the RGB color channel.

[0033] Based on brightness and saturation Calculate the prior map of the reflective area : , in, This represents the Sigmoid function. The brightness threshold. The saturation threshold, and These are the corresponding coefficients.

[0034] When a certain area has high brightness and low saturation A value close to 1 indicates that the area may have reflections or be an unreliable indicator.

[0035] S206. Correct the color image Locally enhanced images and reflective prior diagram The image is fused to obtain a standardized enhanced image. , represented as: , in, To enhance the fusion coefficient, This is the reflection suppression coefficient.

[0036] when A smaller value indicates a lower reflectivity in the area, resulting in an output that incorporates more locally enhanced images. ;when A larger value indicates that there may be strong reflections in that area, and the output will retain more of the color-corrected image. This suppresses the enhancement of reflective error.

[0037] The reflective prior image is stitched together with the normalized enhanced image: ,at this time: The first three channels are standardized and enhanced RGB images, and the fourth channel is a priori map of the reflective area. This output provides a more stable image input for the subsequent spraying state segmentation network and helps it identify uncertain areas.

[0038] S3: Based on the enhanced four-channel image and the trained lightweight spraying state segmentation network, the spraying state probability matrix and the uncertainty region probability matrix are obtained. Specifically, the steps include the following: S301: Extract detail feature matrix based on enhanced four-channel image and generate prior matrix of reflective area; Specifically, the steps include the following: S3011: Extracts the initial detail feature matrix based on a four-channel image, and simultaneously extracts detail features at two scales based on the initial detail feature matrix, including small missed spray edges and large-area spray texture variations; Using the four-channel image output from the previous stage as input, features of RTV spraying boundary, missed spray edge, local texture and small abnormal area are extracted at a higher resolution. At the same time, the reflection prior matrix is ​​used to suppress the interference of unreliable areas on detailed features.

[0039] First, a shallow convolution mapping is performed on the four-channel input, and the image resolution is downsampled to half of the original image to obtain the initial detail feature matrix: , in, This represents a convolution operation with a kernel size of 3×33 and a stride of 2. BN ( ) indicates batch normalization. delta ( ) represents a nonlinear activation function.

[0040] Output: ,in, Number of channels; Due to limitations in the computing power of edge computing devices, and to reduce computational load, depthwise separable convolution is used to extract local spatial features. Depthwise separable convolution is defined as: , in, This represents depthwise convolution, used to extract spatial details; This represents pointwise convolution, used to fuse channel information.

[0041] To simultaneously extract both fine unsprayed edges and larger-area spray texture variations, 3×3 and 5×5 depth-separable convolutions were used respectively: , , in, The focus is on extracting fine edges, local textures, and small-area unsprayed features; The focus is on extracting spraying continuity and regional consistency within a larger neighborhood.

[0042] S3012: Channel splicing of detail features from two scales is performed to obtain a multi-scale fused detail feature matrix. The prior matrix of the reflective area is downsampled to the detail feature scale, and the reliability matrix is ​​calculated. The detail features at two scales are concatenated by channel, and then channel compression and fusion are achieved through 1×1 convolution to obtain a multi-scale fused detail feature matrix. : , Since highly reflective areas are prone to boundary misjudgment and texture misidentification, the prior matrix of reflective areas is downsampled to the detail feature scale: , , Further construct the reliability matrix : , Represents the prior matrix of reflective regions downsampled to the detailed feature scale; When the reflectivity is high Larger A smaller value indicates lower reliability of detailed features in that area; when the reflectivity is low,... A larger value indicates that the detailed features of the area are more reliable.

[0043] S3013: Fusing multi-scale detail feature matrices With reliability matrix Concatenate the data to generate a detail attention matrix. : , , in, It is used to enhance the spray boundaries, missed spray edges and local texture response in reliable areas, while reducing the interference of reflective areas on the extraction of detailed features.

[0044] S3014: Combining Detail Attention Matrix For the initial detail feature matrix Multi-scale fusion detail feature matrix and reliability matrix Integrate into the final detailed feature matrix; The main output of the detail feature extraction branch is: , Here, ⊙ represents element-wise multiplication. Used to enhance effective detail features Used to suppress unreliable reflective areas, residual term This is used to preserve shallow edge information and improve training stability.

[0045] The final detailed feature matrix is ​​obtained as follows: .

[0046] To represent the response intensity of spray boundaries, missed spray edges, and local abnormal edges, and to provide boundary constraints for subsequent spray state segmentation heads, a boundary prior matrix is ​​generated from the detail feature matrix. : , , S302: Extracting semantic context feature matrix based on enhanced four-channel image; The semantic branch employs a lightweight multi-scale attention module to extract global spraying state context information from low-resolution feature maps, thereby improving the model's robustness to reflections, shadows, and local color fluctuations. The semantic context extraction branch extracts the overall spraying state, background relationships, and large-scale contextual information from the insulator RTV spraying image. Unlike the detail feature extraction branch, which focuses on boundaries and local textures, the semantic context branch prioritizes global region judgment, reducing misjudgments caused by reflections, shadows, and local color fluctuations. Specifically, the steps include the following: A shallow convolution mapping is performed on the four-channel input, and the image resolution is downsampled to half of the original image to obtain the initial semantic feature matrix: , in, This represents a convolution operation with a kernel size of 3×3 and a stride of 2. Denotes batch normalization, δ( ) represents a nonlinear activation function; This yields... , C 0 indicates the initial number of semantic feature channels.

[0047] Based on the inverse residual module, the spatial resolution is gradually reduced to obtain the downsampled semantic feature matrix; Semantic feature extraction is performed using an inverse residual module. The inverse residual module is defined as follows: , in, E ( ) represents a 1×1 up-dimensional convolution. Ds ( ) represents a 3×3 depthwise convolution with stride s. P ( ) represents a 1×1 dimension-reduced convolution. S ( ) indicates a residual connection.

[0048] When the input and output sizes are the same: , When the input and output dimensions are inconsistent, the residual connection is omitted, and the size matching is achieved by 1×1 convolution.

[0049] Among them, dimensionality-up convolution Depth convolution Dimensionality reduction convolution They are respectively: , , , in, This represents pointwise convolution. This represents a depthwise convolution with a stride of s.

[0050] Based on the inverse residual module, the semantic branch gradually reduces the spatial resolution and enhances the semantic expressive power: , , in, , C1 represents the number of intermediate semantic feature channels, and Cs represents the number of semantic context feature channels.

[0051] After two downsampling operations, the feature map resolution is reduced to one-eighth of the original image, which helps to expand the receptive field and reduce the amount of subsequent computation.

[0052] Multi-scale context features are obtained by extracting context features from the downsampled semantic feature matrix using depthwise convolutions with different dilation rates. To enhance the network's ability to perceive sprayed areas at different scales, Multi-scale dilated depthwise convolution is introduced. Contextual features are extracted using depthwise convolutions with different dilation rates: , , , in, r This represents the void ratio. A smaller void ratio is used to extract local semantic information, while a larger void ratio is used to expand the receptive field and capture the spraying state context over a larger area.

[0053] Contextual features at different scales , and The data is concatenated, and channel compression and fusion are performed using 1×1 convolution to obtain a multi-scale up-down fused feature matrix: , in: .

[0054] Global average pooling is performed on the multi-scale context feature matrix, and semantic attention vectors are generated through lightweight channel mapping. To enable the network to adaptively adjust the importance of different semantic channels based on the overall spraying status, global semantic attention is introduced. First, global average pooling is performed on the multi-scale context fusion features: , in: .

[0055] To enhance semantic channel responses relevant to the painting state and suppress irrelevant channel responses corresponding to background, shadows, and local disturbances, a semantic attention vector is generated through lightweight channel mapping: , in, .

[0056] The final semantic context feature matrix is ​​obtained based on the semantic attention vector, the multi-scale up-down fusion feature matrix, and the downsampled semantic feature matrix; The semantic context feature matrix is ​​represented as follows: , in, ;⊙ indicates channel-by-channel multiplication. These are residual semantic features used to preserve the original contextual information and improve training stability.

[0057] The semantic context extraction branch completes the extraction from the four-channel standardized input. To obtain a low-resolution, highly semantically expressive context feature matrix The mapping. This feature can be upsampled to the detail feature scale in the subsequent lightweight attention fusion module, and mapped with the detail feature matrix. The process is fused to balance the boundary accuracy of the spraying state segmentation with the global semantic robustness.

[0058] S303: Adaptively fuse the detail feature matrix, semantic context feature matrix, and reflective region prior matrix to obtain the fused feature matrix; The multi-scale lightweight attention fusion module is used to fuse the high-resolution detail features output from the detail feature extraction branch with the low-resolution semantic features output from the semantic context extraction branch, enabling the model to simultaneously possess local boundary recognition capabilities and global semantic judgment capabilities.

[0059] First, the semantic context feature matrix Upsampling is performed to adjust its spatial size and detail feature matrix. Maintain consistency: , Then, the upsampled semantic features are compressed using a 1×1 convolution, reducing the number of channels from... Become This yields a semantic feature matrix whose scale and channels are consistent with the detailed features. : , To enhance the expressive power of semantic features for sprayed areas at different scales, multi-scale semantic features were extracted using depthwise convolutions with different hole ratios: , , , in, The void ratio represents the porosity. A smaller void ratio is used to preserve local semantic details, while a larger void ratio is used to expand the receptive field and capture a wider range of spraying state context.

[0060] Semantic features at three scales , and The data are concatenated and fused using 1×1 convolution to obtain a multi-scale semantically enhanced feature matrix: , Subsequently, the multi-scale semantic enhancement feature matrix was processed. Perform global average pooling to obtain the channel description vector. : , To enhance feature channels related to painted, unpainted, and abnormal painted states, and suppress invalid channels related to background, shadow, and reflection interference, channel attention vectors are generated based on channel description vectors. : , Further Perform average pooling and max pooling along the channel direction respectively: , , in, Represents the average response matrix. Represents the maximum response matrix; To enhance the spatial response of spray boundaries, missed spray edges, and local abnormal areas, the average response matrix is... Maximum response matrix Concatenate the boundary prior matrix with the boundary prior matrix to generate a boundary-guided spatial attention matrix. : , Based on channel attention vector Spatial attention matrix The multi-scale semantic enhancement features are recalibrated to obtain effective semantic features. ; , in, This represents element-wise multiplication, a step that simultaneously enhances effective semantic features from both the channel and spatial dimensions.

[0061] To adaptively select detailed or semantic features based on different regions, a gated fusion matrix is ​​constructed. : , When a certain area requires more boundary and texture information A larger value indicates that the fusion result is more biased towards detailed features; when a certain region requires more contextual semantic judgment, ... When the value is small, the fusion result is more biased towards semantic features.

[0062] Finally, the multi-scale lightweight attention fusion module outputs a fused feature matrix: , Fusion feature matrix It also includes spraying boundary, missed spray edge and local texture information in the detail branch, as well as overall spraying state, background suppression and context judgment information in the semantic branch, which serve as inputs for subsequent spraying state output head, boundary output head and uncertain area output head.

[0063] S304: Combine the fusion feature matrix, the boundary prior matrix and the output head to obtain the spraying state probability matrix, the boundary probability matrix and the uncertainty region probability matrix; First, fuse the feature matrix Upsample the image to restore it to the original resolution: , Then the fused feature matrix is ​​restored to the original image resolution. Feature refinement is performed using lightweight convolution to obtain a high-resolution feature matrix shared by the output head. : , Simultaneously, the boundary prior matrix is ​​upsampled to the original image resolution to obtain the boundary prior matrix that restores the original image resolution. : , The spraying condition segmentation head specifically includes: Let the number of spraying state categories be . , respectively represent: :background, Areas already painted Unpainted areas Abnormal spraying area.

[0064] The spraying status segmentation head outputs a classification score matrix through 1×1 convolution. : , The probability matrix of the spraying state is obtained through Softmax. The pixel position (x, y) belongs to the first... c Probability of spray-painted state : , The output is: , The final pixel-level category prediction is: , Boundary-assisted output heads for feature sharing Taking the sampled boundary prior matrix as input, the output boundary probability matrix is ​​generated. ; First, we obtain the boundary probability matrix: , in, is the boundary prior adjustment coefficient.

[0065] The uncertain area output head is used to identify areas with reflections, occlusions, blurriness, or unstable model judgments.

[0066] Establish the probability matrix of the uncertainty region : .

[0067] During the training phase, the output head is constrained using a multi-task joint loss: , in, Loss due to segmentation during spraying process. For boundary loss, Losses in uncertain areas, For Dice's loss, , , The corresponding loss weights.

[0068] , in, For category weights, For the first c Similar to real labels.

[0069] The boundary loss is: , in, Indicates the true label of the boundary.

[0070] The loss in the uncertain region is: , in, This indicates the true label for an uncertain region.

[0071] Dice's loss is: , , in, Indicates the predicted probability of unpainted areas; The label represents the true label of the unpainted area; Dice loss is used to enhance the model's ability to identify small areas of missed spraying.

[0072] A lightweight spraying state segmentation network is adopted, which combines detailed feature extraction, semantic context extraction and multi-scale lightweight attention fusion to achieve pixel-level recognition of sprayed areas, unsprayed areas, abnormal sprayed areas and uncertain areas. Compared with traditional grid partitioning or image block classification methods, it can more accurately locate the position of spraying defects and improve the recognition accuracy of spraying state on complex curved surfaces.

[0073] S4: Establish an insulator surface unfolding coordinate system, and map the spraying state probability matrix and the uncertainty region probability matrix from multiple perspectives to a unified unfolding space; perform confidence-weighted fusion of the multi-view mapping results according to the perspective angle weight, image quality weight, and model uncertainty weight to obtain the fused spraying state probability map and fused uncertainty map in the unified unfolding space. The image segmentation results acquired by the painting robot from three different orientations are mapped to a unified insulator surface unfolding space and fused according to visual visibility, viewing angle, image quality and model uncertainty.

[0074] For the unfolded diagram, let the probability matrix of the spraying state at the i-th viewpoint be... for: , Where B is the batch size.

[0075] No. i Probability matrix of uncertain regions from various perspectives for: , Establish expansion coordinates: The surface of the insulator is approximately unfolded into a two-dimensional plane: ,in, z Indicates the axial position of the insulator. phi It represents a circumferential angle.

[0076] The dimensions of the merged result after unfolding are: ,in, and These represent the height and width of the insulator surface development diagram, respectively.

[0077] As a further implementation method, the multi-view mapping results are subjected to confidence-weighted fusion based on viewpoint weights, image quality weights, and model uncertainty weights, specifically including: Viewpoint weight: For any point in the unfolded diagram ( z , phi ), calculate its relative to the first i The difference in angle from each perspective: ,in, This represents the angle normalization function. Indicates the first i The installation angle of each visual acquisition viewpoint relative to the central axis of the insulator.

[0078] Then calculate the viewpoint weights: ,in, gamma The angle attenuation coefficient is usually set to 2; that is, the closer to the direction the camera is facing, the greater the weight; the closer to the side or back, the smaller the weight.

[0079] Image quality weights: Image quality can be simply represented by the proportion of the uncertain region: , Indicates the first i Image quality weights for each viewpoint This represents the probability matrix of the uncertain region corresponding to the i-th viewpoint at the two-dimensional coordinate position (x, y). In other words, the greater the reflection, occlusion, and blurring at a certain viewpoint, the larger the proportion of the uncertain region. It gets smaller.

[0080] Image quality weighted by model confidence: The first i The segmentation probability matrix and the probability matrix of the uncertain region from each perspective are mapped onto the expanded coordinates to obtain... , , in, Indicates the first i Mapping operations from one viewpoint to the insulator unfolded diagram.

[0081] Fusion weight:i Each perspective at the unfolding point The fusion weights at each point are: , in, Indicates the first i Probability of uncertain regions after mapping from each perspective; Three-view probabilistic fusion: The fused spraying state probability matrix is ​​as follows: ,in, Indicates the unfolded position after the three perspectives are merged. The probability of being in the c-th type of spraying state. Indicates the first i The probability of the c-th type of spraying state after mapping from each viewpoint.

[0082] The final category is: , This indicates the final coating state category after fusion.

[0083] The probability of the uncertain region after fusion is: ,in, This indicates the coordinate position after the three perspectives are merged. The probability of uncertainty at a given point. Indicates the first i The probability value of the uncertain region after a viewpoint is mapped by surface unfolding.

[0084] By establishing a coordinate system for the unfolded surface of the insulator, the spraying states from different perspectives are uniformly mapped to the same unfolded space. The system combines the weights of the perspective angle, image quality, and model uncertainty to perform fusion calculations, effectively overcoming the evaluation errors caused by repeated statistics from multiple perspectives, local occlusion, edge reflection, and differences in image quality. This improves the stability and robustness of spraying quality evaluation in complex field environments.

[0085] S5: Based on the fused spraying state probability map and fused uncertainty map, calculate the overall spraying coverage, overall missed spraying rate, abnormal spraying ratio, and uncertain area ratio, and output the spraying effect evaluation results and spraying grade.

[0086] To further derive image-level spraying evaluation metrics from pixel-level output, we first define the probability matrix of effective spraying areas. : ,in, This represents the probability matrix of the background category.

[0087] The effective area is: , Indicates the unfolded position of the insulator surface Probability of an effective evaluation region at a given location; The spraying ratio is: , This indicates the position after the three perspectives are merged. The probability of belonging to the already painted category; The uncoated proportion is: , Indicates position The probability of belonging to the unpainted category; The abnormal spraying ratio is: , Indicates position The probability of belonging to the abnormal spraying category; The proportion of the uncertain region is: , This represents the probability of uncertainty in the coating state after three-view fusion. Final output coating evaluation vector: , The overall score is calculated as follows: , where α, β, γ, and η are weighting coefficients.

[0088] Then, the spraying grades are determined based on the scores, specifically including: When the overall score The coating grade is level five. When the overall score The coating grade is level four. When the overall score The coating grade is level three. When the overall score The coating grade is Level 2. When the overall score The coating grade is Level 1. , represented as: The higher the grade, the better the quality of the coating.

[0089] Based on the fused spraying state probability map, the system calculates the overall sprayed coverage rate, overall missed spraying rate, abnormal spraying ratio, and uncertain area ratio. It further identifies the differences between continuous missed spraying areas and scattered small-area defects, ultimately outputting spraying effect evaluation results and spraying grade. This provides a quantitative, stable, and interpretable evaluation basis for the RTV spraying quality acceptance of insulators. This makes the spraying quality evaluation results more consistent with actual engineering needs and provides a reliable basis for RTV spraying quality acceptance.

[0090] Example 2 This embodiment provides an online evaluation system for the spraying effect of an insulator RTV spraying robot, including: The multi-view image enhancement module is used to perform normalization enhancement processing on the acquired multi-view images to obtain enhanced four-channel images. A lightweight image segmentation module is used to obtain the spraying state probability matrix and the uncertainty region probability matrix based on the enhanced four-channel image and the trained lightweight spraying state segmentation network. The construction process of the lightweight spraying state segmentation network includes: Based on the enhanced four-channel image, detail feature matrix and semantic context feature matrix are extracted, and boundary prior matrix is ​​generated. The detail feature matrix, semantic context feature matrix and boundary prior matrix are adaptively fused to obtain fused feature matrix. The fused feature matrix, boundary prior matrix and output head are combined to obtain spraying state probability matrix, boundary probability matrix and uncertainty region probability matrix. The multi-view mapping weighting module is used to establish the insulator surface unfolding coordinate system and map the spraying state probability matrix and the uncertainty region probability matrix under multiple views to a unified unfolding space. Based on the view angle weight, image quality weight and model uncertainty weight, the multi-view mapping results are fused with confidence weight to obtain the fused spraying state probability map and fused uncertainty map under the unified unfolding space. The spraying effect evaluation module is used to calculate the overall sprayed coverage, overall missed spraying rate, abnormal spraying ratio, and uncertain area ratio based on the fused spraying state probability map and fused uncertainty map, and output the spraying effect evaluation results and spraying grade.

[0091] It should be noted that the specific implementation of the online evaluation system for the coating effect of the RTV coating robot of the insulator in this embodiment of the invention is similar to the specific implementation of the online evaluation method for the coating effect of the RTV coating robot of the insulator in this embodiment of the invention. For details, please refer to the description in the method section. In order to reduce redundancy, it will not be repeated here.

[0092] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the online evaluation method for the spraying effect of the insulator RTV spraying robot as described above.

[0093] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the online evaluation method for the spraying effect of the insulator RTV spraying robot as described above.

[0094] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0095] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0096] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0097] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0098] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An online evaluation method for the spraying effect of an RTV spraying robot for insulators, characterized in that, Includes the following steps: The acquired multi-view images are standardized and enhanced to obtain enhanced four-channel images. Based on the enhanced four-channel image and the trained lightweight spraying state segmentation network, the spraying state probability matrix and the uncertainty region probability matrix are obtained; the construction process of the lightweight spraying state segmentation network includes: Based on the enhanced four-channel image, detail feature matrix and semantic context feature matrix are extracted, and boundary prior matrix is ​​generated. The detail feature matrix, semantic context feature matrix and boundary prior matrix are adaptively fused to obtain fused feature matrix. The fused feature matrix, boundary prior matrix and output head are combined to obtain spraying state probability matrix, boundary probability matrix and uncertainty region probability matrix. A coordinate system for the expansion of the insulator surface is established, and the probability matrices of the spraying state and the probability matrices of the uncertain region under multiple viewpoints are mapped to a unified expansion space. The multi-view mapping results are then fused with confidence weights based on the viewpoint weights, image quality weights, and model uncertainty weights to obtain a fused spraying state probability map and a fused uncertainty map under the unified expansion space. Based on the fused spraying state probability map and fused uncertainty map, the overall spraying coverage, overall missed spraying rate, abnormal spraying ratio, and uncertain area ratio are calculated, and the spraying effect evaluation results and spraying grade are output.

2. The online evaluation method for the spraying effect of the insulator RTV spraying robot as described in claim 1, characterized in that, The normalization and enhancement processing of the acquired multi-view images includes: Local illumination components of RGB images from each single viewpoint are estimated using local mean filtering. ; The original image is normalized by incorporating local illumination components to obtain the normalized image. ; Perform channel-by-channel color correction on the illumination-normalized image and output a color-corrected image. ; Local contrast enhancement is performed on the color-corrected image to obtain the locally enhanced image. ; Based on color-corrected image Prior estimation of reflective areas is performed to obtain a prior map of reflective areas. ; Color-corrected image Locally enhanced images and reflective prior diagram The image is fused to obtain a standardized enhanced image. The reflective prior image With normalized enhanced images The images are stitched together to obtain an enhanced four-channel image. The first three channels are the standardized enhanced RGB images, and the fourth channel is the prior image of the reflective area.

3. The online evaluation method for the spraying effect of the insulator RTV spraying robot as described in claim 1, characterized in that, The extraction of the detail feature matrix based on the enhanced four-channel image includes: A shallow convolution mapping is performed on the four-channel input, and the image resolution is downsampled to half of the original image to obtain the initial detail feature matrix. ; Based on the initial detail feature matrix, multi-scale detail features of small missed spray edges and spray texture variations are extracted simultaneously. The extracted multi-scale detail features are concatenated through channels to obtain a multi-scale fused detail feature matrix. ; The prior matrix of the reflective region is downsampled to the scale of detailed features to calculate the reliability matrix. ; Multi-scale fusion of detail feature matrices With reliability matrix Concatenate the data to generate a detail attention matrix. ; Combining detail attention matrix For the initial detail feature matrix Multi-scale fusion detail feature matrix and reliability matrix Fusion into the final detail feature matrix .

4. The online evaluation method for the spraying effect of the insulator RTV spraying robot as described in claim 1, characterized in that, The semantic context feature matrix is ​​extracted based on the enhanced four-channel image, including: A shallow convolution mapping is performed on the four-channel input, and the image resolution is downsampled to half of the original image to obtain the initial semantic feature matrix. ; Based on the inverse residual module, the spatial resolution is gradually reduced to obtain the downsampled semantic feature matrix. ; Multi-scale context features are obtained by extracting context features from the downsampled semantic feature matrix using depthwise convolutions with different dilation rates. Contextual features at different scales are concatenated, and channel compression and fusion are performed using 1×1 convolution to obtain a multi-scale up-down fused feature matrix. ; Global average pooling is performed on the multi-scale context feature matrix, and semantic attention vectors are generated through lightweight channel mapping. ; Based on semantic attention vectors Multi-scale up-down fusion feature matrix and the downsampled semantic feature matrix The final semantic context feature matrix is ​​obtained. .

5. The online evaluation method for the spraying effect of the insulator RTV spraying robot as described in claim 1, characterized in that, The adaptive fusion of the detail feature matrix, semantic context feature matrix, and reflective region prior matrix includes: semantic context feature matrix Upsampling is performed, and the upsampled semantic features are channel-compressed using 1×1 convolutions to obtain a semantic feature matrix whose scale and number of channels are consistent with the detailed features. ; Depth convolutions with different dilation rates were used to extract semantic feature matrices. Multi-scale semantic features are extracted and then concatenated and fused to obtain a multi-scale semantic enhancement feature matrix. Multi-scale semantic enhancement feature matrix Perform global average pooling to obtain the channel description vector. Generate channel attention vectors based on channel description vectors ; right Average pooling and max pooling are performed along the channel direction to obtain the average response matrix. and maximum response matrix ; Average response matrix Maximum response matrix Concatenate the boundary prior matrix with the boundary prior matrix to generate a boundary-guided spatial attention matrix. ; Based on channel attention vector Spatial attention matrix Multi-scale semantic enhancement feature matrix Relabeling is performed to obtain effective semantic features. ; Based on the adaptive selection of detailed features or semantic features in different regions, a gated fusion matrix is ​​constructed. ; Combined with gated fusion matrix Effective semantic features and detailed feature matrix Obtain the fusion feature matrix .

6. The online evaluation method for the spraying effect of the insulator RTV spraying robot as described in claim 1, characterized in that, The loss function of the lightweight spraying state segmentation network during training. for: ， ， ， ， ， in, Loss due to segmentation during spraying process. For category weights, For the first c Similar to real-world labels, This indicates that the pixel position (x, y) belongs to the first... c The probability of a spray-painted state. Indicates the number of spraying status categories. For boundary loss, Represents the boundary probability matrix. Indicates the true label of the boundary. Losses in uncertain areas, Represents the probability matrix of the region of uncertainty. Indicates the true label of an uncertain area. For Dice's loss, This indicates the predicted probability of unpainted areas. This indicates the actual label in the unpainted area. To prevent division by zero of extremely small constants, , , The corresponding loss weights.

7. The online evaluation method for the spraying effect of the insulator RTV spraying robot as described in claim 1, characterized in that, The fusion spraying state probability diagram and fusion uncertainty diagram are represented as follows: ， ， in, Indicates the unfolding point after the three perspectives are merged. The probability of being in the c-th type of spraying state. Indicates the i-th perspective at the unfolding point. The fusion weight at point z represents the axial position of the insulator. Represents the circumferential angle. Let represent the probability of the c-th spraying state after the i-th viewpoint is mapped by surface unfolding. To prevent division by zero of extremely small constants, This represents the probability of uncertainty after fusion. This represents the probability of the uncertain region after mapping the i-th viewpoint.

8. An online evaluation system for the spraying effect of an RTV spraying robot for insulators, characterized in that, include: The multi-view image enhancement module is used to perform normalization enhancement processing on the acquired multi-view images to obtain enhanced four-channel images. A lightweight image segmentation module is used to obtain the spraying state probability matrix and the uncertainty region probability matrix based on the enhanced four-channel image and the trained lightweight spraying state segmentation network. The construction process of the lightweight spraying state segmentation network includes: Based on the enhanced four-channel image, detail feature matrix and semantic context feature matrix are extracted, and boundary prior matrix is ​​generated. The detail feature matrix, semantic context feature matrix and boundary prior matrix are adaptively fused to obtain fused feature matrix. The fused feature matrix, boundary prior matrix and output head are combined to obtain spraying state probability matrix, boundary probability matrix and uncertainty region probability matrix. The multi-view mapping weighting module is used to establish the insulator surface unfolding coordinate system and map the spraying state probability matrix and the uncertainty region probability matrix under multiple views to a unified unfolding space. Based on the view angle weight, image quality weight and model uncertainty weight, the multi-view mapping results are fused with confidence weight to obtain the fused spraying state probability map and fused uncertainty map under the unified unfolding space. The spraying effect evaluation module is used to calculate the overall sprayed coverage, overall missed spraying rate, abnormal spraying ratio, and uncertain area ratio based on the fused spraying state probability map and fused uncertainty map, and output the spraying effect evaluation results and spraying grade.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the online evaluation method for the spraying effect of the insulator RTV spraying robot as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the online evaluation method for the spraying effect of the insulator RTV spraying robot as described in any one of claims 1-7.

Images