Stereoscopic target detection method and system of transformer substation inspection robot and computer equipment

Abstract

The invention discloses a transformer substation inspection robot three-dimensional target detection method and system and computer equipment, and the method comprises the steps: obtaining a high-voltage transformer substation inspection multi-view image, carrying out the preprocessing of different three-dimensional targets of the multi-view image, and constructing a multi-view three-dimensional target detection network based on a deep learning network. Comprising a 2D image feature extraction module, a 3D spatial feature extraction module and a detection module, the 2D image feature extraction module comprises a backbone network and a continuous visual state space module in a discrete form, a multi-view stereoscopic target detection network is trained, and a stereoscopic target detection result is finally output. Through the combination of deep learning and a multi-view image technology, the problems of blind spots of traditional manual inspection and false inspection and missing inspection caused by a traditional visual technology are reduced, the intelligent level of inspection is effectively improved, the labor cost is saved, a more reliable technical guarantee is provided for safe operation of the high-voltage transformer substation, and the high-voltage transformer substation inspection method is suitable for popularization and application. And the stability of a power system and the competitiveness of an enterprise are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power inspection, and in particular relates to a three-dimensional target detection method, system and computer equipment for a substation inspection robot. Background Art

[0002] With the rapid development of the power industry, high-voltage substations, as a vital component of the power system, are crucial for the safe and stable operation of the entire power supply chain. Traditional substation inspections rely primarily on manual inspections. While these methods can detect equipment failures promptly, they are inefficient, have limited coverage, and are often influenced by human factors, making them inadequate for the high safety and reliability demands of modern power systems. Therefore, an efficient and intelligent inspection method is urgently needed to enhance substation detection capabilities.

[0003] In recent years, the rapid development of computer vision and image processing technologies has provided new solutions for intelligent inspection of power equipment. Multi-view imaging technology, by capturing device images from different angles, can more comprehensively capture the device's appearance and status information. However, existing multi-view image object detection methods still have some shortcomings in terms of feature extraction, target recognition, and data fusion for power equipment, especially in terms of detection accuracy and robustness in complex environments.

[0004] Currently, some research attempts to combine deep learning with multi-view imagery to improve the accuracy of stereo object detection. However, in the specific environment of high-voltage substations, the wide variety of equipment and complex operating conditions make existing methods difficult to achieve ideal results in practical applications. Therefore, it is urgent to develop a stereo object detection method for intelligent inspection of high-voltage substations using multi-view imagery.

[0005] Currently, deep learning methods primarily use cameras as image sensors, feeding a large number of multi-view images of high-voltage substations into the model for training. This results in the neural network's target detection loss, guiding further training and ultimately completing stereo target detection in high-voltage substations. However, the implementation faces the following difficulties: 1. The complexity of 3D object detection during high-voltage substation inspections: High-voltage substation inspections involve both large objects such as transformers and high-voltage towers, as well as smaller objects such as workers and insulators. Conventional convolutional networks cannot fully capture both global and local features of the objects. This limits the model's detection capabilities for objects of varying sizes, ultimately leading to false or missed detections.

[0006] 2. Object occlusion: In high-voltage substations, due to the numerous and complexly distributed equipment, inspection targets are often obscured. However, traditional vision technologies detect obscured objects, which reduces the missed detection rate. Therefore, it is necessary to improve the model's contextual understanding capabilities. Furthermore, a feature fusion module in a continuous visual state space is incorporated to enhance the perception of obscured objects, effectively alleviating the missed detection of obscured objects.

[0007] Based on the above problems, a three-dimensional target detection solution for a substation inspection robot is proposed in the present invention. Summary of the Invention

[0008] The present invention provides a three-dimensional target detection method, system and computer equipment for a substation inspection robot. Its purpose is to overcome the problems faced in high-voltage substation inspections such as complex detection targets and easy obstruction, and to solve technical problems such as low detection efficiency and insufficient robustness of traditional methods.

[0009] The technical solution adopted by the present invention to solve the technical problem is: A three-dimensional target detection method for a substation inspection robot, the method comprising the following steps: S100: An image sensor is installed on a power inspection robot to capture multi-view images of high-voltage substations. The image capture is performed on high-voltage equipment from different angles and positions. S200: Calibrate the image sensor to obtain the camera's internal and external parameters. Label the collected multi-view images, clearly mark different stereo objects, and perform data enhancement on the labeled images to obtain a multi-view stereo object detection dataset for high-voltage substation inspections. S300: Builds a multi-view stereo object detection network based on a deep learning network, including a 2D image feature extraction module, a 3D spatial feature extraction module, and a detection module. The 2D image feature extraction module includes a Resnet-50 backbone network and a discrete continuous visual state space module. S400: The multi-view images in the dataset are simultaneously input into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is performed based on the Resnet-50 backbone network combined with the continuous visual state space module to obtain 2D feature information. The 2D feature information is mapped into 3D space using the camera's back-projection operation to obtain a 3D feature map. The 3D feature map is input into the 3D spatial feature extraction module to extract spatial features of stereo objects of different sizes to obtain a spatial feature map. The spatial feature map is input into the detection head for detection, and the detection bounding box and classification results are output. S500: Design a loss function, and perform supervised training on the multi-view stereo object detection network based on the loss function. When the preset training end condition is reached, the trained multi-view stereo object detection network is obtained, which is used to detect the multi-view images acquired in real time and output the category and bounding box of the stereo object.

[0010] Preferably, S200 includes: S210: Use the ROS toolkit to calibrate the camera sensor internal parameters and obtain the camera's internal parameters; S220: Using the inspection robot's onboard image sensor to collect multi-view images from different positions and angles, annotating the stereo objects of interest in the multi-view images, including the object's category and stereo bounding box; S230: Perform data enhancement processing on the labeled data set, and divide the high-voltage substation inspection multi-view image data set into a training set and a test set according to a preset ratio.

[0011] Preferably, in S400, the multi-view images in the data set are simultaneously input into the 2D image feature extraction module, and the 2D feature extraction of the multi-view images is implemented based on the Resnet-50 backbone network combined with the discrete continuous visual state space module to obtain the 2D feature information, including: S410: The multi-view image is input into the 2D image feature extraction module. The Resnet-50 backbone network performs preliminary 2D feature extraction on the multi-view image to generate a preliminary 2D feature map. S420: Each preliminary two-dimensional feature map is then converted into an image sequence, and these image sequences are reconstructed by splicing the number of channels. Then, according to different input orders, the reconstructed image sequences are sent to the continuous visual state space. In the continuous visual state space, the input sequence blocks are processed through temporal recursive calculation. By combining the temporal continuity of the multi-view image sequence, not only the receptive field of the model is expanded, but also the hidden state is updated recursively, effectively capturing the temporal information in the sequence, and finally obtaining a two-dimensional feature map that integrates the temporal information. S430: Send the two-dimensional features with time sequence information output by the continuous visual state space to a multi-layer perceptron to obtain two-dimensional feature information.

[0012] Preferably, in S420, the input sequence blocks are processed by temporal recursive calculation in the continuous visual state space. By combining the temporal continuity of the multi-view image sequence, not only the receptive field of the model is expanded, but also the hidden state is updated recursively, effectively capturing the temporal information in the sequence, and finally obtaining a two-dimensional feature map that integrates the temporal information, including: S421: Image sequences with different scanning orders The input to the continuous visual state space block is first normalized, specifically: ; S422: After normalization, it is divided into two branches. One of them is linearized and then input into the convolution and activation function and then into the SSM state space model. Specifically: ; S423: Another information branch is a combination of a linear function and an activation function, specifically: ; S424: Finally, a residual connection is designed to prevent performance degradation caused by the network being too deep, and a two-dimensional feature map that integrates temporal information is obtained. , specifically: .

[0013] Preferably, the mathematical formula of the SSM state space model in S422 is expressed as: ; ; in express The hidden state of the moment, represents the input image sequence, , They are all matrix parameters, which are used to control the conversion relationship between state and input.

[0014] Preferably, in S400, the two-dimensional feature information is mapped into a three-dimensional space using a back-projection operation of the camera to obtain a three-dimensional feature map, specifically: ; in , Represent three-dimensional and two-dimensional feature information respectively, represents the intrinsic parameter matrix of the camera, Camera extrinsic matrix.

[0015] Preferably, the loss function in S500 is specifically: ; in, is the number of positive samples detected, is the position regression loss, which represents the error between the predicted position and the true position; Represents classification loss, which is used to calculate the accuracy of classification prediction; It is the center loss, which measures the center positioning accuracy of the target detection box; The loss function is defined as: ; Centerness loss Used to measure the relative distance between the center and edge of the target box, defined as: ; in, Indicates the actual location, represents the predicted position, represents a smoothing function, the purpose of which is to reduce the impact of error outliers on the loss, and M is the total number of samples; The loss function is defined as: ; in Indicates the true value of the label, represents the model’s predicted probability for the true label, Indicates the important factor for balancing positive and negative samples, used to balance the influence of positive and negative samples. represents a parameter that controls the impact of difficult and easy samples; The loss function is defined as: ; ; in, is the shortest distance from the center of the target box to the edge of the target, is the longest distance from the center of the target box to the target box; ; in, is the true category label, is the model's predicted probability for the class.

[0016] The substation inspection robot stereo target detection system includes a multi-view image acquisition module, a data set acquisition module, a multi-view stereo target detection network construction module, a stereo target detection module, and a training module. The multi-view image acquisition module is used to install an image sensor on the power inspection robot to collect multi-view images of the high-voltage substation, and to capture multi-view images of the high-voltage equipment from different angles and positions; The dataset acquisition module is used to calibrate the image sensor to obtain the camera's internal and external parameters. It then annotates the collected multi-view images, clearly marking different stereo targets. It then performs data enhancement on the annotated images to obtain a multi-view stereo target detection dataset for high-voltage substation inspections. A multi-view stereo object detection network construction module is used to build a multi-view stereo object detection network based on a deep learning network. It includes a 2D image feature extraction module, a 3D spatial feature extraction module, and a detection module. The 2D image feature extraction module includes a Resnet-50 backbone network and a discrete continuous visual state space module. The stereo object detection module is used to simultaneously input the multi-view images in the dataset into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is realized based on the Resnet-50 backbone network combined with the continuous visual state space module. The 2D feature information is mapped into the 3D space using the camera's back projection operation to obtain a 3D feature map. The 3D feature map is input into the 3D spatial feature extraction module to extract the spatial features of stereo objects of different sizes to obtain a spatial feature map. The spatial feature map is input into the detection head for detection, and the detection bounding box and classification results are output. The training module is used to design a loss function and perform supervised training on the multi-view stereo object detection network based on the loss function. When the preset training end conditions are reached, the trained multi-view stereo object detection network is obtained, which is used to detect multi-view images acquired in real time and output the category and bounding box of the stereo object.

[0017] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of a three-dimensional target detection method for a substation inspection robot when executing the computer program.

[0018] In response to the problems of complex targets and equipment occlusion in high-voltage substation inspections, the present invention proposes a three-dimensional target detection method, system, and computer equipment for substation inspection robots based on a multi-perspective image state space model. This method can significantly improve the accuracy and efficiency of target detection during high-voltage substation inspections, especially in complex environments with equipment occlusion and a wide variety of targets. By combining deep learning with multi-perspective image technology, the blind spots of traditional manual inspections and the problems of false detection and missed detection caused by traditional visual technology are reduced, effectively improving the intelligence level of inspections, saving labor costs, and providing more reliable technical guarantees for the safe operation of high-voltage substations, thereby improving the stability of the power system and the competitiveness of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flowchart of a three-dimensional target detection method for a substation inspection robot according to one embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a continuous visual state space module in one embodiment of the present invention; Figure 3 Schematic diagram of the structure of a continuous visual state space block in one embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0021] In one embodiment, Figure 1 As shown, a three-dimensional target detection method for a substation inspection robot includes the following steps: S100: An image sensor is installed on a power inspection robot to capture multi-view images of high-voltage substations. The image capture is performed on high-voltage equipment from different angles and positions. S200: Calibrate the image sensor to obtain the camera's internal and external parameters. For the collected multi-view images, label the images to clearly identify different stereo objects (such as wires, insulators, transformers, circuit breakers, and personnel). Perform data enhancement on the labeled images to obtain a multi-view stereo object detection dataset for high-voltage substation inspections. S300: Builds a multi-view stereo object detection network based on a deep learning network, including a 2D image feature extraction module, a 3D spatial feature extraction module, and a detection module. The 2D image feature extraction module includes a Resnet-50 backbone network and a discrete continuous visual state space module. S400: The multi-view images in the dataset are simultaneously input into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is performed based on the Resnet-50 backbone network combined with the continuous visual state space module to obtain 2D feature information. The 2D feature information is mapped into 3D space using the camera's back-projection operation to obtain a 3D feature map. The 3D feature map is input into the 3D spatial feature extraction module to extract spatial features of stereo objects of different sizes to obtain a spatial feature map. The spatial feature map is input into the detection head for detection, and the detection bounding box and classification results are output. S500: Design a loss function, and perform supervised training on the multi-view stereo object detection network based on the loss function. When the preset training end condition is reached, the trained multi-view stereo object detection network is obtained, which is used to detect the multi-view images acquired in real time and output the category and bounding box of the stereo object.

[0022] Specifically, this paper combines multi-view image acquisition with deep learning to propose a method for intelligent 3D object detection during high-voltage substation inspections. To address the inefficiencies associated with target complexity and occlusion encountered by traditional inspection recognition algorithms, a continuous visual state space module is proposed, significantly improving the accuracy and robustness of 3D object detection during inspections.

[0023] In one embodiment, S200 includes: S210: Use the ROS toolkit to calibrate the camera sensor internal parameters and obtain the camera's internal parameters; S220: Using the inspection robot's onboard image sensor to collect multi-view images from different positions and angles, annotating the stereo objects of interest in the multi-view images, including the object's category and stereo bounding box; S230: Perform data augmentation processing (scaling, rotation, flipping, color change and other data augmentation operations) on the labeled dataset to improve the data richness of the dataset and thus improve the generalization ability of the trained model. The high-voltage substation inspection multi-view image dataset is divided into a training set and a test set according to a preset ratio (7:3).

[0024] Specifically, in a high-voltage substation inspection application, an inspection robot equipped with an image sensor captures multi-view images of high-voltage equipment from various angles and positions. The camera's internal and external parameters are calibrated using the ROS toolkit to ensure image acquisition accuracy and consistency. The captured images are manually annotated to identify object categories and define stereo bounding boxes, providing a high-quality dataset for subsequent deep learning training. Furthermore, the data undergoes augmentation operations such as rotation, scaling, and flipping to enhance the diversity and robustness of the training set.

[0025] Furthermore, in S400, the multi-view stereo object detection model converts the input multi-view image into spatial information for detection. Specifically, it involves 2D image feature extraction combined with the continuous visual state space module, camera back-projection calculation, and 3D spatial feature extraction.

[0026] In one embodiment, Figure 2 As shown, in S400, the multi-view images in the data set are simultaneously input into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is realized based on the backbone network of Resnet-50 combined with the discrete continuous visual state space module, and the 2D feature information is obtained, including: S410: The multi-view image is input into the 2D image feature extraction module. The Resnet-50 backbone network performs preliminary 2D feature extraction on the multi-view image to generate a preliminary 2D feature map. S420: Each preliminary two-dimensional feature map is then converted into an image sequence, and these image sequences are reconstructed by splicing the number of channels. Then, according to different input orders, the reconstructed image sequences are sent to the continuous visual state space. In the continuous visual state space, the input sequence blocks are processed through temporal recursive calculation. By combining the temporal continuity of the multi-view image sequence, not only the receptive field of the model is expanded, but also the hidden state is updated recursively, effectively capturing the temporal information in the sequence, and finally obtaining a two-dimensional feature map that integrates the temporal information. S430: Send the two-dimensional features with time sequence information output by the continuous visual state space to a multi-layer perceptron to obtain two-dimensional feature information.

[0027] In one embodiment, Figure 3 As shown, in S420, the input sequence blocks are processed through temporal recursive calculation in the continuous visual state space. By combining the temporal continuity of the multi-view image sequence, not only the receptive field of the model is expanded, but also the temporal information in the sequence is effectively captured by recursively updating the hidden state. Finally, a two-dimensional feature map that integrates the temporal information is obtained, including: S421: Image sequences with different scanning orders The input to the continuous visual state space block is first normalized, specifically: ; S422: After normalization, it is divided into two branches. One of them is linearized and then input into the convolution and activation function and then into the SSM state space model. Specifically: ; S423: Another information branch is a combination of a linear function and an activation function, specifically: ; S424: Finally, a residual connection is designed to prevent performance degradation caused by the network being too deep, and a two-dimensional feature map that integrates temporal information is obtained. , specifically: .

[0028] In one embodiment, the mathematical formula of the SSM state space model in S422 is expressed as: ; ; in express The hidden state of the moment, represents the input image sequence, , They are all matrix parameters, which are used to control the conversion relationship between state and input. The hidden state expression at the moment is composed of the current input and Moment correlation enables the perception of continuous time-series images, thereby understanding the more accurate geometric relationships of targets in multi-view images, and ultimately achieving better detection of occluded targets in a single image.

[0029] Specifically, the continuous visual state space is an improvement on the visual state space model, incorporating the temporal continuity of multi-view images. The spliced image sequences are fed into the continuous visual state space block in different input orders to obtain a larger receptive field.

[0030] In one embodiment, in S400, the two-dimensional feature information is mapped into a three-dimensional space using a back-projection operation of the camera to obtain a three-dimensional feature map, specifically: ; in , Represent three-dimensional and two-dimensional feature information respectively, represents the intrinsic parameter matrix of the camera, Camera extrinsic matrix.

[0031] Furthermore, the 3D features obtained by back-projection calculation are input into the three-dimensional spatial feature extraction module (composed of multi-scale three-dimensional convolution), and the obtained features are input into the detection network.

[0032] In one embodiment, the loss function in S500 is specifically: ; in, is the number of positive samples detected, is the position regression loss, which represents the error between the predicted position and the true position; Represents classification loss, which is used to calculate the accuracy of classification prediction; It is the center loss, which measures the center positioning accuracy of the target detection box; The loss function is defined as: ; Centerness loss Used to measure the relative distance between the center and edge of the target box, defined as: ; in, Indicates the actual location, represents the predicted position, represents a smoothing function, the purpose of which is to reduce the impact of error outliers on the loss, and M is the total number of samples; The loss function is defined as: ; in Indicates the true value of the label, represents the model’s predicted probability for the true label, Indicates the important factor for balancing positive and negative samples, used to balance the influence of positive and negative samples. represents a parameter that controls the impact of difficult and easy samples; The loss function is defined as: ; ; in, is the shortest distance from the center of the target box to the edge of the target, is the longest distance from the center of the target box to the target box; ; in, is the true category label, is the model's predicted probability for the class.

[0033] Finally, backpropagation is used to jointly train the continuous visual state space module and the multi-view stereo detection network to minimize the aforementioned loss function. The network weights are then updated to their optimal value, completing the joint training. The test dataset is then fed into the jointly trained model. Images collected in real time during high-voltage substation inspections are then fed into the jointly trained stereo object detection network, which outputs the stereo object category and bounding box.

[0034] The above-mentioned stereoscopic object detection method for substation inspection robots uses multi-view image acquisition to comprehensively scan equipment from different angles. Deep learning techniques are used to construct a multi-view stereoscopic object detection network. Combined with the feature extraction and fusion capabilities of visual state space, this method can better identify equipment targets in complex scenes, especially those that are occluded or have large size variations. This significantly improves the accuracy and robustness of object detection. This addresses the challenges of traditional power inspection methods, which often rely on single-view images or manual inspections, and struggle to fully capture equipment details. Furthermore, it addresses the challenges faced by high-voltage substations, which are characterized by a large variety of equipment, severe occlusion, and large object size variations. After joint training, the network can process image data collected during inspections in real time, output device categories and bounding box information, and promptly detect equipment faults and anomalies. This network effectively improves the model's ability to recognize occluded objects and objects with large size variations, reducing missed and false detections. Compared to traditional manual inspection methods, this technology significantly improves detection accuracy, demonstrating greater robustness in detecting equipment occlusion and small objects. This new technology improves the inspection efficiency and accuracy of high-voltage substations, enhances the ability to perceive targets in complex environments, and provides intelligent support for early warning and maintenance of equipment failures.

[0035] In one embodiment, a stereo object detection system for a substation inspection robot is provided, the system comprising a multi-view image acquisition module, a data set acquisition module, a multi-view stereo object detection network construction module, a stereo object detection module, and a training module; The multi-view image acquisition module is used to install an image sensor on the power inspection robot to collect multi-view images of the high-voltage substation, and to capture multi-view images of the high-voltage equipment from different angles and positions; The dataset acquisition module is used to calibrate the image sensor to obtain the camera's internal and external parameters. It then annotates the collected multi-view images, clearly marking different stereo targets. It then performs data enhancement on the annotated images to obtain a multi-view stereo target detection dataset for high-voltage substation inspections. A multi-view stereo object detection network construction module is used to build a multi-view stereo object detection network based on a deep learning network. It includes a 2D image feature extraction module, a 3D spatial feature extraction module, and a detection module. The 2D image feature extraction module includes a Resnet-50 backbone network and a discrete continuous visual state space module. The stereo object detection module is used to simultaneously input the multi-view images in the dataset into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is realized based on the Resnet-50 backbone network combined with the continuous visual state space module. The 2D feature information is mapped into the 3D space using the camera's back projection operation to obtain a 3D feature map. The 3D feature map is input into the 3D spatial feature extraction module to extract the spatial features of stereo objects of different sizes to obtain a spatial feature map. The spatial feature map is input into the detection head for detection, and the detection bounding box and classification results are output. The training module is used to design a loss function and perform supervised training on the multi-view stereo object detection network based on the loss function. When the preset training end conditions are reached, the trained multi-view stereo object detection network is obtained, which is used to detect multi-view images acquired in real time and output the category and bounding box of the stereo object.

[0036] The specific definition of the three-dimensional target detection system for the substation inspection robot can be found in the definition of the three-dimensional target detection method for the substation inspection robot above, and will not be repeated here. The various modules in the above-mentioned three-dimensional target detection system for the substation inspection robot can be implemented in whole or in part through software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of a three-dimensional target detection method for a substation inspection robot when executing the computer program.

[0038] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0039] The above is a detailed introduction to the multi-view stereoscopic target detection method, system and computer equipment for the working robot provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional target detection method for a substation inspection robot, characterized in that: The method comprises the following steps: S100: An image sensor is installed on a power inspection robot to capture multi-view images of high-voltage substations. The image capture is performed on high-voltage equipment from different angles and positions. S200: Calibrate the image sensor to obtain the camera's internal and external parameters. Label the collected multi-view images, clearly mark different stereo objects, and perform data enhancement on the labeled images to obtain a multi-view stereo object detection dataset for high-voltage substation inspections. S300: Builds a multi-view stereo object detection network based on a deep learning network, including a 2D image feature extraction module, a 3D spatial feature extraction module, and a detection module. The 2D image feature extraction module includes a Resnet-50 backbone network and a discrete continuous visual state space module. S400: The multi-view images in the dataset are simultaneously input into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is performed based on the Resnet-50 backbone network combined with the continuous visual state space module to obtain 2D feature information. The 2D feature information is mapped into 3D space using the camera's back-projection operation to obtain a 3D feature map. The 3D feature map is input into the 3D spatial feature extraction module to extract spatial features of stereo objects of different sizes to obtain a spatial feature map. The spatial feature map is input into the detection head for detection, and the detection bounding box and classification results are output. S500: Design a loss function, and perform supervised training on the multi-view stereo object detection network based on the loss function. When the preset training end condition is reached, the trained multi-view stereo object detection network is obtained, which is used to detect the multi-view images acquired in real time and output the category and bounding box of the stereo object.

2. The method according to claim 1, characterized in that S200 includes: S210: Use the ROS toolkit to calibrate the camera sensor internal parameters and obtain the camera's internal parameters; S220: Using the inspection robot's onboard image sensor to collect multi-view images from different positions and angles, annotating the stereo objects of interest in the multi-view images, including the object's category and stereo bounding box; S230: Perform data enhancement processing on the labeled data set, and divide the high-voltage substation inspection multi-view image data set into a training set and a test set according to a preset ratio.

3. The method according to claim 2, characterized in that In S400, the multi-view images in the dataset are simultaneously input into the 2D image feature extraction module. The 2D feature extraction of the multi-view images is performed based on the Resnet-50 backbone network combined with the discrete continuous visual state space module. The 2D feature information is obtained, including: S410: The multi-view image is input into the 2D image feature extraction module. The Resnet-50 backbone network performs preliminary 2D feature extraction on the multi-view image to generate a preliminary 2D feature map. S420: Each preliminary two-dimensional feature map is then converted into an image sequence, and these image sequences are reconstructed by splicing the number of channels. Then, according to different input orders, the reconstructed image sequences are sent to the continuous visual state space. In the continuous visual state space, the input sequence blocks are processed through temporal recursive calculation. By combining the temporal continuity of the multi-view image sequence, not only the receptive field of the model is expanded, but also the hidden state is updated recursively, effectively capturing the temporal information in the sequence, and finally obtaining a two-dimensional feature map that integrates the temporal information. S430: Send the two-dimensional features with time sequence information output by the continuous visual state space to a multi-layer perceptron to obtain two-dimensional feature information.

4. The method according to claim 3, characterized in that In S420, the input sequence blocks are processed through temporal recursive calculations within the continuous visual state space. By combining the temporal continuity of the multi-view image sequence, not only does the model's receptive field expand, but the hidden state is also updated recursively, effectively capturing the temporal information in the sequence. Ultimately, a two-dimensional feature map that incorporates temporal information is obtained, including: S421: Image sequences with different scanning orders The input to the continuous visual state space block is first normalized, specifically: ； S422: After normalization, it is divided into two branches. One of them is linearized and then input into the convolution and activation function and then into the SSM state space model. Specifically: ； S423: Another information branch is a combination of a linear function and an activation function, specifically: ； S424: Finally, a residual connection is designed to prevent performance degradation caused by the network being too deep, and a two-dimensional feature map that integrates temporal information is obtained. , specifically: 。 5. The method according to claim 4, characterized in that The mathematical formula of the SSM state space model in S422 is: ； ； in express The hidden state of the moment, represents the input image sequence, , They are all matrix parameters, which are used to control the conversion relationship between state and input.

6. The method according to claim 5, characterized in that In S400, the back-projection operation of the camera is used to map the two-dimensional feature information into the three-dimensional space to obtain a three-dimensional feature map, specifically: ； in , Represent three-dimensional and two-dimensional feature information respectively, represents the intrinsic parameter matrix of the camera, Camera extrinsic matrix.

7. The method according to claim 6, characterized in that The loss function in S500 is specifically: ； in, is the number of positive samples detected, is the position regression loss, which represents the error between the predicted position and the true position; Represents classification loss, which is used to calculate the accuracy of classification prediction; It is the center loss, which measures the center positioning accuracy of the target detection box; The loss function is defined as: ； Centerness loss Used to measure the relative distance between the center and edge of the target box, defined as: ; in, Indicates the actual location, represents the predicted position, represents a smoothing function, the purpose of which is to reduce the impact of error outliers on the loss, and M is the total number of samples; The loss function is defined as: ； in Indicates the true value of the label, represents the model’s predicted probability for the true label, Indicates the important factor for balancing positive and negative samples, used to balance the influence of positive and negative samples. represents a parameter that controls the impact of difficult and easy samples; The loss function is defined as: ； ； in, is the shortest distance from the center of the target box to the edge of the target, is the longest distance from the center of the target box to the target box; ； in, is the true category label, is the model's predicted probability for the class.

8. The three-dimensional target detection system of the substation inspection robot is characterized by: The system includes a multi-view image acquisition module, a data set acquisition module, a multi-view stereo object detection network construction module, a stereo object detection module and a training module; The multi-view image acquisition module is used to install an image sensor on the power inspection robot to collect multi-view images of the high-voltage substation, and to capture multi-view images of the high-voltage equipment from different angles and positions; The dataset acquisition module is used to calibrate the image sensor to obtain the camera's internal and external parameters. It then annotates the collected multi-view images, clearly marking different stereo targets. It then performs data enhancement on the annotated images to obtain a multi-view stereo target detection dataset for high-voltage substation inspections. A multi-view stereo object detection network construction module is used to build a multi-view stereo object detection network based on a deep learning network. It includes a 2D image feature extraction module, a 3D spatial feature extraction module, and a detection module. The 2D image feature extraction module includes a Resnet-50 backbone network and a discrete continuous visual state space module. The stereo object detection module is used to simultaneously input the multi-view images in the dataset into the 2D image feature extraction module. Based on the Resnet-50 backbone network combined with the continuous visual state space module, it realizes the 2D feature extraction of the multi-view images to obtain 2D feature information. The 2D feature information is mapped into the 3D space using the camera's back projection operation to obtain a 3D feature map. The 3D feature map is input into the 3D spatial feature extraction module to extract the spatial features of stereo objects of different sizes to obtain a spatial feature map. The spatial feature map is input into the detection head for detection, and the detection bounding box and classification results are output. The training module is used to design a loss function and perform supervised training on the multi-view stereo object detection network based on the loss function. When the preset training end conditions are reached, the trained multi-view stereo object detection network is obtained, which is used to detect multi-view images acquired in real time and output the category and bounding box of the stereo object.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images