Transformer substation scene binocular depth estimation method based on knowledge extension learning normal form training

Abstract

The invention discloses a transformer substation scene binocular depth estimation method based on knowledge extension learning normal form training. The method comprises the following steps: collecting a large amount of transformer substation scene binocular image data based on a binocular telephoto camera; pre-training the student model based on the public virtual data set; generating a pseudo-true value label of the real power transmission channel scene data set based on the large model; on the basis of the false label on the label-free real data, the real scene data is incorporated into an adjustment and optimization training data set of the student model, so that the performance of the student model in the real scene is improved; based on a model capacity improvement strategy in a learning iteration process, the complexity and parameter quantity of a student model are gradually improved in the iteration process, so that the model obtains larger capacity.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a binocular depth estimation method for substation scenes based on a knowledge extension learning paradigm. Background Technology

[0002] With the continuous development of modern power systems, near-electric work in substations faces increasingly higher safety risks. These operations involve close contact with high-voltage equipment, making their safety paramount and directly related to the lives of workers and the stability of the power grid. Traditional safety management methods mainly rely on manual monitoring and experience-based judgment, but these methods suffer from poor real-time performance, insufficient accuracy, low efficiency, and susceptibility to errors.

[0003] To improve the safety of near-electric work at substations, 3D point cloud technology can be used. This technology uses high-resolution cameras to acquire images of the work area from multiple perspectives in real time, and uses semantic segmentation to distinguish between workers and high-voltage equipment. Combined with binocular depth estimation, the system can generate 3D point cloud coordinates for both workers and high-voltage equipment, thereby accurately calculating the distance between them. When the distance approaches or falls below a set safety threshold, the system will issue a warning signal in real time to ensure the safety of the work process.

[0004] The 3D point cloud-based safety distance control system boasts advantages such as high precision, real-time performance, and comprehensive coverage. It not only accurately measures the distance between workers and high-voltage equipment, avoiding human error, but also monitors safety distances in real time, improving operational safety. Furthermore, through multi-view image acquisition and processing, the system can achieve comprehensive monitoring of all near-electrical work areas within the substation, reducing reliance on manual monitoring, lowering labor costs, and improving monitoring efficiency.

[0005] By introducing advanced 3D point cloud technology, the safety management of near-electric work in substations has achieved a qualitative leap. This technology can not only accurately measure the distance between workers and high-voltage equipment in real time, but also automatically issue warnings when the distance approaches a safety threshold, greatly improving the safety of the work process. Furthermore, through multi-view image acquisition and processing, the system achieves comprehensive monitoring of all near-electric work areas within the substation, effectively compensating for the shortcomings of traditional manual monitoring. This high-precision and high-efficiency monitoring method not only reduces manpower consumption and monitoring costs, but also provides detailed data support for substation safety management, enabling the recording and analysis of historical data. This data not only helps to promptly identify and resolve potential safety hazards, but also provides a strong reference for future safety management strategies. Through this advanced technology, the safety management level of substations will be significantly improved, providing solid technical support for the stable operation of the power system and the safety of personnel.

[0006] A search revealed that Chinese patent application CN202311720500.8 discloses a robust high-voltage power line segmentation method for sparse point clouds and its application. This method preprocesses sparse point clouds generated by a single-wavelength lidar, performs point cloud rasterization on the preprocessed point cloud frames, and coarsely segments the power lines within the point cloud frames. Finally, based on the coarse segmentation results, density filtering and random sampling consistency line fitting are used to accurately extract power line points in the three-dimensional space of the point cloud. However, this method requires large-scale scanning using lidar to acquire point cloud data, resulting in high manpower and material costs. Furthermore, it can only acquire sparse point cloud data, making it difficult to obtain background point cloud data, which is not conducive to further development of related tasks in substation scenarios.

[0007] To better achieve 3D reconstruction of substation scenes and obtain 3D information, a feasible method utilizes deep learning to acquire depth information of all objects from the camera's perspective through binocular depth estimation for 3D reconstruction. The biggest challenge of this method is obtaining high-precision depth information on easily deployable models. Therefore, we use a binocular depth estimation method for substation scenes trained using a knowledge-extended learning paradigm. This method employs data distillation, data augmentation, and model enhancement within the knowledge-extended learning paradigm to overcome noise interference in model training, increase the capacity of the student model, and achieve performance improvements in real-world scenarios. Summary of the Invention

[0008] To improve the accuracy of binocular depth estimation networks in substation scenarios, this invention provides a binocular depth estimation method for substation scenarios based on a knowledge extension learning paradigm. This method involves acquiring a large amount of binocular image data of substation scenarios using a binocular telephoto camera, and then generating pseudo-ground value labels for a real substation scenario dataset using a large model. The student model is pre-trained on a publicly available virtual dataset, and then fine-tuned for real substation scenarios using pseudo-labels on unlabeled real data. Finally, based on a model capacity enhancement strategy during the learning iteration process, the complexity and number of parameters of the student model are gradually increased during the iteration process, enabling the model to achieve greater capacity.

[0009] To achieve the above objectives, this invention discloses a binocular depth estimation method for substation scenarios based on a knowledge extension learning paradigm, comprising the following steps: Step S10: Based on a dedicated binocular telephoto camera, a large amount of real and unlabeled data from the substation scene is efficiently collected, thus providing rich and accurate information resources for subsequent data analysis and processing. Step S20: Use publicly available virtual datasets as training materials to systematically pre-train the student model, thereby laying a solid foundation for its excellent performance in subsequent practical applications and significantly improving its learning and adaptability. Step S201: There are now a large number of open-source, publicly available stereo depth estimation virtual datasets. Select a random object dataset such as Sceneflow. Step S202: To enhance the generalization ability of the model, the student model is pre-trained using a large-scale virtual dataset; Step S30: Generate pseudo-ground value labels for the real substation scenario dataset based on the large model, i.e., the teacher model; Step S40: Optimize the student model based on pseudo-labels on unlabeled real data to improve its performance in substation scenarios; Step S401: First, use a large model to predict real-world substation data, and then filter the results appropriately to remove undesirable parts. Step S402: Use the dataset where the inference results of the large model are close to the true value as the tuning dataset to tune the student model; Step S50: By adopting a model capacity enhancement strategy during the learning iteration process, the complexity and number of parameters of the student model are gradually increased, so that it can obtain greater capacity to better adapt to and handle complex task requirements. Step S501: For parts such as model disparity optimization, set up an interface that can increase the number of times the disparity optimization is performed. The more times the disparity optimization is performed, the better the inference results and the larger the number of model parameters. Step S502: Iterate from a small model capacity to a larger model capacity to ensure that the model fully learns features at different levels. Attached Figure Description

[0010] Figure 1 This invention describes the process and specific strategies for training a binocular depth estimation network.

[0011] Figure 2 This is a schematic diagram of the teacher / student model of the present invention. Detailed Implementation

[0012] The following describes the embodiments of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0013] This embodiment presents a binocular depth estimation method for substation scenes based on a knowledge extension learning paradigm. The method includes: acquiring a large amount of binocular image data of substation scenes using a binocular telephoto camera; pre-training a student model using a publicly available virtual dataset; generating pseudo-ground value labels for a real power transmission channel scene dataset based on the large model; incorporating real scene data into the student model's optimization training dataset based on the pseudo-labels on unlabeled real data, thereby improving the student model's performance in real scenes; and gradually increasing the complexity and number of parameters of the student model during the iterative learning process using a model capacity enhancement strategy, thereby enabling the model to achieve greater capacity. This invention addresses the problem of reduced network learning ability caused by limiting the complexity of the student model to prevent overfitting in the initial stage when the amount of labeled data is small, by developing a multi-stage iterative learning method under the knowledge extension paradigm. By leveraging the knowledge extension training paradigm of model augmentation and data distillation, the generalization of depth perception is improved, solving the problem of difficulty in creating labels for real-world depth estimation datasets. Through training a student binocular depth estimation network, accurate depth estimation for substation scenarios is achieved. It has application potential in multiple scenarios, such as achieving 360-degree panoramic coverage of near-electric work areas, eliminating blind spots, accurately measuring and controlling safe distances, minimizing the risk of electric shock, and accurately depicting personnel behavior trajectories in complex occlusion scenarios. It has advantages such as high universality, low cost, and high accuracy.

[0014] The specific steps are as follows: Step S10: As... Figure 2 As shown, the invention proposes to adopt a teacher-student training and iterative optimization model. Given labeled data... and unlabeled data First, a teacher model is trained based on labeled data: (1) Predicting pseudo-labels on unlabeled real data using a trained teacher model: (2) This leads to the incorporation of real-world scenario data into the student model. During training: (3) This improves the performance of the student model in real-world scenarios.

[0015] Step S20: Improve the generalization of deep perception based on the knowledge expansion training paradigm of model augmentation and data distillation; Step S201: Add a confidence estimation module to the multi-view depth estimation framework, and design a corresponding data distillation strategy based on the confidence to optimize model training and reduce interference to the student model. The formula is shown below: (4) p represents the estimated confidence level, and c represents the target. Step S202: By enhancing real-world scene data in terms of color, geometry, etc., and addressing complex situations such as camera misalignment and dirt interference, design and implement corresponding data augmentation methods to improve the generalization ability of the knowledge extension paradigm to the model in real-world scenes; Step S30: Use the model augmentation method under the knowledge extension paradigm to enable the student model to train deep perception of real scenes under noisy pseudo-label data. The student model should have sufficient model capacity in its design to enable the model to be compatible with noise and have error correction capabilities, so as to make the model's performance in real scenes approach the theoretical limit based on the knowledge extension paradigm. Step S301: Dropout is a regularization technique to prevent overfitting in deep neural networks. It increases the generalization ability of the model by randomly shutting down some neurons during training. The objective formula is shown below: (5)

[0016] Step 5 S302: Random view transformation can significantly enhance the diversity of the dataset, improve the model's generalization ability and robustness to various situations; Step S40: Depth estimation belongs to the dense label prediction problem. Relying on the teacher model to predict unlabeled data to generate pseudo-labels to expand the training data will introduce a lot of noise. The project intends to focus on studying how to improve the capacity of the learning model under large label noise through methods such as data distillation, model augmentation and novel adaptive penalty functions. Step S401: The dataset distillation algorithm takes a large real dataset (training set) as input and processes it to generate a simplified synthetic distillation dataset. Then, the performance of the model trained on this distillation dataset is evaluated using a separate real dataset (validation / test set). Step S402: The dataset distillation algorithm takes a large real dataset (training set) as input and processes it to generate a simplified synthetic distillation dataset. Then, the performance of the model trained on this distillation dataset is evaluated using a separate real dataset (validation / test set), with the target formula as follows: (6) (7) The dynamic penalty function is a calculation where the penalty factor increases over time, where C, , Let t be a constant and t be an algebra.

Claims

1. A binocular depth estimation method for substation scenarios based on a knowledge extension learning paradigm, characterized in that, Includes the following steps: S10: Based on a binocular telephoto camera, a large amount of real, unlabeled data from substation scenes is collected; S20: Pre-train the student model based on a publicly available virtual dataset; S30: Generating pseudo-true labels for real substation scenario datasets based on a large model, i.e., a teacher model; S40: Based on pseudo-labels on unlabeled real data, optimize the student model to improve its performance in substation scenarios. S50: Based on the model capacity enhancement strategy in the learning iteration process, the complexity and number of parameters of the student model are gradually increased in the iteration process, so that the student model can obtain greater capacity.

2. The binocular depth estimation method for substation scenarios based on knowledge extension learning paradigm training as described in claim 1, characterized in that, The method of pre-training student models using publicly available virtual datasets enhances the generalization ability of the models by utilizing a large number of virtual datasets.

3. The binocular depth estimation method for substation scenarios based on knowledge extension learning paradigm training as described in claim 1, characterized in that, The method described above for generating pseudo-true labels for real substation scenario datasets using large models is difficult to deploy on edge devices, but the large number of parameters allows it to obtain near-true prediction results in multiple scenarios.

4. The binocular depth estimation method for substation scenarios based on knowledge extension learning paradigm training as described in claim 1, characterized in that, Based on the method of optimizing the model using real-world scenario data, further optimization was performed using a newly created power transmission channel dataset containing pseudo-labels, bringing the model's performance in this specific scenario closer to the theoretical limit.

5. The binocular depth estimation method for substation scenarios based on knowledge extension learning paradigm training as described in claim 1, characterized in that, Based on the model capacity enhancement strategy in the learning iteration process, the complexity and number of parameters of the student model are gradually increased during the iteration process to enable the student model to obtain a larger capacity. By gradually increasing the number of model parameters and conducting iterative training, the model parameters of different layers can learn different features in a targeted manner, thereby accelerating the model convergence and improving the model's learning capacity in the knowledge expansion iteration.

Images