Construction method and device of pre-training visual large model for power transmission service scene

Abstract

The invention belongs to the technical field of electric power visual model pre-training, and provides a construction method and device of a pre-training visual large model oriented to a power transmission service scene, and a power transmission service model optimization training method based on the pre-training large model through the technologies of sample quality evaluation, class imbalance sample learning and the like for the power transmission service scene. A business model special for the power transmission line is constructed, the problem of specific business adaptation of a general visual large model and the power transmission line is solved, and accurate recognition of defects of power transmission and transformation equipment is achieved. On the premise of ensuring the precision of a professional model, a parameter optimization method is utilized to further compress model parameters, so that the model performance is comprehensively improved, computing power resources are saved, the reusability of a scene model is improved, and the improvement of the intelligence and digitization levels of a novel power system is powerfully supported.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power vision model pre-training, and in particular to a method and device for constructing a large pre-trained vision model for power transmission business scenarios. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] At present, traditional deep learning models have been applied in the intelligent identification of defects in power transmission equipment, but the existing models still have problems such as limited model recognition accuracy, large demand for labeled sample data, and weak external model transferability during construction and use. Current deep learning models rely on pre-trained models trained with natural data, lack knowledge of power scenarios, and require a large number of labeled samples in the target detection stage, which greatly increases the labeling cost. For some business scenarios, there is a shortage of defective samples, which seriously affects the accuracy of the model; in addition, due to the complex and changeable application environment and differences in data distribution, the existing models are difficult to migrate and reuse, resulting in the need to continuously develop similar models, resulting in a large amount of human, computing power and other resources. Summary of the invention

[0004] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a construction and training method of a pre-trained visual large model for power transmission business scenarios. In the face of power transmission business scenarios, through sample quality assessment, category imbalance sample learning and other technologies, it conducts research on the optimization training method of the power transmission business model based on the pre-trained large model, constructs a business model dedicated to power transmission lines, solves the problem of adapting the general visual large model to the specific business of the transmission line, and realizes the accurate identification of defects in power transmission and transformation equipment. Under the premise of ensuring the accuracy of the professional model, the parameter optimization method is used to further compress the model parameters, comprehensively improve the model performance, save computing resources, improve the reusability of the scene model, and strongly support the improvement of the intelligence and digitalization level of the new power system.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A first aspect of the present invention provides a method for constructing a pre-trained visual large model for a power transmission business scenario, comprising the following steps:

[0007] The visual basic model is trained based on the power sample data to obtain a trained visual basic model;

[0008] On the basis of the basic visual model, the misclassification weight of the cost-sensitive vector is specified to adjust the balance relationship of the samples during training, thus obtaining the large electric power visual model.

[0009] The large electric power vision model is trained for transfer, and the knowledge distillation technology is used to perform one-to-one knowledge transfer from the teacher model in the deep model library to the student model with a given structure, so that deep models with the same task but different structures can obtain the same structure and obtain a business scenario model.

[0010] Furthermore, when the visual basic model is trained based on the power sample data, the DnCNN network model is used to suppress the noise in the transmission line defect image, and the noisy foreign body image is subtracted from the extracted residual image. When training DnCNN, the mean square error between the residual image and the estimated residual image is controlled to optimize the hyperparameters in the model.

[0011] Furthermore, the calculation formula of the mean square error is:

[0012]

[0013] In the formula, l(θ) is the mean square error, y i and x i They represent the i-th pair of noisy and denoised foreign body images respectively, and θ is a hyperparameter.

[0014] Furthermore, the misclassification weight of the cost-sensitive vector is specified to adjust the balance relationship of the samples during training, and the accuracy loss of the sample category is obtained by adding weights to the cross entropy using the Focal Loss function, and the training weight of the category is adjusted according to the accuracy loss of the sample category.

[0015] Furthermore, the loss function of the accuracy of the sample category is:

[0016] FL(p t )=-α t (1-p t ) γ log(p t )

[0017] In the formula, p t Represents the accuracy of different categories, α t It represents the weights set for different categories. The parameter γ is the attenuation coefficient. When the class is 1, it is multiplied by the weight α. When the class is 0, it is multiplied by the weight 1-α.

[0018] Furthermore, the method of performing one-to-one knowledge transfer from the teacher model in the deep model library to the given structure student model through the knowledge distillation technology so that the deep models with the same task but different structures obtain the same structure includes:

[0019] For single-task reorganization of deep models, the intermediate-layer features generated by the teacher model for the input image are used as intermediate-layer supervision information, and the predicted soft labels of the teacher model are used as task supervision information to construct a model deep reorganization algorithm with layer-by-layer fusion and compression.

[0020] For multi-task reorganization of deep models, the multi-task teacher model in the deep model library is used to construct a branching upper and lower bound estimation algorithm for given multi-tasks. Combined with the teacher model's predicted soft labels and layer-by-layer fusion and compression of the model deep reorganization algorithm, a deep model multi-task reorganization algorithm based on online branching is constructed to achieve the optimal multi-task student model construction.

[0021] A second aspect of the present invention provides a system for constructing a pre-trained visual large model for power transmission business scenarios, comprising:

[0022] A sample preprocessing module, which is used to train the visual basic model based on the power sample data to obtain a trained visual basic model;

[0023] The category learning module is used to adjust the balance relationship of samples during training by specifying the misclassification weights of cost-sensitive vectors on the basis of the basic visual model to obtain a large electric power visual model;

[0024] The scene lightweight module is used to perform transfer training on the large electric power vision model. Through the knowledge distillation technology, the teacher model in the deep model library is used to perform one-to-one knowledge transfer to the student model with a given structure, so that deep models with the same task but different structures obtain the same structure and obtain the business scene model.

[0025] Furthermore, in the scene lightweight module, the teacher model in the deep model library is subjected to one-to-one knowledge transfer to the given structure student model through the knowledge distillation technology, so that the deep models with the same task but different structures obtain the same structure, including:

[0026] For single-task reorganization of deep models, the intermediate-layer features generated by the teacher model for the input image are used as intermediate-layer supervision information, and the predicted soft labels of the teacher model are used as task supervision information to construct a model deep reorganization algorithm with layer-by-layer fusion and compression.

[0027] For multi-task reorganization of deep models, the multi-task teacher model in the deep model library is used to construct a branching upper and lower bound estimation algorithm for given multi-tasks. Combined with the teacher model's predicted soft labels and layer-by-layer fusion and compression of the model deep reorganization algorithm, a deep model multi-task reorganization algorithm based on online branching is constructed to achieve the optimal multi-task student model construction.

[0028] A third aspect of the present invention provides a computer-readable storage medium.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for constructing a pre-trained visual large model for power transmission business scenarios as described above.

[0030] A fourth aspect of the present invention provides a computer device.

[0031] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for constructing a pre-trained visual large model for power transmission business scenarios as described above are implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The present invention aims to solve the problem that due to the influence of shooting angle, lighting conditions, distance, collection equipment and other conditions, the quality of collected image samples of power business samples is quite different. Some image targets are large and clear, while some image targets are small and blurred. Samples of different quality will have different impacts on the business model. The visual basic model is trained based on the power sample data to construct a high-quality transmission equipment image sample library.

[0034] 2. When faced with small sample power scene data, traditional artificial intelligence methods are prone to overfitting and are difficult to have good generalization capabilities. The present invention uses small sample data to perform fine-tuning on the basis of a large pre-trained power vision model, so that the large model can be continuously iterated and optimized to achieve small sample or even zero sample power scene vision tasks. The pre-trained large model is superior to the traditional model in the ability to characterize image features, and can learn more feature information faster with less data. Therefore, for categories with few samples, the pre-trained large model has greater advantages for categories that have bottlenecks in accuracy improvement due to the small sample size in traditional models.

[0035] 3. The present invention makes the model training process more efficient through the power vision large model parameter optimization method based on model compression, and significantly reduces the number of model parameters without significantly reducing the accuracy. While improving the algorithm inference speed, it is beneficial to the deployment of the algorithm model on the terminal.

[0036] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0038] Figure 1 It is a schematic diagram of the construction method provided by an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of DnCNN image denoising provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0042] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0043] In the power transmission business scenario, the original small-scale pre-trained models all use natural data during training and do not contain proprietary knowledge of power scenarios. Their ability to be applicable to power business at a basic level is poor. Since the model parameter scale is smaller than that of large models and the model feature expression ability is poor, the model has many missed detections and false detections when applied in actual field applications.

[0044] Traditional models are more dependent on the number of samples, are not friendly to categories with few samples, and have low recognition accuracy. Pre-trained large models are better than traditional models in representing image features and can learn more feature information faster with less data. Therefore, for categories with few samples, pre-trained large models have greater advantages over categories where traditional models have bottlenecks in improving accuracy due to the small number of samples.

[0045] There are many types of defects in power transmission scenarios, and sometimes new defect categories may be added according to business needs. For traditional models, a large amount of sample data is required for the newly added defect categories, and training is started from scratch. The overall debugging cycle is long, the efficiency is low, and there is no reusability. However, for pre-trained large models, only a small number of samples are needed to achieve the required recognition capabilities, which is highly reusable and saves the cost of professional model research and development.

[0046] The technical route of the present invention is to train the electric power vision big model and build the scene model based on the visual basic big model. Use a large amount of electric power sample data to train the visual basic model L0, build the L1 electric power vision big model through model distillation and other technologies, and then transfer the training to obtain the L2 scene model.

[0047] The overall technical solution is: to study related technologies for optimizing training of pre-trained large-scale visual models in power transmission business scenarios. In the face of power transmission business scenarios, by studying model optimization technologies such as image quality improvement technology and sample imbalance learning technology, the model lightweight technology of the power transmission business model based on the pre-trained large model is used for model compression to build a business model dedicated to power transmission lines, solve the problem of adapting the general visual large model to the specific business of the transmission scenario, reduce the number of model parameters, improve the algorithm inference speed, and realize accurate identification of defects in power transmission equipment.

[0048] Embodiment 1

[0049] This embodiment provides a method for constructing and training a pre-trained visual large model for power transmission business scenarios, including the following steps:

[0050] Step 1: Train the visual basic model based on the power sample data to obtain a trained visual basic model;

[0051] Considering that the characteristics of the defects of the transmission line itself are significantly weak and are easily affected by factors such as viewing angle, background, and occlusion, the sample images usually contain a lot of noise, which may affect the detection results of the target detection model. Therefore, the input image samples need to be denoised preprocessed.

[0052] This embodiment uses the DnCNN network model to suppress noise in the power transmission line defect image. DnCNN is a convolutional neural network used for image denoising, which is optimized based on the VGG16 feedforward convolutional neural network.

[0053] The structure of DnCNN is as follows Figure 1 As shown, it mainly includes three parts: convolution layer (Conv), ReLU activation function and batch normalization (BN). In this embodiment, SGDM optimizer and L2 regularization are used in the training process of DnCNN, and the number of training rounds is 30, Batchsize is 64, and momentum is 0.9.

[0054] The principle of image denoising is to subtract the noisy foreign body image y from the extracted residual image v. In addition, when training DnCNN, the mean square error (MSE) between the residual image v and the estimated residual image R(y) is controlled to optimize the hyperparameter θ in the model. The smaller the MSE value, the closer the estimated residual image R(y) is to the residual image v, and the better the denoising effect.

[0055] The calculation formula of MSE is:

[0056]

[0057] In the formula, l(θ) is the mean square error, y i and x i They represent the i-th pair (i∈1, 2, …, N) of noise and noise-removed foreign body images respectively.

[0058] Since DnCNN is a denoising model for two-dimensional images, in practical applications, it is necessary to denoise the RGB channels of the defect image separately, and then superimpose the three denoised channels to achieve denoising of the color foreign body image.

[0059] The advantages of the above technology are:

[0060] Step 2: Based on the basic visual model, by specifying the misclassification weight of the cost-sensitive vector, the balance relationship of the samples during training is adjusted to obtain the large power visual model;

[0061] Power pre-trained visual large model sample imbalance category learning technology,In the power transmission business scenario, due to the different occurrence frequency of each defect category, the number of samples collected for each defect category is different, so there is a sample imbalance problem.

[0062] To alleviate the problem of class imbalance, this implementation uses the algorithm-level cost-sensitive method to deal with the sample imbalance problem. By specifying the misclassification weight of the cost-sensitive vector and increasing the loss of misclassified samples of the minority class, the impact of sample imbalance can be alleviated. Combining cost sensitivity with deep learning to solve classification problems can further improve the classification performance of the model.

[0063] This embodiment uses the Focal Loss (FL) loss function to deal with this problem.

[0064] The new loss function does not ignore any samples during training, which eliminates category imbalance and allows the model to focus more on difficult-to-distinguish samples during training.

[0065] Focal Loss first solves the problem of sample imbalance, that is, adding weight to the cross entropy.

[0066] Taking binary classification as an example, when the class is 1, multiply it by the weight α, and when the class is 0, multiply it by the weight 1-α, p t Represents the accuracy of different categories, α t Represents the weights set for different categories. On this basis, the core of Focal loss is to multiply the cross entropy by (1-p t ) to the power of γ, the formula is as follows:

[0067] FL(p t )=-α t (1-p t ) γ log(p t )

[0068] The parameter γ is called the decay coefficient. The formula means that the higher the accuracy, the smaller the overall loss value. The higher the accuracy of the class, the greater the decay. This allows the class with low accuracy to occupy most of the loss and dominate the training, thereby improving the accuracy of the difficult-to-classify class.

[0069] Step 3: Perform transfer training on the large electric vision model. Through the knowledge distillation technology, perform one-to-one knowledge transfer from the teacher model in the deep model library to the given structure student model, so that the deep models with the same task but different structures obtain the same structure and obtain the business scenario model.

[0070] The model lightweight technology of the power pre-trained visual large model in the power transmission scenario, and the research on the parameter optimization technology of the power vision large model based on model compression, make the model training process more efficient, and significantly reduce the number of model parameters without significantly reducing the accuracy. While improving the algorithm inference speed, it is conducive to the deployment of the algorithm model on the terminal.

[0071] Adopt model parameter optimization technology based on knowledge distillation: Since the deep model library has a wide range of sources, diverse types and different structures, the knowledge distillation technology is used to perform one-to-one knowledge transfer from the teacher model in the deep model library to the student model of a given structure, so that deep models with the same tasks but different structures can obtain the same structure.

[0072] Firstly, knowledge distillation technology is used to isomorphize the models in the deep model library to generate isomorphic models. For single-task reconstruction of deep models, the intermediate-layer features generated by the teacher model for the input image are used as the intermediate-layer supervision information. At the same time, the predicted soft labels of the teacher model are used as the task supervision information to construct a layer-by-layer fusion and compressed model deep reconstruction algorithm.

[0073] For multi-task restructuring of deep models, we first use the multi-task teacher model in the model library to construct a branching upper and lower bound estimation algorithm for given multi-tasks. Then, we combine the predicted soft labels of the teacher model and the aforementioned layer-by-layer fusion compression algorithm to construct a multi-task restructuring algorithm for deep models based on online branching, and achieve the optimal multi-task student model construction and training.

[0074] Embodiment 2

[0075] This embodiment provides a system for constructing a pre-trained visual large model for power transmission business scenarios, including:

[0076] A sample preprocessing module, which is used to train the visual basic model based on the power sample data to obtain a trained visual basic model;

[0077] The category learning module is used to adjust the balance relationship of samples during training by specifying the misclassification weights of cost-sensitive vectors on the basis of the basic visual model to obtain a large electric power visual model;

[0078] The scene lightweight module is used to perform transfer training on the large electric power vision model. Through the knowledge distillation technology, the teacher model in the deep model library is used to perform one-to-one knowledge transfer to the student model with a given structure, so that deep models with the same task but different structures obtain the same structure and obtain the business scene model.

[0079] Furthermore, in the scene lightweight module, the teacher model in the deep model library is subjected to one-to-one knowledge transfer to the given structure student model through the knowledge distillation technology, so that the deep models with the same task but different structures obtain the same structure, including:

[0080] For single-task reorganization of deep models, the intermediate-layer features generated by the teacher model for the input image are used as intermediate-layer supervision information, and the predicted soft labels of the teacher model are used as task supervision information to construct a model deep reorganization algorithm with layer-by-layer fusion and compression.

[0081] For multi-task reorganization of deep models, the multi-task teacher model in the deep model library is used to construct a branching upper and lower bound estimation algorithm for given multi-tasks. Combined with the teacher model's predicted soft labels and layer-by-layer fusion and compression of the model deep reorganization algorithm, a deep model multi-task reorganization algorithm based on online branching is constructed to achieve the optimal multi-task student model construction.

[0082] Embodiment 3

[0083] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the method for constructing a pre-trained visual large model for power transmission business scenarios as described in Embodiment 1 are implemented.

[0084] Embodiment 4

[0085] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for constructing a pre-trained visual large model for power transmission business scenarios as described in Embodiment 1 are implemented.

[0086] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0090] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a pre-trained visual large model for power transmission business scenarios, characterized in that: The steps include: The visual basic model is trained based on the power sample data to obtain a trained visual basic model; On the basis of the basic visual model, the misclassification weight of the cost-sensitive vector is specified to adjust the balance relationship of the samples during training, thus obtaining the large electric power visual model. The large electric power vision model is trained for transfer, and the knowledge distillation technology is used to perform one-to-one knowledge transfer from the teacher model in the deep model library to the student model with a given structure, so that deep models with the same task but different structures can obtain the same structure and obtain a business scenario model.

2. The method for constructing a pre-trained visual large model for power transmission business scenarios according to claim 1, characterized in that: When the visual basic model is trained based on the power sample data, the DnCNN network model is used to suppress the noise in the transmission line defect image, and the noisy foreign body image is subtracted from the extracted residual image. When training DnCNN, the mean square error between the residual image and the estimated residual image is controlled to optimize the hyperparameters in the model.

3. The method for constructing a pre-trained visual large model for power transmission business scenarios according to claim 1, characterized in that: The calculation formula of the mean square error is: In the formula, l(θ) is the mean square error, y i and x i They represent the i-th pair of noisy and denoised foreign body images respectively, and θ is a hyperparameter.

4. The method for constructing a pre-trained visual large model for power transmission business scenarios according to claim 1, characterized in that: The method adjusts the balance relationship of samples during training by specifying the misclassification weight of the cost-sensitive vector, obtains the loss of accuracy of the sample category by adopting the FocalLoss function and adding weights to the cross entropy, and adjusts the training weight of the category according to the loss of accuracy of the sample category.

5. The method for constructing a pre-trained visual large model for power transmission business scenarios according to claim 4, characterized in that: The loss function of the accuracy of the sample category is: FL(p t )=-a t (1-p t ) γ log(p t ) In the formula, p t Represents the accuracy of different categories, α t It represents the weights set for different categories. The parameter γ is the attenuation coefficient. When the class is 1, it is multiplied by the weight α. When the class is 0, it is multiplied by the weight 1-α.

6. The method for constructing a pre-trained visual large model for power transmission business scenarios according to claim 1, characterized in that: The method of performing one-to-one knowledge transfer from the teacher model in the deep model library to the given structure student model through the knowledge distillation technology so that the deep models with the same tasks but different structures obtain the same structure includes: For single-task reorganization of deep models, the intermediate-layer features generated by the teacher model for the input image are used as intermediate-layer supervision information, and the predicted soft labels of the teacher model are used as task supervision information to construct a model deep reorganization algorithm with layer-by-layer fusion and compression. For multi-task reorganization of deep models, the multi-task teacher model in the deep model library is used to construct a branching upper and lower bound estimation algorithm for given multi-tasks. Combined with the teacher model's predicted soft labels and layer-by-layer fusion and compression of the model deep reorganization algorithm, a deep model multi-task reorganization algorithm based on online branching is constructed to achieve the optimal multi-task student model construction.

7. A system for constructing a pre-trained visual large model for power transmission business scenarios, characterized in that: include: A sample preprocessing module, which is used to train the visual basic model based on the power sample data to obtain a trained visual basic model; The category learning module is used to adjust the balance relationship of samples during training by specifying the misclassification weights of cost-sensitive vectors on the basis of the basic visual model to obtain a large electric power visual model; The scene lightweight module is used to perform transfer training on the large electric power vision model. Through the knowledge distillation technology, the teacher model in the deep model library is used to perform one-to-one knowledge transfer to the student model with a given structure, so that deep models with the same task but different structures obtain the same structure and obtain the business scene model.

8. The system for constructing a pre-trained visual large model for power transmission business scenarios according to claim 7, characterized in that: In the scene lightweight module, the knowledge distillation technology is used to perform one-to-one knowledge transfer from the teacher model in the deep model library to the given structure student model, so that the deep models with the same task but different structures obtain the same structure, including: For single-task reorganization of deep models, the intermediate-layer features generated by the teacher model for the input image are used as intermediate-layer supervision information, and the predicted soft labels of the teacher model are used as task supervision information to construct a model deep reorganization algorithm with layer-by-layer fusion and compression. For multi-task reorganization of deep models, the multi-task teacher model in the deep model library is used to construct a branching upper and lower bound estimation algorithm for given multi-tasks. Combined with the teacher model's predicted soft labels and layer-by-layer fusion and compression of the model deep reorganization algorithm, a deep model multi-task reorganization algorithm based on online branching is constructed to achieve the optimal multi-task student model construction.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for constructing a pre-trained visual large model for power transmission business scenarios as described in any one of claims 1-6 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for constructing a pre-trained visual large model for power transmission business scenarios as described in any one of claims 1-6 are implemented.

Images