Collaborative training method, device, equipment and storage medium for large electric power vision model

Abstract

The present invention discloses a collaborative training method, device, equipment and storage medium for a large electric power vision model. The collaborative training method for the large electric power vision model includes: iteratively training a pre-set large participant vision model based on a participant sample set; during the training process of the large participant vision model, setting a corresponding sparse target threshold for at least one model structure layer in the large participant vision model, and performing sparse processing on the model structure layer based on the sparse target threshold; after each iterative training process of the large participant vision model, determining the optimal model parameter of each model parameter type from at least one model parameter of at least one large participant vision model; and updating the server vision model deployed on the central server based on the optimal model parameter. Through the above technical solution, the training efficiency of the model is improved, and the model performance of the trained model is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a collaborative training method, device, equipment and storage medium for a large electric power vision model. Background Art

[0002] With the development of smart grids and the digital transformation of power systems, the power industry has an increasing demand for data-driven intelligent decision-making. Visual big models are being used more and more widely in the power industry to optimize equipment maintenance and fault diagnosis by analyzing image data of power systems.

[0003] However, the training of large power vision models usually requires a large amount of image data. The existing training methods for large power vision models do not consider the problem of training models with larger models, such as those with hundreds of millions or even billions of parameters. There is no efficient solution to the model partitioning method and the low efficiency of model training, making it difficult to efficiently train large vision models. Summary of the invention

[0004] The present invention provides a collaborative training method, device, equipment and storage medium for a large electric power vision model, so as to improve the training efficiency and model performance of the large television vision model.

[0005] According to one aspect of the present invention, a collaborative training method for a large power vision model is provided, the method comprising:

[0006] Based on the participant sample set, the pre-set participant visual large model is iteratively trained until the preset training end condition is reached; wherein the participant sample set is a data set of power equipment fault images, and the participant sample sets corresponding to each participant are different from each other;

[0007] During the training process of the participant's visual big model, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual big model, and based on the sparseness target threshold, the model structure layer is sparsely processed;

[0008] After each iterative training process of the participant vision big model is completed, determining the optimal model parameter of each model parameter type from at least one model parameter of at least one participant vision big model;

[0009] The server visual big model deployed on the central server is updated based on the optimal model parameters, and the participant visual big model is synchronously updated based on the update result of the server visual big model.

[0010] According to another aspect of the present invention, a collaborative training device for a large electric power vision model is provided, the device comprising:

[0011] A training module, used for iteratively training a preset participant visual large model based on a participant sample set until a preset training end condition is reached; wherein the participant sample set is a data set of fault images of power equipment, and the participant sample sets corresponding to each participant are different from each other;

[0012] A sparseness module, used for setting a corresponding sparseness target threshold for at least one model structure layer in the participant's visual big model during the training process of the participant's visual big model, and performing sparseness processing on the model structure layer based on the sparseness target threshold;

[0013] An optimal parameter determination module, used for determining the optimal model parameter of each model parameter type from at least one model parameter of at least one participant's visual large model after each iterative training process of the participant's visual large model is completed;

[0014] The updating module is used to update the server visual big model deployed on the central server based on the optimal model parameters, and synchronously update the participant visual big model based on the update result of the server visual big model.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the collaborative training method of the power vision large model described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the collaborative training method of the power vision large model described in any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program, and when the computer program is executed by a processor, the collaborative training method for the large electric power vision model described in any embodiment of the present invention is implemented.

[0021] The technical solution of the embodiment of the present invention proposes a data privacy protection and collaborative training method for a large electric power visual model. The central server and multiple participants use encryption and aggregation protocols to collaboratively train the large visual model, and determine the optimal parameter model for each model parameter type in the participant's large visual model to update the server's large visual model. The model is trained by multiple participants collaborating with the central server to construct a network parameter sparse matrix, which reduces the operating load of the central server and alleviates the pressure on the limited computer resources of the central server. At the same time, the training efficiency of the model is improved, and an evaluation strategy is added. The model performance of the trained model can be further improved by determining the optimal parameter model.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a collaborative training method for a large electric power vision model provided according to the first embodiment of the present invention;

[0025] Figure 2 is a flow chart of a collaborative training method for a large electric power vision model provided according to the second embodiment of the present invention;

[0026] Figure 3 is a flow chart of a collaborative training method for a large electric power vision model provided according to Embodiment 3 of the present invention;

[0027] Figure 4 It is a structural schematic diagram of a collaborative training device for a large electric power vision model provided according to a fourth embodiment of the present invention;

[0028] Figure 5 It is a structural schematic diagram of an electronic device for implementing the collaborative training method of the large electric power vision model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 A flowchart of a collaborative training method for a large electric power vision model is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a large electric power equipment model is trained in a distributed manner among participants. The method can be executed by a collaborative training device for a large electric power vision model. The collaborative training device for a large electric power vision model can be implemented in the form of hardware and / or software. The collaborative training device for a large electric power vision model can be configured in various general computing devices, such as a client device or server device integrated with a large visual model. Figure 1 As shown, the method includes:

[0033] S110. Based on the participant sample set, iteratively train the pre-set participant vision large model until the preset training end condition is reached.

[0034] The participant sample set is a data set of power equipment fault images, and the participant sample sets corresponding to each participant are different.

[0035] In an embodiment of the present invention, there are multiple participants and a central server that collaboratively participate in the training of the electric power vision big model. The participant vision big model deployed on the parameter side can be uniformly provided by the central server or set by the participant. However, the participant vision big model deployed by the participant has the same model structure to ensure that all participants are trained from the same starting point. It should be noted that before the training begins, the participants can pre-negotiate with the central server the encryption protocol and data transmission protocol used by both parties to ensure the consistency of the encryption strategy.

[0036] The central server can establish communication connections with multiple participants. The participants and the central server can generate their own corresponding encryption key pairs, and the participants and the central server can share their public keys. When the data sender sends data to the data receiver, it can use the data receiver's public key to encrypt and send it. After receiving the data, the data receiver can decrypt the received data according to its own stored private key. Optionally, the encryption keys of the participants and the server can be updated periodically to prevent key leakage or failure.

[0037] Optionally, the training end condition can be adaptively set according to those skilled in the art, for example, the training end condition can be set according to the number of training iterations.

[0038] Optionally, in an embodiment of the present invention, before training the participant's visual large model based on the participant's sample set, the participant may normalize the participant's sample set in advance to obtain sample data in a standard data format, and encrypt the normalized sample data to ensure data security while improving data quality.

[0039] S120. During the training process of the participant's visual big model, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual big model, and based on the sparseness target threshold, the model structure layer is sparsely processed.

[0040] Among them, the sparsification target threshold can be used to characterize the sparsity of the model parameters in the model structure layer. Optionally, the model parameters can be structural parameters of the model structure layer in the visual big model. It should be noted that in different iterative training processes of the participant's visual big model deployed by the participant, the sparsification target threshold can be dynamically adjusted according to the previous iterative training results of the participant's visual big model, so that the iterative training process can adapt to the dynamic changes of the data and the gradual optimization of the model to improve the model performance.

[0041] In the embodiment of the present invention, the visual big model has different model structure layers for realizing different model functions, such as feature extraction layer and decision layer, etc. According to the importance of different model structure layers, corresponding sparse target thresholds can be set for the model structure layers, and the model structure layers can be sparsely processed to meet the sparse target thresholds corresponding to the model structure layers.

[0042] Optionally, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual big model, and based on the sparseness target threshold, the model structure layer is sparsely processed, including: setting a corresponding sparseness target threshold for at least one model structure layer in the participant's visual big model according to the model function of the model structure layer; using a preset sparseness processing method to sparse the model structure layer so that the number of model parameters in the model structure layer reaches the sparseness target threshold corresponding to the model structure layer.

[0043] Specifically, the model structure layer in the participant's visual large model can be set with corresponding sparse target thresholds according to the importance of the model structure layer determined by the model function of the model structure layer, and the model structure layer is sparsely processed by a pre-set sparse processing method so that the number of model parameters in the model structure layer reaches the sparse target threshold corresponding to the model structure layer. For example, the feature extraction layer is subjected to shallow sparse processing so that the feature extraction layer can retain more model parameters to ensure that high-quality feature data can be extracted, and a high degree of sparse processing is performed at the decision layer to reduce the complexity of the model. Optionally, the importance of the model structure layer can be determined according to the degree of influence of the model structure layer on the model output. The sparse processing method can be adaptively set according to those skilled in the art. For example, the sparse processing method can include Lasso regression (Least Absolute Shrinkage and Selection Operator) method, L0 regularization method, and importance-based pruning method.

[0044] By dynamically sparsely processing the large visual model of the participants, the degree of sparsity of different model structure layers can be finely controlled, thereby improving the model performance.

[0045] S130. After each iterative training process of the participant's visual large model is completed, the optimal model parameter of each model parameter type is determined from at least one model parameter of at least one participant's visual large model.

[0046] Among them, the optimal model parameters may refer to the model parameters that best match the server vision large model.

[0047] Specifically, after the current iterative training process of the participant vision big model deployed by each participant is completed, the optimal parameter model of each model parameter type can be determined from multiple model parameters in the participant vision big model deployed by each participant.

[0048] S140. Update the server vision big model deployed on the central server based on the optimal model parameters, and synchronously update the participant vision big model based on the update result of the server vision big model.

[0049] In an embodiment of the present invention, when the preset training end conditions are met, the server visual big model is the final trained power fault big model. Before the power fault big model is transmitted and deployed, the server can encrypt it again to ensure that it will not be leaked during the transmission of the power fault big model. The power fault big model can be used for practical applications in power production environments, such as equipment fault detection, predictive maintenance and other scenarios.

[0050] The technical solution of the embodiment of the present invention is to collaboratively train a visual big model with a central server and multiple participants, and determine the optimal parameter model for each model parameter type in the participant's visual big model to update the server's visual big model. By having multiple participants collaborate with the central server to perform model training, the operating load of the central server is reduced, the pressure on the limited computer resources of the central server is alleviated, and the training efficiency of the model is improved. In addition, the determination of the optimal parameter model can further improve the model performance of the trained model.

[0051] Embodiment 2

[0052] Figure 2 The flowchart of a collaborative training method of a power vision big model provided in the second embodiment of the present invention is further refined on the basis of the above embodiment, and provides specific steps for determining the optimal model parameters of each model parameter type from at least one model parameter of at least one participant's vision big model after each iterative training process of the participant's vision big model is completed. It should be noted that, for the undetailed parts of the embodiment of the present invention, please refer to the relevant descriptions of other embodiments, which will not be repeated here. Figure 2 As shown, the method includes:

[0053] S210: Based on the participant sample set, iteratively train the pre-set participant vision large model until the preset training end condition is reached.

[0054] S220. During the training process of the participant's visual big model, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual big model, and based on the sparseness target threshold, the model structure layer is sparsely processed.

[0055] S230. After each iterative training process of the participant's visual big model is completed, the model contribution of each model parameter in the participant's visual big model is determined according to the loss changes of the model parameters of the participant's visual big model and the server's visual big model.

[0056] Among them, the model contribution can be used to characterize the contribution of model parameters in the model training process.

[0057] In an embodiment of the present invention, the model parameters in each participant's visual big model can be used to determine the model contribution of each model parameter to the server visual big model based on the loss changes caused to the participant's visual big model and the server visual big model after each iterative training process.

[0058] Optionally, the model contribution can be determined by the following formula:

[0059]

[0060] Among them, C ij is the model contribution of the jth model parameter in the i-th participant’s visual model, w j is the weight of the jth model parameter, ΔL i is the loss change of the visual model of the ith participant, ΔL global Loss changes for server vision large models.

[0061] S240. For the model parameter with the highest model contribution in each model parameter type, calculate the model acceptance degree of the model parameter by the server visual large model, and determine the model parameter as the optimal model parameter when the model acceptance degree meets the preset acceptance conditions.

[0062] Among them, the model acceptance degree can be used to characterize the matching degree between the model parameters and the server vision large model.

[0063] In an embodiment of the present invention, after determining the model contribution of each model parameter in the visual big model of each participant, the model parameters of the same type in the visual big model of each participant are classified into the same model parameter type, and for each model parameter type, the model parameter with the highest model contribution in the category is determined as the candidate model parameter, and the model acceptance degree of the candidate model parameter accepted by the server visual big model is calculated, and when the model acceptance degree of the candidate model parameter meets the preset acceptance condition, the candidate model parameter is determined as the optimal model parameter. Optionally, the model acceptance degree of the model parameter of the server visual big model can be determined by the Metropolis-Hastings (Markov Chain Monte Carlo, MCMC) algorithm. The preset acceptance condition can be set according to those skilled in the art and Xining adaptability.

[0064] Optionally, in an embodiment of the present invention, model parameters in each model parameter type whose model contribution is greater than a preset contribution threshold value may be determined as candidate model parameters. Furthermore, based on the order of model contribution, the model acceptance degree of the candidate model parameter with the highest model contribution may be determined first. When the model acceptance degree meets the preset acceptance condition, the candidate model parameter is determined as the optimal model parameter. When the model acceptance degree does not meet the preset acceptance condition, the model acceptance degree of the next candidate model parameter is determined. If the model acceptance degrees of all candidate model parameters do not meet the preset acceptance condition, the candidate model parameter with the highest model contribution is determined as the optimal model parameter.

[0065] The model performance of the server vision large model is improved by determining the optimal model parameters.

[0066] Optionally, the model contribution of each model parameter in the participant's visual big model is determined based on the change in the loss of model parameters of the participant's visual big model and the server's visual big model, including: for at least one model parameter in the participant's visual big model deployed by itself, the model contribution corresponding to each model parameter is determined in parallel based on the Markov chain, and when the number of iterations of the participant's visual big model reaches a preset optimization number, the model parameters are exchanged between the Markov chains.

[0067] Specifically, each participant can maintain multiple Markov chains, the number of which is consistent with the number of model parameters in the participant's visual large model. Each Markov chain can independently sample different model parameters to determine the model contribution of multiple model parameters in parallel. In order to prevent the Markov chain from falling into a local optimal solution, when the number of iterative training of the participant's visual large model reaches a preset number of optimizations, the parameter acceptance probability between different Markov chains can be determined, and the model parameters can be exchanged when the parameter acceptance probability is 1. It should be noted that the preset number of optimizations can be adaptively set according to those skilled in the art.

[0068] Optionally, the parameter acceptance probability can be determined by the following formula:

[0069]

[0070] Among them, ∝(θ i ,θ j ) is the model parameter θ i With model parameters θ j The parameter acceptance probability between the corresponding Markov chains, C(θ i ) is the model parameter θ i The model contribution, C(θ j ) is the model parameter θ j The contribution of the model.

[0071] Parallel sampling of Markov chains can not only improve the exploration capability of the model parameter space, but also enhance the stability and convergence of the model during the training process through inter-chain parameter exchange.

[0072] S250: updating the server visual big model deployed on the central server based on the optimal model parameters, and synchronously updating the participant visual big model based on the update result of the server visual big model.

[0073] The technical solution of the embodiment of the present invention determines the optimal model parameters for the best training effect of the server vision big model in each model parameter type after each iterative training process of the participating parties, thereby laying a foundation for the subsequent update of the server vision big model and improving the training efficiency of distributed collaborative training and the model performance of the trained model.

[0074] Embodiment 3

[0075] Figure 3This is a flowchart of a collaborative training method for a large electric power vision model provided in the third embodiment of the present invention. This embodiment further refines the above embodiment and provides specific steps for updating the large server vision model deployed on the server based on the optimal model parameters. It should be noted that for the undetailed parts of the embodiment of the present invention, please refer to the relevant descriptions of other embodiments, which will not be repeated here. Figure 3 As shown, the method includes:

[0076] S310: Based on the participant sample set, iteratively train the pre-set participant vision large model until the preset training end condition is reached.

[0077] S320. During the training process of the participant's visual big model, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual big model, and based on the sparseness target threshold, the model structure layer is sparsely processed.

[0078] S330. After each iterative training process of the participant's visual large model is completed, the optimal model parameter of each model parameter type is determined from at least one model parameter of at least one participant's visual large model.

[0079] S340: Encrypt the optimal model parameters according to the public key information of the central server.

[0080] S350, split and encrypt the encrypted optimal model parameters, and send them to the central server; wherein the central server aggregates the received encrypted split parameter data, reorganizes the optimal model parameters, and updates the server vision model deployed by itself according to the optimal model parameters.

[0081] In an embodiment of the present invention, after each participant determines the optimal model parameters, the participant can encrypt the optimal model parameters according to the public key of the central server, and divide the encrypted optimal model parameters to generate a preset number of divided parameter data (model sub-parameters). The participant will also encrypt the model sub-parameters twice according to the public key of the central server and send them to the central server. After receiving the preset number of model sub-parameters, the central server can reorganize the preset number of model sub-parameters to obtain the optimal model parameters after aggregation, and update the server visual large model deployed by itself according to the optimal model parameters. It should be noted that the central server can use homomorphic encryption technology, so that the central server can directly aggregate the model sub-parameters in an encrypted state without decrypting the model sub-parameters sent by each participant, thereby further ensuring the privacy of the data.

[0082] After the participants split the optimal model sub-parameters and transmit them to the central server, the central server can restore the optimal model parameters only by collecting all the model sub-parameters. During the transmission process, even if a certain transmission node is attacked, the attacker cannot obtain the complete information of the optimal model sub-parameters through a single model sub-parameter, thus ensuring the security of data during the transmission process.

[0083] Optionally, in an embodiment of the present invention, when the participants and the central server transmit data, the Transport Layer Security Protocol (TLS) can be used to encrypt the data transmission channel to prevent man-in-the-middle attacks and data packet interception.

[0084] Optionally, in an embodiment of the present invention, based on the update result of the server visual big model, the participant visual big model is synchronously updated, including: receiving the model parameters of the updated server visual big model sent by the central server, and updating the participant visual big model deployed by itself according to the model parameters.

[0085] Specifically, after the central server completes the update of the server visual large model according to the aggregated optimal model parameters, it can encrypt the optimal model parameters according to the public keys of each participant and transmit them to the corresponding participant. After receiving the aggregated optimal model parameters, each participant uses the local private key to decrypt them and update the local participant's visual large model. It should be noted that the aggregated optimal model parameters received by the participant are only used for local updates and training to ensure that the decrypted data will not be accessed by external systems.

[0086] Optionally, after the participant's local visual big model is updated, the participant can perform a performance evaluation on the updated participant's visual big model, including performance indicators such as model accuracy and recall rate, to ensure that the quality of the model meets expectations, and dynamically adjust the sparsification target threshold corresponding to the model structure layer according to the performance evaluation results, so that the training process can adapt to the dynamic changes of data and the gradual optimization of the model.

[0087] The technical solution in the embodiment of the present invention improves the training effect of the model during training by updating the server visual model and the participant visual model according to the optimal parameter model, thereby improving the model performance of the model after the final training is completed.

[0088] Embodiment 4

[0089] Figure 4 This is a schematic diagram of the structure of a collaborative training device for a large electric vision model provided by the fourth embodiment of the present invention. Figure 4 As shown, the device comprises:

[0090] The training module 410 is used to iteratively train the preset participant visual large model based on the participant sample set until the preset training end condition is reached; wherein the participant sample set is a data set of power equipment fault images, and the participant sample sets corresponding to each participant are different from each other;

[0091] A sparse module 420, used for setting a corresponding sparse target threshold for at least one model structure layer in the participant's visual big model during the training process of the participant's visual big model, and performing sparse processing on the model structure layer based on the sparse target threshold;

[0092] An optimal parameter determination module 430, configured to determine the optimal model parameter of each model parameter type from at least one model parameter of at least one participant's visual large model after each iterative training process of the participant's visual large model is completed;

[0093] The updating module 440 is used to update the server visual big model deployed on the central server based on the optimal model parameters, and synchronously update the participant visual big model based on the update result of the server visual big model.

[0094] The technical solution of the embodiment of the present invention is to collaboratively train a visual big model with a central server and multiple participants, and determine the optimal parameter model for each model parameter type in the participant's visual big model to update the server's visual big model. By having multiple participants collaborate with the central server to perform model training, the operating load of the central server is reduced, the pressure on the limited computer resources of the central server is alleviated, and the training efficiency of the model is improved. In addition, the model performance of the trained model can be further improved by determining the optimal parameter model.

[0095] Optionally, the optimal parameter determination module 430 includes:

[0096] A contribution determination unit is used to determine the model contribution of each model parameter in the participant's visual big model according to the loss change of the model parameters of the participant's visual big model and the server's visual big model after each iterative training process of the participant's visual big model is completed; wherein the model contribution is used to characterize the contribution degree of the model parameter in the model training process;

[0097] The optimal parameter determination unit is used to calculate the model acceptance degree of the model parameter with the highest model contribution in each model parameter type by the server visual large model, and determine the model parameter as the optimal model parameter when the model acceptance degree meets the preset acceptance conditions.

[0098] Optionally, the contribution determination unit can also be used to: determine the model contribution corresponding to each model parameter in parallel based on the Markov chain for at least one model parameter in the participant's visual big model deployed by itself, and exchange model parameters between Markov chains when the number of iterations of the participant's visual big model reaches a preset optimization number.

[0099] Optionally, the sparsification module 420 can be specifically used to: set a corresponding sparsification target threshold for at least one model structure layer in the participant's visual large model according to the model function of the model structure layer; use a preset sparsification processing method to perform sparsification processing on the model structure layer so that the number of model parameters in the model structure layer reaches the sparsification target threshold corresponding to the model structure layer; wherein the model parameters are structural parameters of the model structure layer in the visual large model.

[0100] Optionally, the update module 440 includes:

[0101] An encryption unit, used for encrypting the optimal model parameters according to the public key information of the central server;

[0102] The segmentation unit is used to segment and encrypt the encrypted optimal model parameters and send them to the central server; the central server aggregates the received encrypted segmentation parameter data, reorganizes the optimal model parameters, and updates the server vision model deployed by itself according to the optimal model parameters.

[0103] Optionally, the updating module 440 further includes:

[0104] The synchronous updating unit is used to receive the model parameters of the updated server visual big model sent by the central server, and update the participant visual big model deployed by itself according to the model parameters.

[0105] The collaborative training device for the large electric power vision model provided in the embodiment of the present invention can execute the collaborative training method for the large electric power vision model provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0106] Embodiment 5

[0107] Figure 5A schematic diagram of an electronic device 510 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0108] like Figure 5 As shown, the electronic device 510 includes at least one processor 511, and a memory connected to the at least one processor 511 in communication, such as a read-only memory (ROM) 512, a random access memory (RAM) 513, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 511 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 512 or the computer program loaded from the storage unit 518 to the random access memory (RAM) 513. In the RAM 513, various programs and data required for the operation of the electronic device 510 can also be stored. The processor 511, the ROM 512, and the RAM 513 are connected to each other via a bus 514. An input / output (I / O) interface 515 is also connected to the bus 514.

[0109] A number of components in the electronic device 510 are connected to the I / O interface 515, including: an input unit 516, such as a keyboard, a mouse, etc.; an output unit 517, such as various types of displays, speakers, etc.; a storage unit 518, such as a disk, an optical disk, etc.; and a communication unit 519, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 519 allows the electronic device 510 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The processor 511 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 511 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 511 performs the various methods and processes described above, such as the collaborative training method of the power vision large model.

[0111] In some embodiments, the collaborative training method of the power vision large model can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 518. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 510 via the ROM 12 and / or the communication unit 519. When the computer program is loaded into the RAM 513 and executed by the processor 511, one or more steps of the collaborative training method of the power vision large model described above can be performed. Alternatively, in other embodiments, the processor 511 can be configured to execute the collaborative training method of the power vision large model in any other appropriate manner (for example, by means of firmware).

[0112] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0114] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0116] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0117] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0118] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0119] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A collaborative training method for a large power vision model, characterized in that: Applied to a participant, the method comprises: Based on the participant sample set, the pre-set participant visual large model is iteratively trained until the preset training end condition is reached; wherein the participant sample set is a data set of power equipment fault images, and the participant sample sets corresponding to each participant are different from each other; During the training process of the participant's visual big model, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual big model, and based on the sparseness target threshold, the model structure layer is sparsely processed; After each iterative training process of the participant vision big model is completed, determining the optimal model parameter of each model parameter type from at least one model parameter of at least one participant vision big model; The server visual big model deployed on the central server is updated based on the optimal model parameters, and the participant visual big model is synchronously updated based on the update result of the server visual big model.

2. The method according to claim 1, characterized in that After each iterative training process of the participant vision big model is completed, the optimal model parameter of each model parameter type is determined from at least one model parameter of at least one participant vision big model, including: After each iterative training process of the participant's visual big model is completed, the model contribution of each model parameter in the participant's visual big model is determined according to the loss change of the model parameters of the participant's visual big model and the server's visual big model; wherein the model contribution is used to characterize the contribution degree of the model parameter in the model training process; For the model parameter with the highest model contribution in each model parameter type, the model acceptance degree of the model parameter by the server visual large model is calculated, and when the model acceptance degree meets the preset acceptance conditions, the model parameter is determined as the optimal model parameter.

3. The method according to claim 2, characterized in that According to the loss changes of the model parameters of the participant's visual big model and the server's visual big model, the model contribution of each model parameter in the participant's visual big model is determined, including: For at least one model parameter in the participant's visual big model deployed by itself, the model contribution corresponding to each model parameter is determined in parallel based on the Markov chain, and when the number of iterations of the participant's visual big model reaches the preset optimization number, the model parameters between the Markov chains are exchanged.

4. The method according to claim 1, characterized in that A corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual large model, and based on the sparseness target threshold, a sparseness process is performed on the model structure layer, including: According to the model function of the model structure layer, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual large model; The model structure layer is subjected to sparse processing by using a preset sparse processing method, so that the number of model parameters in the model structure layer reaches the sparse target threshold corresponding to the model structure layer; wherein the model parameters are the structural parameters of the model structure layer in the visual large model.

5. The method according to claim 1, characterized in that The server vision model deployed on the server is updated based on the optimal model parameters, including: Encrypting the optimal model parameters according to the public key information of the central server; The encrypted optimal model parameters are segmented and encrypted, and then sent to the central server; the central server aggregates the received encrypted segmented parameter data, reorganizes the optimal model parameters, and updates the server vision model deployed by itself according to the optimal model parameters.

6. The method according to claim 1, characterized in that Based on the update result of the server visual big model, the participant visual big model is synchronously updated, including: Receive the model parameters of the updated server visual big model sent by the central server, and update the participant visual big model deployed by itself according to the model parameters.

7. A collaborative training device for a large electric power vision model, characterized in that: Applied to a participant, the device comprises: A training module, used for iteratively training a preset participant visual large model based on a participant sample set until a preset training end condition is reached; wherein the participant sample set is a data set of fault images of power equipment, and the participant sample sets corresponding to each participant are different from each other; A sparseness module, used for setting a corresponding sparseness target threshold for at least one model structure layer in the participant's visual big model during the training process of the participant's visual big model, and performing sparseness processing on the model structure layer based on the sparseness target threshold; An optimal parameter determination module, used for determining the optimal model parameter of each model parameter type from at least one model parameter of at least one participant's visual large model after each iterative training process of the participant's visual large model is completed; The updating module is used to update the server visual big model deployed on the central server based on the optimal model parameters, and synchronously update the participant visual big model based on the update result of the server visual big model.

8. The device according to claim 7, characterized in that The optimal parameter determination module includes: A contribution determination unit is used to determine the model contribution of each model parameter in the participant's visual big model according to the loss change of the model parameters of the participant's visual big model and the server's visual big model after each iterative training process of the participant's visual big model is completed; wherein the model contribution is used to characterize the contribution degree of the model parameter in the model training process; The optimal parameter determination unit is used to calculate the model acceptance degree of the model parameter with the highest model contribution in each model parameter type by the server visual large model, and determine the model parameter as the optimal model parameter when the model acceptance degree meets the preset acceptance conditions.

9. The device according to claim 8, characterized in that The contribution determination unit is further configured to: For at least one model parameter in the participant's visual big model deployed by itself, the model contribution corresponding to each model parameter is determined in parallel based on the Markov chain, and when the number of iterations of the participant's visual big model reaches the preset optimization number, the model parameters between the Markov chains are exchanged.

10. The device according to claim 7, characterized in that The sparse module is specifically used for: According to the model function of the model structure layer, a corresponding sparseness target threshold is set for at least one model structure layer in the participant's visual large model; The model structure layer is subjected to sparse processing by using a preset sparse processing method, so that the number of model parameters in the model structure layer reaches the sparse target threshold corresponding to the model structure layer; wherein the model parameters are the structural parameters of the model structure layer in the visual large model.

11. The device according to claim 7, characterized in that The update module includes: An encryption unit, used for encrypting the optimal model parameters according to the public key information of the central server; The segmentation unit is used to segment and encrypt the encrypted optimal model parameters and send them to the central server; the central server aggregates the received encrypted segmentation parameter data, reorganizes the optimal model parameters, and updates the server vision model deployed by itself according to the optimal model parameters.

12. The device according to claim 7, characterized in that The update module also includes: The synchronous updating unit is used to receive the model parameters of the updated server visual big model sent by the central server, and update the participant visual big model deployed by itself according to the model parameters.

13. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the collaborative training method for the power vision large model according to any one of claims 1-6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the collaborative training method of the power vision large model according to any one of claims 1 to 6 when executed.

15. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the collaborative training method of the power vision large model described in any one of claims 1-6.

Images