A model deployment method and apparatus

Abstract

The application discloses a model deployment method and device, and the method comprises the following steps: acquiring a pre-trained visual convolution model, setting an accuracy target and search parameters for pruning the visual convolution model, wherein the search parameters at least comprise an initial pruning ratio and a search interval length; searching a first target pruning ratio of an attention mechanism module and a second target pruning ratio of a feedforward neural module respectively; optimizing the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain a target visual convolution model; and deploying the target visual convolution model on an edge computing device of a substation. In this way, after the visual convolution model is pruned, the model size is in a minimum state, so that the requirement for the running environment is reduced. The visual convolution model is deployed on the edge computing device, real-time reception of monitoring data is realized, so that real-time detection and warning can be achieved, and the safety of the power system is improved.

Description

Technical Field

[0001] This invention relates to the field of semiconductor device packaging technology, and in particular to a model deployment method and apparatus. Background Technology

[0002] In power system substations and other locations, it is often necessary to analyze the monitoring data from the installed cameras to identify any violations or other behaviors, thereby achieving automated monitoring.

[0003] While representative methods for image analysis and recognition, such as the Vision Transformer (ViT), have achieved leading results, their large model size and demanding operating environment requirements necessitate the use of such models. Therefore, current monitoring methods require transmitting images captured by cameras to the cloud for analysis and recognition using cloud-deployed Vision Transformer models. However, the cloud-based deployment of these models introduces significant time delays in data transmission, hindering real-time monitoring and alerts, and ultimately impacting power system safety. Summary of the Invention

[0004] This invention aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of this invention proposes a model deployment method, comprising:

[0005] A pre-trained visual convolutional model is obtained, which is used to receive and process monitoring images of the substation area; the visual convolutional model consists of an attention mechanism module and a feedforward neural module.

[0006] Set the accuracy target and search parameters for pruning the visual convolutional model, wherein the search parameters include at least the initial pruning ratio and the search interval length;

[0007] Using the accuracy target and the search parameters, the first target pruning ratio of the attention mechanism module and the second target pruning ratio of the feedforward neural module are searched respectively.

[0008] The visual convolution model is optimized based on the first target pruning ratio and the second target pruning ratio to obtain the target visual convolution model.

[0009] The target visual convolutional model is deployed on the edge computing device of the substation to receive the current monitoring image and process the monitoring image in real time.

[0010] Optionally, before searching for the first target pruning ratio of the attention mechanism module using the accuracy target and the search parameters, the method further includes:

[0011] Load the visual convolutional model and load the pre-trained weights;

[0012] Set the initial running parameters for the pruning operation. The initial running parameters include at least: the dataset reading path, the number of images retrieved from the dataset each time, the optimizer function, the loss function, and the number of fine-tuning rounds.

[0013] Optionally, the setting of the accuracy target and search parameters for pruning the visual convolutional model includes:

[0014] Set a pruning accuracy target; the pruning accuracy target represents the acceptable pruning accuracy of the model for the user.

[0015] The fault-tolerant range is defined; the fault-tolerant range represents the fault-tolerant range of the algorithm.

[0016] Set the initial pruning ratios for the attention mechanism module and the feedforward neural module respectively;

[0017] Set the search interval length; the search interval length represents the range of each search.

[0018] Define a variable that is closest to the target range. If the target is not found after the algorithm has completed its iterations, return the variable that is closest to the target range.

[0019] Define a granularity threshold in the binary search, whereby the granularity threshold represents the minimum interval length that can be halved.

[0020] Optionally, the search interval length is a first ratio, and the step of searching for the first target pruning ratio of the attention mechanism module using the precision target and the search parameters includes:

[0021] Starting from the initial pruning ratio, the interval length of the first ratio is increased each time as the current pruning ratio of the attention mechanism module, and the first accuracy of the pruned model is obtained.

[0022] If the first precision is within the current candidate interval, then the first target pruning ratio of the attention mechanism module is searched within the current candidate interval using a binary search method.

[0023] If the first precision is not within the current candidate interval, then proceed to the next candidate interval to continue searching until a target interval range whose precision value meets the precision target is found. Then, use the binary search method to search for the first target pruning ratio of the attention mechanism module within the target interval range.

[0024] Optionally, the step of searching for the first target pruning ratio of the attention mechanism module within the target interval using a binary search method includes:

[0025] The median is calculated based on the left and right boundaries of the target interval, and the median is used as the pruning ratio of the attention mechanism module for pruning operations.

[0026] Continue to halve the interval length of the binary search method until the current pruning ratio makes the accuracy of the obtained model meet the accuracy target, or the interval length of the binary search method is less than the preset granularity threshold.

[0027] The current pruning ratio is determined as the first target pruning ratio of the attention mechanism module, or the pruning ratio recorded during the search process that is closest to the accuracy target is determined as the first target pruning ratio of the attention mechanism module.

[0028] Optionally, the search interval length is a first ratio, and the step of retrieving the second target pruning ratio of the feedforward neural module using the precision target and the search parameters includes:

[0029] Starting from the initial pruning ratio, the interval length of the first ratio is increased each time as the current pruning ratio of the feedforward neural module, and the second accuracy of the pruned model is obtained.

[0030] If the second precision is within the current candidate interval, then the second target pruning ratio of the feedforward neural module is searched within the current candidate interval using the binary search method;

[0031] If the second precision is not within the current candidate interval, then proceed to the next candidate interval to continue searching until a target interval range whose precision value meets the precision target is found. Then, use the binary search method to search for the second target pruning ratio of the feedforward neural module within the target interval range.

[0032] A second aspect of the present invention provides a model deployment apparatus, comprising:

[0033] The model acquisition module is used to acquire a pre-trained visual convolutional model, which is used to receive and process monitoring images of the substation area; the visual convolutional model consists of an attention mechanism module and a feedforward neural module.

[0034] The setting module is used to set the accuracy target and search parameters for pruning the visual convolutional model. The search parameters include at least the initial pruning ratio and the search interval length.

[0035] The search module is used to search for the first target pruning ratio of the attention mechanism module and the second target pruning ratio of the feedforward neural module using the accuracy target and the search parameters, respectively.

[0036] The optimization module is used to optimize the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain the target visual convolution model;

[0037] The deployment module is used to deploy the target visual convolutional model on the edge computing device of the substation, so as to receive the current monitoring image using the target visual convolutional model and process the monitoring image in real time.

[0038] Optionally, the device further includes:

[0039] The loading module is used to load the visual convolutional model and load the pre-trained weights;

[0040] The initial parameter setting module is used to set the initial running parameters of the pruning operation. The initial running parameters include at least: the dataset reading path, the number of images taken from the dataset each time, the optimizer function, the loss function, and the number of fine-tuning rounds.

[0041] A third aspect of the present invention provides an electronic device comprising:

[0042] processor;

[0043] Memory used to store the processor's executable instructions;

[0044] The processor is configured to execute the instructions to implement the model deployment method as described in the first aspect.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a server's processor, enables the server to perform the model deployment method as described in the first aspect.

[0046] The embodiments of the present invention have the following beneficial effects:

[0047] In this embodiment of the invention, a pre-trained visual convolutional model is obtained. This model is used to receive and process monitoring images of a substation area. The visual convolutional model consists of an attention mechanism module and a feedforward neural module. A precision target and search parameters are set for pruning the visual convolutional model. The search parameters include at least an initial pruning ratio and a search interval length. Using the precision target and the search parameters, a first target pruning ratio for the attention mechanism module and a second target pruning ratio for the feedforward neural module are searched. The visual convolutional model is optimized based on the first and second target pruning ratios to obtain a target visual convolutional model. The target visual convolutional model is deployed on an edge computing device in the substation to receive and process the current monitoring images in real time. Thus, after pruning the visual convolutional model, the model size, computational load, and number of parameters of the target visual convolutional model are minimized, thereby reducing the requirements for the operating environment. Deploying visual convolutional models on edge computing devices reduces the distance between the sender and receiver of monitoring data, enabling real-time reception of monitoring data. This allows for real-time detection and alerts, improving the safety of the power system. Attached Figure Description

[0048] Figure 1 A flowchart illustrating the steps of the model deployment method provided in this embodiment of the invention;

[0049] Figure 2 A schematic diagram of the first target pruning ratio of the search attention mechanism module provided in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the binary search method provided in an embodiment of the present invention;

[0051] Figure 4 This is a structural block diagram of the model deployment device provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0054] In power system substations and similar locations, it is often necessary to analyze the monitoring data from installed cameras to identify any violations or other unauthorized activities, thereby achieving automated monitoring. While representative methods for image analysis and recognition, such as visual convolutional models, have achieved leading results, their large model size and demanding operating environment requirements present challenges. Therefore, current monitoring methods require transmitting camera-captured images to the cloud for analysis and recognition using cloud-deployed visual convolutional models, resulting in significant time delays and hindering real-time monitoring and alerts, thus compromising security.

[0055] During their research into related technologies, the inventors discovered that by compressing the visual convolutional model and deploying it on resource-constrained edge computing devices, analysis can be performed simultaneously with the generation of monitoring images, thereby enabling real-time online monitoring.

[0056] Visual convolutional models can be compressed using pruning techniques. In the field of model pruning, numerous ViT pruning techniques have achieved good pruning results. However, most of these techniques focus on "how to prune" while neglecting the roles of each module in the ViT network structure. Currently known ViT pruning methods apply a uniform pruning ratio to the two crucial modules of the model: the attention mechanism (Multi-head Self Attention) and the feedforward network. This results in suboptimal parameter and computational optimization, limiting the choice of deployment devices and potentially affecting inference speed and accuracy after deployment.

[0057] To address the issue that existing monitoring data transmission suffers from significant time delays, hindering real-time monitoring and alerts and impacting security, this invention proposes a model deployment method.

[0058] Figure 1 A flowchart illustrating the steps of the model deployment method provided in this embodiment of the invention.

[0059] like Figure 1 As shown, the method includes the following steps:

[0060] Step 101: Obtain a pre-trained visual convolutional model, which is used to receive and process monitoring images of the substation area; the visual convolutional model consists of an attention mechanism module and a feedforward neural module.

[0061] In power system substations and other locations, it is often necessary to analyze the monitoring data from the installed cameras to identify any violations or other behaviors, thereby achieving automated monitoring.

[0062] Existing monitoring methods require transmitting images captured by cameras to the cloud and using visual convolutional models deployed in the cloud for analysis and recognition, which introduces significant time delays, making it difficult to achieve real-time monitoring and alerts, thus affecting security.

[0063] This solution compresses the visual convolutional model, enabling its deployment on resource-constrained edge computing devices. It performs analysis while monitoring images are being generated, thus achieving real-time online monitoring.

[0064] The visual convolutional model consists of two important modules: a multi-head self-attention mechanism and a feedforward network. This scheme uses a binary search method to determine the optimal pruning ratio for each module, thereby reducing the model size while maintaining accuracy. This allows the model to be deployed on resource-constrained edge computing devices, accelerating the transmission of monitoring data and enabling real-time monitoring and alerts.

[0065] Step 102: Set the accuracy target and search parameters for pruning the visual convolutional model. The search parameters include at least the initial pruning ratio and the search interval length.

[0066] Before performing formal pruning, the user needs to specify an optimization accuracy target, which is a fixed accuracy value. This method can search for pruning ratio parameters that minimize model size, computational cost, number of parameters, and inference speed based on this accuracy target. At the same time, it sets a fault-tolerant range for the target to prevent the method iteration from falling into an infinite loop, so that the final pruning target is an interval value.

[0067] The accuracy target represents the acceptable post-pruning accuracy for the user and can be expressed as a percentage of the model's original accuracy. For example, setting a percentage (%) and the original accuracy as origin (%), multiplying origin (%) by percentage (%) yields target (%), which is the accuracy target. Here, (%) indicates that the unit is percentage.

[0068] Since it is necessary to find the optimal pruning ratio for the attention mechanism module and the feedforward neural module, search parameters need to be set, including at least the initial pruning ratio and the search interval length.

[0069] For example, the initial pruning ratios att_ratio and ffn_ratio of the attention mechanism module and the feedforward neural module are set to 0%, respectively.

[0070] In addition, the search interval length (%) is defined in the candidate interval search, which refers to the range of each search.

[0071] Taking the attention mechanism module as an example, during the search, it starts from the initial pruning ratio att_ratio and increases the difference length by a fixed length each time.

[0072] Step 103: Using the accuracy target and the search parameters, search for the first target pruning ratio of the attention mechanism module and the second target pruning ratio of the feedforward neural module, respectively.

[0073] In visual convolutional models, the attention mechanism module has a significant impact on accuracy, but its parameters account for a relatively small percentage; the feedforward neural module has a smaller impact on accuracy, but its parameters account for a relatively large percentage. Therefore, we first search for the optimal pruning ratio of the feedforward neural module, and then search for the optimal pruning ratio of the attention mechanism module.

[0074] The search begins with the initial pruning ratio `att_ratio`, and each time a fixed difference length `length` is added. This allows the current pruning to utilize the weight file from the previous step. If the model accuracy obtained through pruning is less than or equal to the target accuracy, the pruning target is within the current candidate interval, and the search proceeds to the next step using binary search. If the model accuracy obtained through pruning is greater than the target accuracy, the pruning target is not within the current candidate interval, and the search process iterates through the next candidate interval.

[0075] Step 104: Optimize the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain the target visual convolution model.

[0076] Specifically, the attention mechanism module is pruned according to the first target pruning ratio, and the feedforward neural module is pruned according to the second target pruning ratio to obtain the target visual convolutional model.

[0077] Step 105: Deploy the target visual convolutional model on the edge computing device of the substation to receive the current monitoring image and process the monitoring image in real time.

[0078] Pruning the visual convolutional model minimizes its size, computational cost, and number of parameters, thus reducing the requirements for the runtime environment.

[0079] By deploying the target visual convolutional model on edge computing devices in substations, monitoring data can be received directly on these devices. Since the cameras are also deployed on edge computing devices, the distance between the sender and receiver of monitoring data is reduced, enabling real-time reception of monitoring data. This allows for real-time detection and alerts, improving the safety of the power system.

[0080] In summary, in this embodiment of the invention, a pre-trained visual convolutional model is obtained. This model is used to receive and process monitoring images of a substation area. The visual convolutional model consists of an attention mechanism module and a feedforward neural module. A precision target and search parameters are set for pruning the visual convolutional model. The search parameters include at least an initial pruning ratio and a search interval length. Using the precision target and the search parameters, a first target pruning ratio for the attention mechanism module and a second target pruning ratio for the feedforward neural module are searched. The visual convolutional model is optimized based on the first and second target pruning ratios to obtain a target visual convolutional model. The target visual convolutional model is deployed on an edge computing device in the substation to receive and process the current monitoring images in real time. Thus, after pruning the visual convolutional model, the model size, computational load, and number of parameters of the target visual convolutional model are minimized, thereby reducing the requirements for the operating environment. Deploying visual convolutional models on edge computing devices reduces the distance between the sender and receiver of monitoring data, enabling real-time reception of monitoring data. This allows for real-time detection and alerts, improving the safety of the power system.

[0081] In one possible implementation, prior to step 103, the method further includes:

[0082] Step 201: Load the visual convolutional model and load the pre-trained weights.

[0083] Specifically, import the weight file of the pre-trained visual convolutional model, load the pre-trained weight model, and use the weight model of the visual convolutional model as the basis for subsequent pruning.

[0084] Step 202: Set the initial running parameters for the pruning operation. The initial running parameters include at least: dataset reading path, number of images taken from the dataset each time, optimizer function, loss function, and fine-tuning rounds.

[0085] Pruning requires fine-tuning and testing, so preparatory work is necessary, including the following steps:

[0086] (1) Set the dataset path so that the dataset can be loaded correctly for training and inference;

[0087] (2) Set the batchsize parameter for reading the dataset, which is the number of images retrieved from the dataset each time. This parameter depends on the device's performance and is generally 64.

[0088] (3) Set the optimizer functions for training and testing. Depending on the specific task, this is generally a stochastic gradient descent policy function.

[0089] (4) Set the loss function for training and testing. Depending on the specific task, for classification tasks, it is generally the cross-entropy function.

[0090] (5) Set the number of fine-tuning epochs, which is the number of epochs for training after pruning. The default is 15 epochs. An epoch is an important concept in the training process of a neural network. One epoch is equal to the process of training once using all the samples in the training set.

[0091] In one possible implementation, step 102 includes:

[0092] Step 1021: Set the pruning accuracy target; the pruning accuracy target represents the acceptable model accuracy after pruning for the user.

[0093] Set the percentage (%), multiply the original precision (origin) by the percentage to get the target (%). The target (%) is the pruning precision target, that is, what percentage of the original precision the user can accept after pruning.

[0094] For example, if the original precision is 98% and the user specifies a percentage of 95%, then the target is 98% * 95% = 93.1%.

[0095] Step 1022: Set the fault tolerance range; the fault tolerance range represents the fault tolerance range of the algorithm.

[0096] After obtaining the pruning target, to avoid the method iteration falling into an infinite loop, the user needs to specify a tolerance range error (%). When the algorithm iterates to the range (target-error, target+error), it is considered to have met the user's expectations.

[0097] For example, if target = 93.1% and error = 0.1%, then the accuracy obtained by the algorithm is considered to have reached the target when it falls within (93.1% - 0.1%, 93.1% + 0.1%).

[0098] Step 1023: Set the initial pruning ratios for the attention mechanism module and the feedforward neural module, respectively.

[0099] Specifically, the initial pruning ratios att_ratio and ffn_ratio of the attention mechanism module and the feedforward neural module are each initialized to 0%.

[0100] Step 1024: Set the search interval length; the search interval length represents the range of each search.

[0101] Step 1025: Define the closest target range variable. If the algorithm has not found the target after iteration, return the closest target range variable.

[0102] Define variables that are closest to the target range, including the best_ffn_ratio of the feedforward neural module pruning ratio and the best_attn_ratio of the attention mechanism module pruning ratio at the target accuracy. These values ​​are returned when the algorithm has completed its iterations and still has not found the target.

[0103] Step 1026: Define the granularity threshold in the binary search, whereby the granularity threshold represents the minimum interval length that can be halved.

[0104] Based on extensive experimental results and relevant theories, generally speaking, the larger the pruning ratio, the greater the decrease in model accuracy. However, if the pruning ratio is adjusted only slightly, the above conclusion no longer holds true; that is, the accuracy is prone to fluctuations, and binary search becomes unsuitable. Therefore, this algorithm defines the minimum interval length that can be halved in binary search, and uses this minimum interval length as the granularity threshold. The granularity threshold is set to 1% by default.

[0105] In one possible implementation, the search interval length is a first proportion, and the step of searching for the first target pruning proportion of the attention mechanism module using the accuracy target and the search parameters includes:

[0106] Step 1031: Starting from the initial pruning ratio, each time the interval length of the first ratio is increased as the current pruning ratio of the attention mechanism module, and the first accuracy of the pruned model is obtained.

[0107] Step 1032: If the first precision is within the current candidate interval, then use the binary search method to search for the first target pruning ratio of the attention mechanism module within the current candidate interval;

[0108] Step 1033: If the first precision is not within the current candidate interval, then proceed to the next candidate interval to continue searching until a target interval range that satisfies the precision target is found. Then, use the binary search method to search for the first target pruning ratio of the attention mechanism module within the target interval range.

[0109] Steps 1031-1033 are mainly divided into two stages.

[0110] The first stage involves finding the approximate range of candidate intervals.

[0111] To incorporate the iterative pruning approach, `ffn_ratio` searches by increasing the initial ratio by a fixed interval length each time, allowing the current pruning to utilize the weight file from the previous step. If the defined accuracy target is within the current candidate interval, the process proceeds to the second stage; otherwise, if the current accuracy value is still greater than the accuracy target, it indicates that the current candidate interval is incorrect, and the process iterates through the first stage steps again to the next candidate interval.

[0112] The second stage involves a binary search within the candidate interval. After finding the candidate interval for ffn_ratio, a binary search is performed to search for a more refined ratio parameter. The initial left and right boundaries of the binary search are the left and right boundaries of the candidate interval. Based on the comparison between the median calculated from the left and right boundaries and the target range, the interval length of the binary search is continuously adjusted by halving until the first target pruning ratio that meets the accuracy requirements is found.

[0113] In one possible implementation, step 1033 includes:

[0114] Step 10331: Calculate the median based on the left and right boundaries of the target interval range, and use the median as the pruning ratio of the attention mechanism module to perform pruning operations.

[0115] Step 10332: Continue to adjust the interval length of the binary search method by halving until the current pruning ratio makes the accuracy of the obtained model meet the accuracy target, or the interval length of the binary search method is less than the preset granularity threshold.

[0116] Step 10333: Determine the current pruning ratio as the first target pruning ratio of the attention mechanism module, or determine the pruning ratio that is closest to the accuracy target recorded during the search process as the first target pruning ratio of the attention mechanism module.

[0117] In steps 10331-10333, based on a large number of experimental results and related theories, it is generally true that the larger the pruning ratio, the greater the decrease in model accuracy. However, if the adjustment of the pruning ratio is small, the above conclusion is no longer satisfied, that is, the accuracy is prone to fluctuation and it is no longer suitable for binary search. Therefore, this algorithm defines the minimum interval length that binary search can halve as the granularity threshold.

[0118] When the interval length of the binary search method is less than the granularity threshold, the binary search is stopped, and the pruning ratio of the closest precision target recorded during the search process is determined as the first target pruning ratio of the attention mechanism module.

[0119] When the interval length of the binary search method is greater than or equal to the granularity threshold, the current interval range is halved and the binary search continues.

[0120] Figure 2 This is a schematic diagram of the first target pruning ratio of the search attention mechanism module provided in an embodiment of the present invention.

[0121] like Figure 2 As shown, the search interval length in this example is 10%, and the granularity threshold is 1%.

[0122] After initialization, the first step is a candidate interval search phase. This phase checks if the target is within the current 10% interval. If not, it searches the next 10% interval. If so, it proceeds to the binary search phase. It determines if the binary search finds the target. If yes, the search ends; otherwise, the interval is halved, and the interval length is checked to ensure it is greater than or equal to 1%. If yes, the interval is halved again, and the binary search continues. If not, the pruning ratio of the closest recorded target is determined as the first target pruning ratio.

[0123] Figure 3 This is a schematic diagram of the binary search method provided in an embodiment of the present invention.

[0124] like Figure 3 As shown, the interval [93.1% + 0.1%, 93.1% - 0.1%] represents the target pruning range. Below the horizontal axis represents the pruning percentage of the module, and above the horizontal axis represents the model accuracy after pruning at that percentage. Binary search operates within the candidate interval; the arrows represent the search path, and the numbers represent the order. In this example, the search path for the pruning percentage is 90% -> 85% -> 88% -> 86% -> 87%, ultimately determining the first target pruning percentage as 87%.

[0125] In one possible implementation, the search interval length is a first ratio, and the step of retrieving the second target pruning ratio of the feedforward neural module using the accuracy target and the search parameters includes:

[0126] Step 1034: Starting from the initial pruning ratio, each time the interval length of the first ratio is increased as the current pruning ratio of the feedforward neural module, and the second precision of the pruned model is obtained.

[0127] Step 1035: If the second precision is within the current candidate interval, then use the binary search method to search for the second target pruning ratio of the feedforward neural module within the current candidate interval;

[0128] Step 1036: If the second precision is not within the current candidate interval, then proceed to the next candidate interval to continue searching until a target interval range that satisfies the precision target is found. Then, use the binary search method to search for the second target pruning ratio of the feedforward neural module within the target interval range.

[0129] Steps 1034-1036 describe the method by which the feedforward neural module searches for the pruning ratio of the second target. This method is similar to the method by which the attention mechanism module searches for the pruning ratio of the first target. For details, please refer to steps 1031-1033, which will not be repeated here.

[0130] Figure 4 This is a structural block diagram of the model deployment device provided in an embodiment of the present invention. Figure 4 As shown, the device 300 includes:

[0131] The model acquisition module 301 is used to acquire a pre-trained visual convolutional model, which is used to receive and process monitoring images of the substation area; the visual convolutional model consists of an attention mechanism module and a feedforward neural module.

[0132] The setting module 302 is used to set the accuracy target and search parameters for pruning the visual convolutional model. The search parameters include at least the initial pruning ratio and the search interval length.

[0133] Search module 303 is used to search for the first target pruning ratio of the attention mechanism module and the second target pruning ratio of the feedforward neural module using the accuracy target and the search parameters, respectively.

[0134] Optimization module 304 is used to optimize the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain the target visual convolution model;

[0135] The deployment module 305 is used to deploy the target visual convolutional model on the edge computing device of the substation, so as to receive the current monitoring image using the target visual convolutional model and process the monitoring image in real time.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0138] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0139] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A model deployment method, characterized in that, include: Obtain a pre-trained visual convolutional model, which is used to receive and process monitoring images of the substation area; The visual convolutional model consists of an attention mechanism module and a feedforward neural module; Set the accuracy target and search parameters for pruning the visual convolutional model, wherein the search parameters include at least the initial pruning ratio and the search interval length; Using the accuracy target and the search parameters, the first target pruning ratio of the attention mechanism module and the second target pruning ratio of the feedforward neural module are searched respectively. The visual convolution model is optimized based on the first target pruning ratio and the second target pruning ratio to obtain the target visual convolution model. The target visual convolutional model is deployed on the edge computing device of the substation to receive the current monitoring image and process the monitoring image in real time.

2. The method according to claim 1, characterized in that, Before searching for the first target pruning ratio of the attention mechanism module using the accuracy target and the search parameters, the method further includes: Load the visual convolutional model and load the pre-trained weights; Set the initial running parameters for the pruning operation. The initial running parameters include at least: the dataset reading path, the number of images taken from the dataset each time, the optimizer function, the loss function, and the number of fine-tuning rounds.

3. The method according to claim 1, characterized in that, The setting of the accuracy target and search parameters for pruning the visual convolutional model includes: Set a pruning accuracy target; the pruning accuracy target represents the acceptable pruning accuracy of the model for the user. The fault-tolerant range is defined; the fault-tolerant range represents the fault-tolerant range of the algorithm. Set the initial pruning ratios for the attention mechanism module and the feedforward neural module respectively; Set the search interval length; the search interval length represents the range of each search. Define a variable that is closest to the target range. If the target is not found after the algorithm has completed its iterations, return the variable that is closest to the target range. Define a granularity threshold in the binary search, whereby the granularity threshold represents the minimum interval length that can be halved.

4. The method according to claim 1, characterized in that, The search interval length is a first ratio, and the step of searching for the first target pruning ratio of the attention mechanism module using the accuracy target and the search parameters includes: Starting from the initial pruning ratio, the interval length of the first ratio is increased each time as the current pruning ratio of the attention mechanism module, and the first accuracy of the pruned model is obtained. If the first precision is less than or equal to the target precision, then the first target pruning ratio of the attention mechanism module is searched within the current candidate interval using a binary search method. If the first precision is greater than the target precision, then proceed to the next candidate interval to continue searching until a target interval range whose precision value meets the target precision is found. Then, use the binary search method to search for the first target pruning ratio of the attention mechanism module within the target interval range.

5. The method according to claim 4, characterized in that, The step of searching for the first target pruning ratio of the attention mechanism module within the target interval using a binary search method includes: The median is calculated based on the left and right boundaries of the target interval, and the median is used as the pruning ratio of the attention mechanism module for pruning operations. Continue to halve the interval length of the binary search method until the current pruning ratio makes the accuracy of the obtained model meet the accuracy target, or the interval length of the binary search method is less than the preset granularity threshold. The current pruning ratio is determined as the first target pruning ratio of the attention mechanism module, or the pruning ratio recorded during the search process that is closest to the accuracy target is determined as the first target pruning ratio of the attention mechanism module.

6. The method according to claim 1, characterized in that, The search interval length is a first ratio, and the step of retrieving the second target pruning ratio of the feedforward neural module using the accuracy target and the search parameters includes: Starting from the initial pruning ratio, the interval length of the first ratio is increased each time as the current pruning ratio of the feedforward neural module, and the second accuracy of the pruned model is obtained. If the second precision is less than or equal to the target precision, then the second target pruning ratio of the feedforward neural module is searched within the current candidate interval using a binary search method. If the second precision is greater than the target precision, then proceed to the next candidate interval to continue searching until a target interval range whose precision value meets the target precision is found. Then, use the binary search method to search for the second target pruning ratio of the feedforward neural module within the target interval range.

7. A model deployment device, characterized in that, include: The setting module is used to set the accuracy target and search parameters for pruning the visual convolutional model. The search parameters include at least the initial pruning ratio and the search interval length. The search module is used to search for the first target pruning ratio of the attention mechanism module and the second target pruning ratio of the feedforward neural module using the accuracy target and the search parameters, respectively. The optimization module is used to optimize the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain the target visual convolution model; The deployment module is used to deploy the target visual convolutional model on the edge computing device of the substation, so as to receive the current monitoring image using the target visual convolutional model and process the monitoring image in real time.

8. The apparatus according to claim 7, characterized in that, The device further includes: The loading module is used to load the visual convolutional model and load the pre-trained weights; The initial parameter setting module is used to set the initial running parameters of the pruning operation. The initial running parameters include at least: the dataset reading path, the number of images taken from the dataset each time, the optimizer function, the loss function, and the number of fine-tuning rounds.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the model deployment method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the server's processor, the server is able to perform the model deployment method as described in any one of claims 1 to 6.

Images