Model structure adjustment method and apparatus
By using automated search algorithms and equipment computing power during the model pruning process, the pruning parameters are automatically adjusted, and the high cost and low efficiency problems caused by relying on human resources in the existing technology are solved, and the efficient automatic pruning and parameters of the model are realized. optimization.
Patent Information
- Application Number
- PCT/CN2024/091529
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-09
- Filing Date
- 2024-05-07
- Publication Date
- 2024-11-14
AI Technical Summary
In the prior art, the model pruning process relies too much on human resources, resulting in high implementation costs and low efficiency.
By using automated search algorithms and equipment computing power during the model pruning process, the pruning parameters are automatically adjusted according to performance parameters until preset conditions are met, and the model is automatically pruning and parameter optimization.
Reliance on human resources is reduced, pruning efficiency and efficiency are improved, and realization costs are reduced.
Smart Images

Figure CN2024091529_14112024_PF_FP_ABST
Abstract
Description
Model structure adjustment method and device
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 9, 2023, with application number 202310518136.0 and application name “Model Structural Adjustment Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of computer technology, and in particular to a model structure adjustment method, a speech recognition method, a text translation method, a device, an electronic device, and a machine-readable medium. Background Art
[0003] Machine learning models have been widely used and successfully applied in various fields. The efficiency of problem solving has been greatly improved through model calculations. However, the models require computing resources, and there are cost limitations in deploying the models on running devices.
[0004] Currently, pruning can be used to partially remove parts of a model's structure. This reduces model parameters and computational complexity while ensuring accuracy, making the model lightweight and reducing deployment costs. During pruning, pruning parameters are manually determined. If the resulting model doesn't meet requirements, a new set of pruning parameters can be manually designed and the model pruned again.
[0005] However, after research, the inventors found that the implementation of the current solution is too dependent on human resources, resulting in high implementation costs and low efficiency.
[0006] Summary of the Invention
[0007] The embodiments of the present application provide a model structure adjustment method, a speech recognition method, and a text translation method to solve the problem in related technologies of over-reliance on human resources, resulting in high implementation costs and low efficiency.
[0008] Correspondingly, the embodiments of the present application also provide a model structure adjustment device, a speech recognition device, a text translation device, an electronic device and a storage medium to ensure the implementation and application of the above method.
[0009] In order to solve the above problems, the present invention discloses a method for adjusting the structure of a model, which includes:
[0010] Performing a pruning operation on the structural layer of the first model according to the first pruning parameter to obtain a second model;
[0011] running the second model to obtain performance parameters of the second model;
[0012] When the performance parameter of the second model does not meet the preset conditions, obtaining a second pruning parameter according to a preset search algorithm;
[0013] After the structural layer of the first model is pruned according to the second pruning parameter to obtain a third model, if the performance parameter obtained by running the third model meets the preset condition, the third model is output for use.
[0014] The present application discloses a speech recognition method, comprising:
[0015] Inputting the audio file into a third model to obtain a text recognition result of the audio file output by the third model;
[0016] The third model is obtained after being processed by the structural adjustment method of the model.
[0017] The present application discloses a text translation method, including:
[0018] Inputting the text recognition result of the audio file into a third model to obtain a translation result of the audio file output by the third model;
[0019] The third model is obtained after being processed by the structural adjustment method of the model.
[0020] The present application discloses a device for adjusting the structure of a model, the device comprising:
[0021] A first pruning module is used to prune the structural layer of the first model according to the first pruning parameter to obtain a second model;
[0022] An operating module, configured to operate the second model and obtain performance parameters of the second model;
[0023] A search module, configured to obtain a second pruning parameter according to a preset search algorithm when the performance parameter of the second model does not meet a preset condition;
[0024] The second pruning module is used to prune the structural layer of the first model according to the second pruning parameters to obtain a third model, and if the performance parameters obtained by running the third model meet the preset conditions, output the third model for use.
[0025] The present application discloses a speech recognition device, comprising:
[0026] a speech recognition module, configured to input an audio file into a third model and obtain a text recognition result of the audio file output by the third model;
[0027] The third model is obtained after being processed by the structure adjustment device of the model.
[0028] The present application discloses a text translation device, comprising:
[0029] a text translation module, configured to input the text recognition result of the audio file into a third model and obtain a translation result of the audio file output by the third model;
[0030] The third model is obtained after being processed by the structure adjustment device of the model.
[0031] An embodiment of the present application further discloses an electronic device, comprising: a processor; and a memory on which executable code is stored. When the executable code is executed, the processor executes one or more methods described in the embodiments of the present application.
[0032] The embodiments of the present application also disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.
[0033] Compared with the related art, the embodiments of the present application have the following advantages:
[0034] In the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model. If the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] FIG1 is a system architecture diagram of an embodiment of the present application;
[0036] FIG2 is a schematic diagram of a pruning scenario of a speech recognition model according to an embodiment of the present application;
[0037] FIG3 is a schematic diagram of a pruning scenario of a text translation model according to an embodiment of the present application;
[0038] FIG4 is a schematic diagram of a pruning scenario of a computer vision model according to an embodiment of the present application;
[0039] FIG5 is a flowchart of a method for adjusting the structure of a model according to an embodiment of the present application;
[0040] FIG6 is a flowchart showing the specific steps of a method for adjusting the structure of a model according to an embodiment of the present application;
[0041] FIG7 is a schematic diagram of pruning of a local structural layer of a first model according to an embodiment of the present application;
[0042] FIG8 is a schematic diagram of a change in the range of pruning parameters in a search algorithm according to an embodiment of the present application;
[0043] FIG9 is a flowchart of the steps of a speech recognition method according to an embodiment of the present application;
[0044] FIG10 is a flowchart of a text translation method according to an embodiment of the present application;
[0045] FIG11 is a block diagram of a structure adjustment device for a model according to an embodiment of the present application;
[0046] FIG12 is a schematic structural diagram of a device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0047] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0048] To help those skilled in the art better understand this application, the concepts involved in this application are explained below:
[0049] Model pruning: A method of adjusting the model structure. The pruning process is based on pruning parameters. It determines the proportion of structural layers that can be pruned in each structural layer of the model. Then, the parameters in the structural layers that can be pruned are deleted in their entirety according to the proportion, thereby achieving a lightweight model structure and reducing the number of parameters and the amount of computation during model runtime.
[0050] Pruning parameter: defines the ratio of the structural layers that can be pruned in the model. Specifically, it can be a ratio value in the range of 0 to 1. For example, if the pruning ratio of the convolutional layer of the neural network model is 0.4, then during pruning, 40% of the input channels of the convolutional layer can be pruned and 40% of the output channels can be pruned.
[0051] Performance parameters: Parameters that consider the running speed of the model, which can be specifically defined as the time it takes for the model to obtain output data from input data.
[0052] Search algorithm: A method that uses the high performance of a computer to purposefully exhaustively enumerate parts or possible scenarios of a problem's solution space, thereby finding a solution to the problem. Currently, common algorithms include enumeration algorithms, depth-first search, breadth-first search, backtracking algorithms, Monte Carlo tree search, and hash functions. Embodiments of the present application can utilize the computing resources of the device and, through a search algorithm, efficiently search and determine a pruning parameter within a pruning parameter range based on meeting the pruning objectives of the model.
[0053] Model: In the embodiments of the present application, it can be a machine learning model. A machine learning model is a mathematical structure that is expressed in a general or approximate manner using mathematical language with reference to the characteristics or quantitative dependencies of a certain system of things. This mathematical structure is a pure relational structure of a certain system portrayed with the help of mathematical symbols. In a broad sense, machine learning models include various concepts, formulas and theories in mathematics. In a narrow sense, machine learning models only refer to those mathematical relational structures that reflect specific problems or specific concrete systems of things. Specifically, machine learning models learn the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in interpreting data such as text, images and sounds. The ultimate goal of a machine learning model is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. A machine learning model can be a neural network model. A neural network model is a model that simulates the working principles of the human brain to implement artificial intelligence-like machine learning technology. It supports processing multiple types of data such as images, text, speech and sequences, and can implement classification, regression and prediction.
[0054] Structural layer: Also known as the model-level module, it is the computational layer in the model used to implement specific computing functions, such as the convolutional layer and the fully connected layer.
[0055] Target speedup: It is the ratio of the performance parameters before model pruning to the performance parameters after model pruning. Specifically, it is a parameter greater than 1. The target speedup is used to represent the indicators that the model performance can achieve.
[0056] The model structure adjustment method in the embodiments of the present application can be applied to scenarios where the model structure is pruned, including pruning computer vision models, speech recognition models, and natural language processing models. Taking the speech recognition model pruning scenario as an example, the embodiments of the present application can prune the structural layer of the speech recognition model while ensuring the accuracy of the speech recognition model, with the goal of reducing the deployment cost of the speech recognition model. According to the ratio set by the pruning parameters, some parameters in the structural layer of the speech recognition model are deleted, thereby reducing the number of parameters in the speech recognition model and the amount of computation required at runtime. This can reduce the cost of deploying the speech recognition model on the device.
[0057] 1, which shows a system architecture diagram of a model structure adjustment method provided by an embodiment of the present application, including: a pruning server and a deployment device. The pruning server includes: a pruning layer, a replacement layer, an inference speed measurement layer, and a search algorithm layer.
[0058] Specifically, the pruning layer of the pruning server can take the original first model as input and prune the first model according to a first pruning parameter to obtain a pruned second model. For the first pruning, the first pruning parameter used can be a preset parameter. The pruning parameter can specifically be a proportional value in the range of 0 to 1. During the pruning process, the first model can be divided layer by layer to obtain multiple model-level modules, and each model-level module can be pruned separately according to the proportion defined by the first pruning parameter.
[0059] After the model-level modules are pruned separately, the pruned model-level modules can be obtained. The replacement layer of the pruning server can directly replace the original model-level modules in the first model with the pruned model-level modules, so that each model-level module is pruned, that is, the pruned model-level modules are used to quickly complete the replacement of the corresponding modules in the first model and obtain a new second model, thereby quickly replacing and updating the pruned model-level modules of the first model and completing the construction of the new model structure.
[0060] After the replacement layer is executed, the second model is obtained. The inference speed measurement layer can run the second model and record the time it takes for the second model to complete the process from inputting data to obtaining output data. This time can be used as the performance parameter of the second model. The performance parameter is a parameter that characterizes the running speed of the model and can be used to judge whether the response speed of the pruned second model meets the requirements. If the performance parameters of the second model meet the preset conditions, the second model can be output and deployed on the deployment device for use.
[0061] If the replacement layer determines that the performance parameters of the second model do not meet the preset conditions, the pruning server can use its own computing power and, based on the search algorithm layer's search algorithm, rapidly enumerate the pruning parameter range to find a new second pruning parameter. Specifically, during the search process, an inference speed measurement can be performed for each pruning parameter enumeration. Based on the difference between the performance parameters obtained by the speed measurement and the target speedup factor, the pruning parameter range is continuously narrowed until the second pruning parameter is determined from the final narrowed pruning parameter range, ensuring that the second pruning parameter meets the expected pruning goal as much as possible.
[0062] After the search algorithm layer obtains the new second pruning parameters, the second pruning parameters can be sent to the pruning layer. The pruning layer uses the new second pruning parameters to re-prune the first model, and obtains the third model after processing by the replacement layer. The logic subsequently executed by the third model in the inference speed measurement layer and the search algorithm layer is the same as the logic executed by the second model in the inference speed measurement layer and the search algorithm layer, and will not be repeated here.
[0063] Ultimately, the inference speed test layer can output the pruned model whose performance parameters meet the preset conditions and deploy it on the deployment device for use, thereby achieving the purpose of pruning the model to achieve model lightweighting and reducing the cost of deploying the model on the deployment device.
[0064] In the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model. If the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs.
[0065] It can be explained that, based on the structural adjustment method of the model provided in the embodiments of the present application, several specific scenarios that can be implemented are as follows:
[0066] In one implementation, referring to FIG2 , a schematic diagram of a pruning scenario of a speech recognition model provided in an embodiment of the present application is shown, including: a user device and a pruning server, wherein the pruning server can prune the speech recognition model through the processing flow of the above embodiment, and iteratively update the pruning parameters in a loop until a speech recognition model with performance parameters that meet preset conditions is output and deployed on the user device, thereby reducing the cost of deploying the speech recognition model on the user device. The pruned speech recognition model can take an audio file as input and output the speech recognition result of the audio file with a faster response time while meeting the recognition accuracy.
[0067] In another implementation, referring to FIG3 , a schematic diagram of a pruning scenario for a text translation model provided by an embodiment of the present application is shown, including a user device and a pruning server. The pruning server can prune the text translation model using the processing flow of the above embodiment, and iteratively update the pruning parameters until a text translation model with performance parameters that meet preset conditions is output and deployed on the user device, thereby reducing the cost of deploying the text translation model on the user device. The pruned text translation model can take the text recognition results of an audio file as input and, while meeting recognition accuracy, output a translation of the text recognition results with a relatively fast response time.
[0068] In another implementation, referring to FIG4 , a schematic diagram of a pruning scenario for a computer vision model provided by an embodiment of the present application is shown, including: a user device and a pruning server. The pruning server can prune the computer vision model through the processing flow of the above embodiment, and iteratively update the pruning parameters until a computer vision model with performance parameters that meet preset conditions is output and deployed on the user device, thereby reducing the cost of deploying the computer vision model on the user device. The pruned computer vision model can take an image as input and, while meeting processing accuracy, output image processing results (recognition results, classification results, etc.) with a faster response time.
[0069] It can be explained that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data can comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0070] 5 , which shows a flowchart of the steps of a model structure adjustment method provided in an embodiment of the present application, including:
[0071] Step 101: Perform a pruning operation on the structural layer of the first model according to the first pruning parameter to obtain a second model.
[0072] In the embodiments of the present application, the application of machine learning models in artificial intelligence in various fields has achieved great success. These successes are driven by three factors: algorithms, data and computing power. Among them, computing power can support the design of large models and the processing of massive amounts of data. With the continuous breakthroughs in hardware technology in recent years, computing power has also increased exponentially, which has enabled many challenging problems in the past to be successfully solved through more complex models and larger amounts of data. Although the increasing demand for computing power of machine learning models is the current mainstream trend, expensive computing resources are cost-limited in real-world environments. For example, typical neural network models are difficult to deploy in Internet of Things (IOT) devices that lack computing power or have strict delay requirements. In addition, in some other scenarios, the pursuit of fast response of machine learning models can be met by compressing the model structure to save the number of model parameters and computing power, so that model pruning processing has been applied.
[0073] Specifically, the pruning process is based on the pruning parameters to determine the proportion of structural layers that can be pruned in each structural layer of the model. Then, the parameters in the structural layers that can be pruned are deleted in their entirety according to the proportion, thereby achieving lightweight model structure and reducing the number of parameters and the amount of computation during model runtime.
[0074] In an embodiment of the present application, the first pruning parameter used for the first pruning of the first model may be a preset parameter. For example, the first pruning parameter may be a value in the range of 0 to 1.
[0075] Step 102: Run the second model to obtain performance parameters of the second model.
[0076] In an embodiment of the present application, in order to determine whether the pruned second model has achieved the expected pruning target, the second model can be first tested to obtain the performance parameters of the second model, and then the performance parameters of the second model can be used to determine whether the expected pruning target has been achieved.
[0077] Optionally, in an embodiment of the present application, step 102 can be specifically implemented by inputting input data into the second model, obtaining output data of the second model, and recording the time consumed from inputting the input data to obtaining the output data, and using the consumed time as a performance parameter of the second model.
[0078] Specifically, the performance parameter of a model can be characterized by the time it takes for the model to input data and output the results. The shorter the time it takes for the model to input data and obtain output data, the faster the model responds. Therefore, in an embodiment of the present application, input data can be collected and input into a second model to obtain output data from the second model. The time it takes from inputting the input data to obtaining the output data is recorded as the performance parameter of the second model.
[0079] For example, for the pruned speech recognition model, an audio file can be input into the pruned speech recognition model, and the input time a is recorded. After calculation, the pruned speech recognition model will output the speech recognition result of the audio file at time b. The time consumed between time a and time b can be used as the performance parameter of the pruned speech recognition model.
[0080] Step 103: When the performance parameters of the second model do not meet the preset conditions, obtain second pruning parameters according to a preset search algorithm.
[0081] In an embodiment of the present application, the performance parameter is a parameter that considers the running speed of the model, and can be used to judge whether the response speed of the pruned second model meets the requirements. If the performance parameter of the second model meets the preset conditions, the second model can be output and deployed on the deployment device for use.
[0082] If the performance parameters of the second model do not meet the preset conditions, the device can use its own computing power, based on the search algorithm, to search for a new second pruning parameter in a fast enumeration manner from within the pruning parameter range. Among them, the search algorithm is a method of using the high performance of the computer to purposefully enumerate part or possible situations of a problem solution space, thereby finding a solution to the problem. In the pruning parameter search scenario of the embodiment of the present application, the computing resources of the device can be used to achieve fast enumeration through the search algorithm, based on meeting the pruning goal of the model, to efficiently search and determine a pruning parameter from a pruning parameter range.
[0083] Specifically, the reason why the embodiment of the present application uses a search algorithm to implement pruning parameters is explained: since pruning parameters are floating-point numbers, there are many possible values. If the values are enumerated one by one, a large number of cycles and time can be consumed. Therefore, in order to reflect the advantages of automation, a search algorithm can be introduced to replace the basic enumeration method, allowing the program to automatically quickly find a value that meets the target as a pruning parameter from a larger search space (pruning parameter range of 0 to 1), or an approximate value that is very close to the target as a pruning parameter. The purpose of doing this is to greatly improve the efficiency of program operation and save a lot of manual intervention costs.
[0084] Furthermore, during the search process, an inference speed test can be performed corresponding to each pruning parameter enumeration. Based on the gap between the performance parameter obtained by the speed test and the target acceleration factor (the ratio of the performance parameter before model pruning to the performance parameter after model pruning, specifically a parameter greater than 1, the target acceleration factor is used to characterize the achievable index of model performance), the pruning parameter range is continuously reduced until a second pruning parameter is determined from the final reduced pruning parameter range, so that the second pruning parameter meets the expected pruning goal as much as possible.
[0085] Step 104: After pruning the structural layer of the first model according to the second pruning parameters to obtain a third model, if the performance parameters obtained by running the third model meet the preset conditions, the third model is output for use.
[0086] In an embodiment of the present application, after the search algorithm obtains the new second pruning parameters, the second pruning parameters can be used to re-prune the first model to obtain a third model. The third model subsequently continues to execute the logic of steps 102-104 until a pruned model with performance parameters that meet the preset conditions is obtained after loop iteration for output and use.
[0087] Deploying the pruned model whose performance parameters meet the preset conditions on the deployment device can achieve the purpose of pruning the model to make it lightweight, and reduce the cost of deploying the model on the deployment device.
[0088] In summary, in the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model, and when the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs.
[0089] 6 , which shows a flowchart of specific steps of a structural adjustment method of a model provided in an embodiment of the present application, including:
[0090] Step 201: Perform a pruning operation on a first structural layer of a first model according to a first pruning parameter to obtain a pruned second structural layer.
[0091] In the embodiments of the present application, a structural layer is an operational layer within a model used to implement specific computational functions. A model may have multiple sequentially connected structural layers, such as convolutional layers and fully connected layers. Pruning is an operation performed on the first structural layer of a first model, aiming to proportionally delete portions of the first structural layer, such as deleting entire rows of parameters within the structural layer. This reduces the model structure of the pruned second structural layer, thereby reducing the number of parameters and the amount of computation required during the operation of the second structural layer.
[0092] Optionally, step 201 may specifically include:
[0093] Sub-step 2011: Divide the first structural layer of the first model layer by layer to obtain multiple model hierarchical modules.
[0094] Sub-step 2012: performing a pruning operation on the model-level module according to the first pruning parameter corresponding to the model-level module, and forming the second structural layer with the pruned model-level module.
[0095] In an embodiment of the present application, for sub-steps 2011 to 2012, the first structural layer of the first model is divided layer by layer to obtain multiple model hierarchical modules. The model hierarchical modules can be regarded as a single structural layer and used to implement specific functions in the overall processing process of the model.
[0096] For example, when the first model is a convolutional neural network model, the convolution layer can be a model-level module of the first model. The convolution layer can perform convolution calculations, and the convolution calculations can extract features of the input data, making it convenient for subsequent feature-based processing of the input data.
[0097] Specifically, after the first structural layer of the first model is divided layer by layer to obtain multiple model-level modules, the first pruning parameter defines the pruning ratio of the model-level modules. Subsequently, the single model-level module can be pruned according to the ratio defined by the first pruning parameter, and the pruned model-level modules constitute the second structural layer.
[0098] For example, referring to FIG7 , which shows a schematic diagram of pruning a local structure layer of a first model, it is assumed that the first model includes a first convolutional layer and a second convolutional layer, and the width W of the first convolutional layer is 64×64×3×3 (representing C out ,C in,k,k), the bias parameter B is 64; the width W of the second convolutional layer is 64×64×3×3, and the bias parameter B is 64. The first pruning parameter of the first convolutional layer is to prune 12.5% of the input channel and 50% of the output channel; the first pruning parameter of the second convolutional layer is to prune 50% of the input channel and 12.5% of the output channel; when pruning is performed, after the first convolutional layer is pruned, the width W becomes 32×56×3×3, and the bias parameter B becomes 32; after the second convolutional layer is pruned, the width W becomes 56×32×3×3, and the bias parameter B becomes 56.
[0099] Furthermore, a weight matrix for a single convolutional layer is provided. The process of structured pruning:
[0100] 1. First, convert (and transpose) the weight matrix W into a two-dimensional matrix:
[0101] Where column = k × k × C in ;
[0102] 2. Calculate the L1 or L2 norm of each row in the weight matrix W:
[0103] V i =||W (i,:) ||1 or W (i,:) ||2,
[0104] 3. Sort the elements in V from smallest to largest. According to the pruning ratio set by the first pruning parameter, find the corresponding smaller elements in V and record their position index Ω0 in the original V before sorting. For example, if the pruning ratio is 30%, find the first 30% of the elements with the smallest values in V after sorting and record their position index before sorting.
[0105] 4. Create a matrix with the same dimensions as W The element values are all 1; according to the index obtained in step 3, set the corresponding row in M to 0, that is, M (i,:) =0,i∈Ω0;
[0106] 5. Multiply all elements in W by the corresponding elements in M, that is, cut the weights with smaller corresponding values according to the row dimension:
[0107] 6. Convert W and M back to their original shapes And output.
[0108] Step 202: Replace the first structural layer of the first model with the second structural layer to obtain a second model.
[0109] In an embodiment of the present application, the second structural layer obtained after cropping can be used as a new object and is not directly saved on the local disk, but directly replaces the original first structural layer of the first model to obtain the second model.
[0110] Optionally, step 202 may specifically include:
[0111] Sub-step 2021: When a pruned model level module is obtained, the pruned model level module is used to replace the corresponding model level module in the first structural layer, and after at least part of the model level modules in the first structural layer are replaced, the second model is obtained.
[0112] Specifically, for example, after the model-level module: model.original_layer in the first structural layer is pruned, the corresponding new model-level module in the second structural layer is: new_layer. Then, the embodiment of the present application can directly replace the member variables and quickly use the pruned model-level module: new_layer to replace the original model-level module: model.original_layer, thereby quickly replacing part of the model-level modules of the original model and generating a new model structure.
[0113] In related technologies, each trimmed model-level module records its new definition and shape information and saves the parameter values to the local disk to save the file of the trimmed model-level module. Then, based on the definition and shape information of the trimmed model-level module, a new model-level module is recreated, and the local parameter values are loaded into the object. After all the trimmed model-level modules are created, inference speed testing is performed. The problem with this is that there are many steps, the inference speed testing process is discontinuous, and many local read and write operations may cause additional delays. In addition, manual intervention is required to recreate the model-level module, making automated process execution impossible.
[0114] The solution of the embodiment of the present application automatically replaces the pruned model hierarchy modules, quickly replaces part of the model hierarchy modules of the original model, and thus generates a new model structure. The inference speed measurement process is continuous and the steps are simple, which reduces a large number of local read and write operations, and reduces latency and manual dependence.
[0115] Step 203: Store the second model in a preset memory area.
[0116] In an embodiment of the present application, the pruned second model can be retained in a preset memory area and cannot be directly saved to a local disk file. In this way, during subsequent inference speed testing, the second model can be directly read from the memory area for inference speed testing, without having to first read the second model file from the local disk into the memory, and then read the second model file from the memory area for inference speed testing. This reduces a large number of read and write operations and reduces latency.
[0117] Step 204: Run the second model in the preset memory area to obtain performance parameters of the second model.
[0118] In an embodiment of the present application, by directly running the second model in the memory area to obtain the performance parameters of the second model, it is possible to avoid saving the second model to a local disk file and then reading the second model file from the local disk to the memory for execution, thereby reducing a large number of read and write operations and reducing latency.
[0119] Step 205: When the performance parameters of the second model do not meet the preset conditions, obtain second pruning parameters according to a preset search algorithm.
[0120] This step may be specifically referred to the above step 103 and will not be described in detail here.
[0121] Optionally, step 205 may specifically include:
[0122] Sub-step 2051: Obtain a target acceleration factor, a first pruning parameter range, and performance parameters of the first model.
[0123] Sub-step 2052: If the ratio of the performance parameter of the first model to the target performance parameter is not equal to the target speedup factor, adjust the boundary value of the pruning parameter range according to the search algorithm and the target pruning parameter determined by the pruning parameter range before adjustment in the current adjustment process; the target performance parameter is the performance parameter of the model obtained by pruning the first model using the target pruning parameter.
[0124] Sub-step 2053: After adjusting the boundary value of the pruning parameter range at least once, determining the second pruning parameter according to the adjusted pruning parameter range.
[0125] In an embodiment of the present application, for sub-steps 2051-2053, when the performance parameters of the second model do not meet the preset conditions, the second pruning parameters can be obtained according to the preset search algorithm for re-pruning. The search algorithm is implemented specifically by using the performance parameters obtained by reasoning the model after pruning as feedback for the search algorithm judgment, and using the target acceleration multiple representing the desired pruning target as a reference standard for the search algorithm judgment. Through automated enumeration, the second pruning parameters that meet the desired pruning target are quickly found from a larger search space, thereby greatly improving the efficiency of program operation and saving a lot of manual intervention costs.
[0126] Specifically, the search algorithm is implemented through multiple rounds of loop iterations. In the first round of iterations, the ratio of the performance parameter of the first model to the target performance parameter can be calculated. This ratio is used to reflect the acceleration factor of the current pruned model. This ratio can be compared with the ideal target acceleration factor to determine whether the current iteration meets the expected requirements. The performance parameter of the first model is a known value. The target performance parameter of the first round of iteration can be: the target pruning parameter in the first round of iteration is the performance parameter obtained after the model inference speed test after pruning the first model. For example, when the pruning parameter range is 0-1, the target pruning parameter of the first round of iteration can be the middle value of the pruning parameter range, 0.5, where the target pruning parameter can be used to prune the original first model. The performance parameter of the pruned model obtained after the inference speed test is the target performance parameter of the first round of iteration.
[0127] In the first iteration, the boundary values of the pruning parameter range are low_r = 0 and high_r = 1. The target performance parameter t1 is the performance parameter of the pruned model obtained by pruning the first model using the target pruning parameter 0.5, after inference speed testing. The ratio of the first model's performance parameter t0 to the target performance parameter t1 is then compared with the target speedup factor x. If the ratio of the first model's performance parameter t0 to the target performance parameter t1 is equal to the target speedup factor x, the target pruning parameter 0.5 from the first iteration is used as the second pruning parameter, meaning that 0.5 can achieve the desired pruning target. If the ratio of the first model's performance parameter t0 to the target performance parameter t1 is not equal to the target speedup factor x, the target pruning parameter 0.5 enumerated in the first iteration is considered to have not met the desired pruning target. The boundary values of the pruning parameter range can be adjusted using the target pruning parameter 0.5 from the first iteration, thereby narrowing the pruning parameter range. The target pruning parameter for the next iteration is then determined from within the narrowed pruning parameter range, and the next iteration is entered.
[0128] In the second iteration, if the target pruning parameter determined from the narrowed pruning parameter range in the first iteration is 0.25, then the second iteration can use the target pruning parameter 0.25 to prune the first model to obtain the pruned model. After inference speed testing, the pruned model obtains the target performance parameter t2 for the second iteration. The ratio of the performance parameter t0 of the first model to the target performance parameter t2 is then compared with the target speedup factor x. If the ratio of the performance parameter t0 of the first model to the target performance parameter t2 is equal to the target speedup factor x, the target pruning parameter 0.25 for the second iteration is used as the second pruning parameter. If the ratio of the performance parameter t0 of the first model to the target performance parameter t2 is not equal to the target speedup factor x, then it is considered that the target pruning parameter 0.25 enumerated in the second iteration does not meet the expected pruning target. The target pruning parameter 0.25 of the second iteration can be used to adjust the boundary value of the pruning parameter range, thereby narrowing the pruning parameter range. The target pruning parameter for the next iteration is then determined from the narrowed pruning parameter range, and the next iteration is entered. For subsequent iterations, you can refer to the above description process and will not go into details here.
[0129] Optionally, the target pruning parameter is a middle value of the pruning parameter range before adjustment in the current adjustment process; sub-step 2052 may specifically include:
[0130] Sub-step 20521: When the ratio of the performance parameter of the first model to the target performance parameter is greater than the target acceleration factor, adjust the upper limit of the pruning parameter range to the target pruning parameter.
[0131] Sub-step 20522: When the ratio of the performance parameter of the first model to the target performance parameter is less than the target acceleration factor, adjust the lower limit of the pruning parameter range to the target pruning parameter.
[0132] In an embodiment of the present application, for sub-steps 20521-20522, the logic of adjusting and reducing the pruning parameter range in each round of iteration of the search algorithm can be specifically implemented by adjusting the upper limit value of the boundary of the pruning parameter range to the target pruning parameter when the ratio of the performance parameter of the first model to the target performance parameter is greater than the target acceleration multiple; and adjusting the lower limit value of the boundary of the pruning parameter range to the target pruning parameter when the ratio of the performance parameter of the first model to the target performance parameter is less than the target acceleration multiple. This method can directly use the target pruning parameter of the current iteration as one of the boundary values of the pruning parameter range, thereby quickly reducing the pruning parameter range.
[0133] 8 , which shows a schematic diagram of the change in the pruning parameter range in a search algorithm, wherein each iteration changes the boundary value of the pruning parameter range (changing the lower limit value low_r or changing the upper limit value high_r), narrowing the pruning parameter range. Until the i-th iteration, the pruning parameter range has been narrowed to a very small range. At this time, the target pruning parameter that meets the expected pruning target has not yet been determined, so a second pruning parameter can be conveniently determined from the very small pruning parameter range.
[0134] Optionally, sub-step 2053 may specifically include:
[0135] Sub-step 20531: After adjusting the boundary value of the pruning parameter range at least once, if the ratio of the target performance parameter to the performance parameter of the second model is equal to the target acceleration factor, determine the target pruning parameter corresponding to the target performance parameter as the second pruning parameter.
[0136] Alternatively, in sub-step 20532, after the process of adjusting the boundary value of the pruning parameter range is performed a target number of times, the second pruning parameter is determined according to the adjusted pruning parameter range.
[0137] In an embodiment of the present application, the search algorithm may have two iterative termination conditions. One iterative termination condition is that after a target number of iterations (e.g., 100), the second pruning parameter is determined directly based on the pruning parameter range adjusted after the last iteration. The other iterative termination condition is that after multiple iterations, if the ratio of the target performance parameter of the last iteration to the performance parameter of the second model is equal to the target speedup factor, the target pruning parameter corresponding to the target performance parameter is determined as the second pruning parameter.
[0138] It can be explained that the above-mentioned iterative enumeration method of the search algorithm of the embodiment of the present application is a binary search method, which can specifically search quickly with the efficiency of log2N to obtain the desired second pruning parameter, and the calculation complexity is O(log2N), which ensures that the present solution can obtain the desired model inference delay and its corresponding second pruning parameter within a logarithmic number of searches. The search algorithm of the present application uses performance parameters as feedback to perform iterative enumeration and search, which can ensure that the searched second pruning parameter does not deviate significantly from the expected pruning target. In addition, the search algorithm using binary search continuously updates the range of pruning parameters, so that the model can be compressed at different scales to obtain model objects of different sizes, and the model after pruning in the present application is not saved on the local disk but saved in the memory area for inference speed testing, so the search process also reduces the number of reads and writes, thereby reducing delay feedback.
[0139] Furthermore, other related technologies for implementing search algorithm functions include:
[0140] 1. Use reinforcement learning to search, using the performance parameters obtained by reasoning as the reward function and the pruning parameters used as input samples. Continuously learn the cost model. After training on a large number of samples, the cost model can predict the actual model inference delay based on the specified pruning parameters.
[0141] 2. Enumeration method: The program selects 1000 pruning parameters from 0-1 with intervals of 0.001. Each pruning parameter is tested for the performance parameters of the corresponding model, and the result closest to the target acceleration factor x is selected.
[0142] 3. Polynomial fitting method: By selecting 10 pruning parameters from 0-1 with intervals of 0.1, each pruning parameter is tested for the performance parameters of the corresponding model. Based on these data pairs, a linear polynomial or a quadratic polynomial is used to fit to obtain a specific polynomial expression. Based on the polynomial, the pruning parameters can be quickly input to obtain the corresponding pruning parameters, and the target pruning parameters can be directly predicted.
[0143] 4. Neural network prediction method: By selecting 10 pruning parameters from 0-1 with intervals of 0.1, each pruning parameter is tested on the corresponding model performance parameter. Based on these data pairs, they are input into a simple fully connected neural network for training, allowing this network to replace the polynomial and directly predict the pruning parameter obtained after any specific performance parameter is input.
[0144] Step 206: After pruning the structural layer of the first model according to the second pruning parameters to obtain a third model, if the performance parameters obtained by running the third model meet the preset conditions, output the third model for use.
[0145] This process can refer to the above step 104 and will not be described in detail here.
[0146] Optionally, step 206 may specifically include:
[0147] Sub-step 2061: If the performance parameters obtained by running the third model meet the preset conditions, the third model is saved as a disk file in a preset format in the device disk.
[0148] In an embodiment of the present application, if the performance parameters obtained by the third model meet the preset conditions, the third model can be saved in the device disk as a disk file in a preset format, thereby realizing persistent storage of the model file. Saving the third model as a file on the local disk means that the third model can be stored in the disk in the form of a specific local file, that is, the third model is relatively separated from the running program and saved as a file in binary format.
[0149] Optionally, the method may further include:
[0150] Step 207: Obtain target acceleration multiple and performance parameters of the first model;
[0151] Step 208: If the ratio of the performance parameter of the first model to the performance parameter of the second model is equal to the target acceleration factor, determine that the performance parameter of the second model meets the preset condition, and output the second model for use;
[0152] Step 209: When the ratio is not equal to the target acceleration multiple, determine that the performance parameter of the second model does not meet the preset condition.
[0153] In an embodiment of the present application, for steps 207-209, the performance parameters of the first model are known values. The ratio of the performance parameters of the first model to the performance parameters of the second model is used to reflect the speedup factor of the current pruned model. This ratio can be compared with the ideal target speedup factor to determine whether the current iteration meets the expected requirements. If the ratio of the performance parameters of the first model to the performance parameters of the second model is equal to the target speedup factor, it is determined that the performance parameters of the second model meet the preset conditions, and the second model is output for use. If the ratio is not equal to the target speedup factor, it is determined that the performance parameters of the second model do not meet the preset conditions, and the search algorithm is entered to search and determine the second pruning parameters.
[0154] Optionally, the method may further include:
[0155] Step 210: After pruning the structural layer of the first model according to the second pruning parameters to obtain a third model, if the performance parameters obtained by running the third model do not meet the preset conditions, new pruning parameters are obtained according to a preset search algorithm.
[0156] Step 211: Use the new pruning parameters to perform a pruning operation on the structural layer of the first model to obtain a new third model.
[0157] Step 212: When the performance parameters obtained by running the new third model meet the preset conditions, the new third model is output for use.
[0158] Step 213: If the performance parameters obtained by running the new third model do not meet the preset conditions, enter the step of obtaining new pruning parameters according to a preset search algorithm.
[0159] In an embodiment of the present application, for steps 210-213, the embodiment of the present application performs pruning and pruning judgment in a cyclic manner. When the performance parameters of the pruned model obtained after a certain pruning meet the preset conditions, the pruned model of this pruning can be output for use; and when the performance parameters of the pruned model obtained after a pruning do not meet the preset conditions, the search algorithm is further entered to search for new pruning parameters until the performance parameters of the pruned model pruned with the new pruning parameters meet the preset conditions, and the search is stopped and the pruned model is output for use.
[0160] It can be explained that the embodiment of the present application can start the training process of the pruned model after finally obtaining the pruned model that meets the expected pruning target, so as to ensure that the accuracy of the final pruned model is improved as much as possible to meet the accuracy requirements.
[0161] In summary, in the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model, and when the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs.
[0162] 9 , which shows a flowchart of the steps of a speech recognition method provided in an embodiment of the present application, including:
[0163] Step 301: Input an audio file into a third model to obtain a text recognition result of the audio file output by the third model.
[0164] Among them, the third model is obtained by processing the model's structural adjustment method.
[0165] The details of this process can be referred to the description of FIG2 in the above embodiment, and will not be described in detail here.
[0166] In summary, in the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model, and when the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs.
[0167] 10 , which shows a flowchart of the steps of a text translation method provided in an embodiment of the present application, including:
[0168] Step 401: Input the text recognition result of the audio file into a third model to obtain the translation result of the audio file output by the third model.
[0169] Among them, the third model is obtained by processing the model's structural adjustment method.
[0170] The details of this process can be found in the description of FIG3 in the above embodiment, and will not be described in detail here.
[0171] In summary, in the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model, and when the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs.
[0172] 11 , which shows a block diagram of a model structure adjustment device provided in an embodiment of the present application, including:
[0173] A first pruning module 501 is configured to perform a pruning operation on the structural layer of the first model according to a first pruning parameter to obtain a second model;
[0174] An operating module 502 is configured to operate the second model to obtain performance parameters of the second model;
[0175] A search module 503 is configured to obtain a second pruning parameter according to a preset search algorithm when the performance parameter of the second model does not meet a preset condition;
[0176] The second pruning module 504 is configured to prune the structural layer of the first model according to the second pruning parameters to obtain a third model, and output the third model for use if the performance parameters obtained by running the third model meet the preset conditions.
[0177] Optionally, the first pruning module 501 includes:
[0178] a pruning submodule, configured to perform a pruning operation on the first structural layer of the first model according to the first pruning parameter to obtain a pruned second structural layer;
[0179] The replacement submodule is used to replace the first structural layer of the first model with the second structural layer to obtain a second model.
[0180] Optionally, the pruning submodule includes:
[0181] a hierarchical unit, configured to divide the first structural layer of the first model layer by layer to obtain a plurality of model hierarchical modules;
[0182] A pruning unit is used to perform a pruning operation on the model-level module according to a first pruning parameter corresponding to the model-level module, and form the second structural layer with the pruned model-level modules.
[0183] Optionally, the pruning unit includes:
[0184] The replacement subunit is used to replace the corresponding model level module in the first structural layer with the pruned model level module when a pruned model level module is obtained, and obtain the second model after at least part of the model level modules in the first structural layer are replaced.
[0185] Optionally, the operation module 502 includes:
[0186] A storage submodule, configured to store the second model in a preset memory area;
[0187] The running submodule is used to run the second model in the preset memory area to obtain performance parameters of the second model.
[0188] Optionally, the device further comprises:
[0189] A parameter acquisition module, used to obtain the target acceleration multiple and the performance parameters of the first model;
[0190] a first judgment module, configured to, when a ratio of the performance parameter of the first model to the performance parameter of the second model is equal to the target acceleration factor, determine that the performance parameter of the second model meets a preset condition, and output the second model for use;
[0191] The second judgment module is used to determine that the performance parameter of the second model does not meet the preset condition when the ratio is not equal to the target acceleration multiple.
[0192] Optionally, the search module 503 includes:
[0193] A parameter acquisition submodule, configured to obtain a target acceleration factor, a first pruning parameter range, and performance parameters of the first model;
[0194] an adjustment submodule configured to adjust a boundary value of the pruning parameter range based on the search algorithm and a target pruning parameter determined by the pruning parameter range before adjustment in the current adjustment process, if a ratio of the performance parameter of the first model to a target performance parameter is not equal to the target speedup factor; the target performance parameter being a performance parameter of a model obtained by pruning the first model using the target pruning parameter;
[0195] The output submodule is configured to determine the second pruning parameter according to the adjusted pruning parameter range after adjusting the boundary value of the pruning parameter range at least once.
[0196] Optionally, the target pruning parameter is a middle value of a pruning parameter range before adjustment in the current adjustment process;
[0197] The adjustment submodule includes:
[0198] a first adjusting unit, configured to adjust an upper boundary value of the pruning parameter range to the target pruning parameter when a ratio of the performance parameter of the first model to the target performance parameter is greater than the target acceleration multiple;
[0199] The second adjustment unit is configured to adjust a lower limit of the pruning parameter range to the target pruning parameter when a ratio of the performance parameter of the first model to the target performance parameter is less than the target acceleration multiple.
[0200] Optionally, the device further comprises:
[0201] a cyclic search module, configured to, after pruning the structural layer of the first model according to the second pruning parameters to obtain a third model, obtain new pruning parameters according to a preset search algorithm if the performance parameters obtained by running the third model do not meet the preset conditions;
[0202] A loop pruning module is used to prune the structural layer of the first model using the new pruning parameters to obtain a new third model
[0203] an output module, configured to output the new third model for use if the performance parameters obtained by running the new third model meet the preset conditions;
[0204] The iterative module is used to enter the step of obtaining new pruning parameters according to a preset search algorithm when the performance parameters obtained by running the new third model do not meet the preset conditions.
[0205] In summary, in the process of pruning a model using pruning parameters, the embodiment of the present application can judge whether the model performance parameters meet the expected targets based on the performance parameters of the pruned model, and when the model performance parameters do not meet the expected targets, the computing power of the device and the search algorithm can be used to quickly and automatically search and determine new pruning parameters, and use pruning parameters that are more in line with the expected pruning targets to re-prune the original model to obtain a new model. The above process can be automatically executed in a loop until the output performance parameters meet the preset conditions for use. The embodiment of the present application uses the search algorithm to efficiently realize the continuous optimization and iteration of pruning parameters, realize the purpose of automated model pruning and pruning parameter readjustment, reduce dependence on human resources, improve pruning efficiency, and reduce pruning implementation costs.
[0206] An embodiment of the present application further provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.
[0207] The present application provides one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In the present application, the electronic device includes various types of devices such as terminal devices and server (cluster).
[0208] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as terminal devices, server terminals (clusters), etc. FIG12 schematically illustrates an exemplary device 1000 that can be used to implement various embodiments described in the embodiments of the present application.
[0209] For one embodiment, Figure 12 shows an exemplary apparatus 1000 having one or more processors 1002, a control module (chip set) 1004 coupled to at least one of the processor(s) 1002, a memory 1006 coupled to the control module 1004, a non-volatile memory (NVM) / storage device 1008 coupled to the control module 1004, one or more input / output devices 1010 coupled to the control module 1004, and a network interface 1012 coupled to the control module 1004.
[0210] The processor 1002 may include one or more single-core or multi-core processors, and the processor 1002 may include any combination of general-purpose processors or dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 1000 can serve as a terminal device, a server (cluster), or other device described in the embodiments of the present application.
[0211] In some embodiments, the apparatus 1000 may include one or more computer-readable media (e.g., memory 1006 or NVM / storage 1008) having instructions 1014 and one or more processors 1002 configured in conjunction with the one or more computer-readable media to execute the instructions 1014 to implement a module to perform the actions described in the present disclosure.
[0212] For one embodiment, the control module 1004 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1002 and / or any suitable device or component in communication with the control module 1004 .
[0213] The control module 1004 may include a memory controller module to provide an interface to the memory 1006. The memory controller module may be a hardware module, a software module, and / or a firmware module.
[0214] The memory 1006 can be used, for example, to load and store data and / or instructions 1014 for the device 1000. For one embodiment, the memory 1006 can include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1006 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).
[0215] For one embodiment, the control module 1004 may include one or more input / output controllers to provide an interface to the NVM / storage device 1008 and the input / output device(s) 1010 .
[0216] For example, NVM / storage 1008 may be used to store data and / or instructions 1014. NVM / storage 1008 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).
[0217] NVM / storage device 1008 may include storage resources that are physically part of the device on which apparatus 1000 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1008 may be accessible over a network via input / output device(s) 1010.
[0218] (One or more) input / output devices 1010 may provide an interface for apparatus 1000 to communicate with any other appropriate device. Input / output devices 1010 may include communication components, audio components, sensor components, etc. Network interface 1012 may provide an interface for apparatus 1000 to communicate via one or more networks. Apparatus 1000 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.
[0219] For one embodiment, at least one of the processor(s) 1002 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1004. For one embodiment, at least one of the processor(s) 1002 may be packaged together with the logic of one or more controllers of the control module 1004 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1002 may be integrated on the same die with the logic of one or more controllers of the control module 1004. For one embodiment, at least one of the processor(s) 1002 may be integrated on the same die with the logic of one or more controllers of the control module 1004 to form a system-on-chip (SoC).
[0220] In various embodiments, the apparatus 1000 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the apparatus 1000 may have more or fewer components and / or a different architecture. For example, in some embodiments, the apparatus 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.
[0221] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in the memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.
[0222] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0223] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other.
[0224] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, terminal device (system), and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.
[0225] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0226] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0227] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0228] Finally, it should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.
[0229] The above is a detailed introduction to the structural adjustment method, speech recognition method, text translation method, device, electronic device, and machine-readable medium of a model provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core idea of this application. Under the same circumstances, for general technical personnel in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.
Claims
1. A method for adjusting the structure of a model, characterized in that: include: Performing a pruning operation on the structural layer of the first model according to the first pruning parameter to obtain a second model; Running the second model to obtain performance parameters of the second model; When the performance parameter of the second model does not meet the preset condition, obtaining the second pruning parameter according to the preset search algorithm; After the structural layer of the first model is pruned according to the second pruning parameters to obtain the third model, if the performance parameters obtained by running the third model meet the preset conditions, the third model is output for use.
2. The method according to claim 1, characterized in that The step of performing a pruning operation on the structural layer of the first model according to the first pruning parameter to obtain the second model includes: Performing a pruning operation on the first structure layer of the first model according to the first pruning parameter to obtain a pruned second structure layer; The second structural layer replaces the first structural layer of the first model to obtain a second model.
3. The method according to claim 2, characterized in that The step of performing a pruning operation on the first structure layer of the first model according to the first pruning parameter to obtain a pruned second structure layer includes: Dividing the first structural layer of the first model layer by layer to obtain multiple model level modules; The model level module is pruned according to the first pruning parameter corresponding to the model level module, and the pruned model level module constitutes the second structural layer.
4. The method according to claim 3, characterized in that The step of replacing the first structural layer of the first model with the second structural layer to obtain the second model comprises: When a pruned model level module is obtained, the pruned model level module is used to replace the corresponding model level module in the first structural layer, and after at least part of the model level modules in the first structural layer are replaced, the second model is obtained.
5. The method according to claim 1, characterized in that The step of running the second model to obtain performance parameters of the second model includes: Storing the second model in a preset memory area; The second model is run in the preset memory area to obtain performance parameters of the second model.
6. The method according to claim 1, characterized in that The method further comprises: Obtaining a target acceleration multiple and performance parameters of the first model; When the ratio of the performance parameter of the first model to the performance parameter of the second model is equal to the target acceleration multiple, determining that the performance parameter of the second model meets a preset condition, and outputting the second model for use; When the ratio is not equal to the target acceleration multiple, it is determined that the performance parameter of the second model does not meet the preset condition.
7. The method according to claim 1, characterized in that The step of obtaining the second pruning parameter according to a preset search algorithm includes: Obtaining a target acceleration multiple, a first pruning parameter range, and performance parameters of the first model; In the case where the ratio of the performance parameter of the first model to the target performance parameter is not equal to the target acceleration multiple, adjusting the boundary value of the pruning parameter range according to the search algorithm and the target pruning parameter determined by the pruning parameter range before adjustment in the current adjustment process; the target performance parameter is the performance parameter of the model obtained after pruning the first model using the target pruning parameter; After adjusting the boundary value of the pruning parameter range at least once, the second pruning parameter is determined according to the adjusted pruning parameter range.
8. The method according to claim 7, characterized in that The target pruning parameter is the middle value of the pruning parameter range before adjustment in the current adjustment process; When the ratio of the performance parameter of the first model to the target performance parameter is not equal to the target acceleration multiple, adjusting the boundary value of the pruning parameter range according to the search algorithm and the target pruning parameter determined by the pruning parameter range before adjustment in the current adjustment process, includes: When the ratio of the performance parameter of the first model to the target performance parameter is greater than the target acceleration multiple, adjusting the upper limit value of the boundary of the pruning parameter range to the target pruning parameter; When the ratio of the performance parameter of the first model to the target performance parameter is less than the target acceleration multiple, the lower limit of the boundary of the pruning parameter range is adjusted to the target pruning parameter.
9. The method according to claim 1, characterized in that: The method further comprises: After performing a pruning operation on the structural layer of the first model according to the second pruning parameter to obtain a third model, if the performance parameter obtained by running the third model does not meet the preset condition, obtaining a new pruning parameter according to a preset search algorithm; Using the new pruning parameters, pruning is performed on the structural layer of the first model to obtain a new third model. When the performance parameter obtained by running the new third model reaches the preset condition, outputting the new third model for use; When the performance parameters obtained by running the new third model do not meet the preset conditions, the step of obtaining new pruning parameters according to a preset search algorithm is entered.
10. A speech recognition method, characterized in that: include: Inputting the audio file into the third model, and obtaining the text recognition result of the audio file output by the third model; The third model is obtained by processing by the method according to any one of claims 1 to 9.
11. A text translation method, characterized in that: include: Inputting the text recognition result of the audio file into a third model to obtain the translation result of the audio file output by the third model; The third model is obtained by processing by the method according to any one of claims 1 to 9.
12. A structural adjustment device for a model, characterized in that: include: A first pruning module, used for performing a pruning operation on the structural layer of the first model according to a first pruning parameter to obtain a second model; An operating module, used for operating the second model to obtain performance parameters of the second model; A search module, configured to obtain a second pruning parameter according to a preset search algorithm when the performance parameter of the second model does not meet a preset condition; The second pruning module is used to perform a pruning operation on the structural layer of the first model according to the second pruning parameters to obtain a third model, and if the performance parameters obtained by running the third model meet the preset conditions, output the third model for use.
13. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when the executable codes are executed, enable the processor to perform the method according to any one of claims 1 to 9.
14. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to perform the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Pruning method and device for semantic comprehension model, electronic equipment and storage medium
CN111539224A
Model automatic optimization method and device, equipment and storage medium
CN112906853A
Model structure adjusting method and device
CN116629341A
Applying a structured language model to information extraction
US20030216905A1
Cited By
Method and system for optimizing smelting efficiency in aluminum machining process in real time
CN120496692A
A method and system for real-time optimization of smelting efficiency in aluminum processing
CN120496692B
