An application method and device of image processing network model parameter pruning and medium

Abstract

The application discloses an application method and device of image processing network model parameter pruning and medium. The image processing network model corresponding to the application software to be installed in the target embedded terminal device is acquired and analyzed, and image model description information is obtained by analysis; the standard calculation storage corresponding to the target embedded terminal device is acquired, and if the image model running calculation storage in the image model description information is greater than the standard calculation storage, the image processing network model to be pruned is determined, and the image processing network model is pruned according to the pre-constructed image model parameter pruning method to obtain the image processing network pruning model. The problem that the image processing network model installed in the application software of the target embedded terminal device fails due to too large calculation storage is solved, the image processing network model is pruned, and the correct installation of the image processing network model in the target embedded terminal device is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an application method, apparatus, and medium for cropping parameters of an image processing network model. Background Technology

[0002] With the widespread development of deep learning technology, deep neural networks (DNNs) have demonstrated outstanding performance in many fields, including computer vision, natural language processing, speech recognition, and image processing. However, because image processing neural networks contain a large number of neurons (convolutional kernels contain many parameters), their structure is relatively complex and the number of parameters is large, resulting in significant computational, storage, and time costs during model training and inference.

[0003] In the process of realizing this invention, the inventors discovered the following defects in the prior art: Currently, image processing neural networks are generally pruned by weight pruning, channel pruning, or structure pruning, but this leads to the loss of parameters with higher weights and the loss of parameter dimensions, resulting in deviations in the image processing neural network and low accuracy. Summary of the Invention

[0004] This invention provides an application method, apparatus, and medium for cropping parameters of an image processing network model, so as to crop the parameters of the image processing network model and to ensure the correct installation of the image processing network model on the target embedded terminal device.

[0005] According to one aspect of the present invention, a method for cropping parameters of an image processing network model is provided, comprising:

[0006] Acquire and parse the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and parse the image model description information corresponding to the image processing network model;

[0007] Obtain the standard computing storage capacity corresponding to the target embedded terminal device. If the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then the image processing network model is determined as the image processing network model to be clipped.

[0008] The image processing network model to be cropped is cropped according to a pre-built image model parameter cropping method to obtain an image processing network cropping model, so as to ensure that the image processing network cropping model is correctly installed on the target embedded terminal device.

[0009] According to another aspect of the present invention, an application apparatus for cropping parameters of an image processing network model is provided, comprising:

[0010] The image model description information parsing module is used to acquire and parse the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and to obtain the image model description information corresponding to the image processing network model.

[0011] The image processing network model determination module is used to obtain the standard computing storage volume corresponding to the target embedded terminal device. If the computing storage volume of the image model in the image model description information is greater than the standard computing storage volume, then the image processing network model is determined as the image processing network model to be cropped.

[0012] The image processing network cropping model determination module is used to crop the image processing network model to be cropped according to a pre-built image model parameter cropping method, and determine the image processing network cropping model so as to ensure that the image processing network cropping model is correctly installed on the target embedded terminal device.

[0013] According to another aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the application method for image processing network model parameter clipping as described in any embodiment of the present invention.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the application method for image processing network model parameter clipping as described in any embodiment of the present invention.

[0015] The technical solution of this invention obtains and parses the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and obtains the image model description information corresponding to the image processing network model; obtains the standard computing storage capacity corresponding to the target embedded terminal device; if the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then the image processing network model is determined to be an image processing network model to be parameterized; the image processing network model to be parameterized is parameterized according to a pre-built image model parameter pruning method to obtain a pruned image processing network model, thereby achieving correct installation of the pruned image processing network model on the target embedded terminal device. This solves the problem of installation failure of the image processing network model in the application software of the target embedded terminal device due to excessive computing storage capacity. By pruning the parameters of the image processing network model, correct installation of the image processing network model on the target embedded terminal device is achieved.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1a This is a flowchart of an application method for cropping image processing network model parameters according to Embodiment 1 of the present invention;

[0019] Figure 1b This is a schematic diagram of the structure of the parameter clipping function in the method provided according to Embodiment 1 of the present invention;

[0020] Figure 2 This is a schematic diagram of the structure of an application device for cropping image processing network model parameters according to Embodiment 2 of the present invention;

[0021] Figure 3 This is a schematic diagram of the structure of an electronic device provided according to Embodiment 3 of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "target," "current," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1

[0025] Figure 1a The flowchart of an application method for parameter cropping of an image processing network model provided in Embodiment 1 of the present invention is applicable to the case of parameter cropping of an image processing network model with too large a computational and storage requirements. The method can be executed by an application device for parameter cropping of an image processing network model, which can be implemented in hardware and / or software.

[0026] Correspondingly, such as Figure 1a As shown, the method includes:

[0027] S110. Obtain and parse the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and parse the image model description information corresponding to the image processing network model.

[0028] Image processing network models are network models capable of performing image processing. Different image processing network models typically have a different number of convolutional layers, and the number of parameters in each convolutional layer also varies. For example, fully connected layers in an image processing network model have more parameters, while the first or second layer has relatively fewer parameters.

[0029] Specifically, for fully connected layers with a large number of parameters, a large number of parameters can be adaptively pruned; for convolutional layers with a small number of parameters, a small range of parameters can be adaptively pruned. However, parameter pruning must ensure the accuracy of the image processing network model.

[0030] The image model description information can be information that describes the parameters of the image processing network model. For example, the image model description information may include the computational and storage requirements of the image model, the convergence threshold of the image model, and the accuracy of the image model.

[0031] In this embodiment, it can be applied to embedded terminal devices. Generally, embedded terminal devices have relatively low computational and storage requirements. For example, if the embedded terminal device is a mobile phone, and an image processing network model for a certain app needs to be installed on the phone, due to the limited computational and storage capacity of the phone, the parameters of each layer of the image processing network model need to be pruned while ensuring accuracy. Furthermore, because the number of parameters in each layer of the image processing network model is different, corresponding sparsity values ​​need to be matched to the number of parameters in different layers.

[0032] S120. Obtain the standard computing storage capacity corresponding to the target embedded terminal device. If the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then determine the image processing network model as the image processing network model to be clipped.

[0033] The standard computational storage requirement can be a threshold value corresponding to the computational and storage requirements of the target embedded terminal device. The computational storage requirement for image model execution can be a value matching the computational and storage requirements of the image processing network model to be installed.

[0034] In this embodiment, it is necessary to obtain the standard computing storage capacity corresponding to the target embedded terminal device, and the image model runtime computing storage capacity in the image model description information. After obtaining the standard computing storage capacity and the image model runtime computing storage capacity, the two need to be compared. If the image model runtime computing storage capacity is greater than the standard computing storage capacity, it indicates that the current image processing network model has a relatively large computing storage capacity and cannot be well installed in the target embedded terminal device. Therefore, it is necessary to perform parameter pruning on the current image processing network model, thus identifying the image processing network model to be parameterized.

[0035] Furthermore, if the computational storage required for the image model is no greater than the standard computational storage required, it indicates that the computational storage of the current image processing network model can be well installed on the target embedded terminal device. Therefore, the current image processing network model does not require cropping and can be directly installed on the target embedded terminal device.

[0036] S130. The image processing network model to be cropped is cropped according to the pre-built image model parameter cropping method to obtain the image processing network cropping model, so as to realize the correct installation of the image processing network cropping model on the target embedded terminal device.

[0037] The image model parameter cropping method can be a method for cropping the parameters of an image processing network model. Specifically, it can perform parameter cropping operations on each convolutional layer of the image processing network model. The image processing network cropped model can be the model obtained after cropping the parameters of the image processing network model to be cropped.

[0038] In this embodiment, the image processing network model to be cropped needs to be cropped according to a preset image model parameter cropping method. This allows for the cropping of the image processing network model, resulting in a cropped image processing network model. The computational and storage requirements of this cropped model are smaller than those of the original image processing network model. Therefore, the cropped image processing network model can be correctly installed on the target embedded terminal device, meaning the computational and storage requirements for running the cropped model are no greater than the standard computational and storage requirements.

[0039] Optionally, the image model description information further includes an image model convergence threshold and an image model accuracy; the step of cropping the image processing network model to be cropped according to a pre-built image model parameter cropping method to obtain an image processing network cropping model includes: parsing the image processing network model to be cropped to obtain the number of parameters in each layer; determining the final sparsity and the number of parameter cropping steps of the image model based on the image model convergence threshold and the image model accuracy, and determining the sparsity value based on the number of parameters in each layer; determining the parameter cropping function based on the final sparsity, the number of parameter cropping steps, and the sparsity value; and cropping the image processing network model to be cropped according to the parameter cropping function to obtain the image processing network cropping model.

[0040] Here, the image model convergence threshold can be the size of the threshold describing the final convergence of the image processing network model. The image model accuracy can be the size describing the fineness of the image processing network model. The final sparsity of the image model can be the size of the sparsity corresponding to the image model convergence threshold; specifically, the final sparsity of the image model can correspond to the size of the convergence threshold of the parameter clipping function.

[0041] The number of parameter cropping iterations can be the number of times the parameters of the image processing network model to be cropped are applied. Alternatively, the number of parameter cropping iterations can be determined by the accuracy of the image model. The sparsity value can be determined based on the number of parameters in each convolutional layer.

[0042] Specifically, generally speaking, the larger the number of parameters in each convolutional layer, the greater the corresponding sparsity value; conversely, the smaller the number of parameters in each convolutional layer, the smaller the corresponding sparsity value. This is because for convolutional layers with a large number of parameters, the number of parameters that can be pruned is also relatively large, thus corresponding to a larger sparsity value. However, it is necessary to ensure that the key feature parameters of each convolutional layer are preserved, thereby guaranteeing the accuracy of the image processing network model.

[0043] The parameter clipping function can be a function that clips the parameters of an image processing network model. This parameter clipping function is determined based on the final sparsity of the image model, the number of parameter clipping iterations, and the sparsity value. Generally, the parameter clipping function is an exponential-like function.

[0044] The advantage of this setup is that by obtaining the image model description information corresponding to the image processing network model, the final sparsity, parameter clipping times, and sparsity value of the image model can be determined. This further determines the parameter clipping function, and then the image processing network model is clipped. As a result, the image processing network clipping model can be better adapted to the target embedded terminal device, thus achieving correct installation of the image processing network clipping model on the target embedded terminal device.

[0045] Optionally, determining the sparsity value based on the number of parameters in each layer includes: obtaining the number of parameters in each layer corresponding to the image processing network model; sequentially matching the number of parameters in each layer with a pre-built parameter quantity sparsity value mapping table to determine the sparsity value corresponding to the number of parameters in each layer.

[0046] The parameter quantity and sparsity value mapping table can be a mapping table that describes the matching relationship between the number of parameters and sparsity values ​​in each convolutional layer.

[0047] In this embodiment, it is necessary to obtain in advance the historical number of parameters for different convolutional layers and the historical sparsity values ​​that match the historical number of parameters, and then construct a parameter number sparsity value mapping table based on the historical number of parameters and the historical sparsity values.

[0048] Furthermore, after constructing the parameter quantity sparsity value mapping table, when the number of parameters corresponding to each convolutional layer is obtained, the size of the sparsity value matching the number of parameters can be determined.

[0049] The advantage of this setup is that by matching different sparsity values ​​to the different number of parameters corresponding to each convolutional layer, the determined sparsity values ​​are more accurate, thereby ensuring more reasonable cropping of the image processing network model.

[0050] Optionally, the parameter clipping function is an exponential function-like function; determining the parameter clipping function based on the final sparsity of the image model, the number of parameter clippings, and the sparsity value includes: matching the final sparsity of the image model, the number of parameter clippings, and the sparsity value with a preset clipping description parameter exponential function matching table to determine the target exponential function, and determining the target exponential function as the parameter clipping function.

[0051] The cropping description parameter exponential function matching table can be a table describing the matching relationship between the cropping description parameters and exponential functions of an image processing network model. The target exponential function can be a function with an explicit exponent.

[0052] In this embodiment, it is necessary to obtain the historical cropping description parameters of the historical image processing network model in advance, namely the final sparsity of the historical image model, the number of historical parameter croppings and the historical sparsity value, as well as the historical exponential function. Based on the above historical parameters, the cropping description parameter exponential function matching table is constructed.

[0053] Specifically, the exponential function is Here, n is a positive number. The final sparsity of the image model, the number of parameter clippings, and the magnitude of the sparsity value need to be matched with a pre-defined clipping description parameter exponential function matching table to further determine the value of n, thereby identifying the target exponential function.

[0054] like Figure 1b The diagram shown illustrates the structure of the parametric cropping function. Here, 0.9 represents the convergence value of the parametric cropping function, which is also the final sparsity of the historical image model. Figure 1b In this context, the historical parameters were pruned 17 times. Figure 1b The value of n corresponding to the median parameter clipping function is 2, that is...

[0055] The advantage of this setting is that the determined exponential function can better prune the parameters of each convolutional layer of the image processing network model, thereby reducing the computational and storage requirements of the image processing network model and making it more suitable for embedded terminal devices.

[0056] Optionally, the step of performing parameter cropping on the image processing network model to be cropped according to the parameter cropping function to obtain the image processing network cropping model includes: performing parameter cropping on the image processing network model to be cropped according to the parameter cropping function, and counting the current number of parameter croppings; determining whether the current number of croppings has reached the number of parameter croppings; if so, detecting the accuracy of the current image processing network model, and determining the image processing network cropping model based on the accuracy detection result; if not, returning to the operation of performing parameter cropping on the image processing network model according to the parameter cropping function, until the number of parameter croppings has been traversed.

[0057] The accuracy detection result can be the magnitude of the accuracy detection result that matches the image processing network model after parameter cropping.

[0058] In this embodiment, after parameter cropping of the image processing network model, it is necessary to count the current number of parameter cropping iterations to determine whether the parameter cropping iteration has been completed. If the iteration is complete, the accuracy of the current image processing network model needs to be verified. If the accuracy detection result meets the requirements, it can be determined that the cropping of the image processing network model has been completed. If the parameter cropping iteration has not been completed, the parameters of the image processing network model need to be cropped again.

[0059] Optionally, the step of detecting the accuracy of the current image processing network model and determining the image processing network cropping model based on the accuracy detection result includes: detecting the accuracy of the current image processing network model and obtaining the accuracy detection result; obtaining the standard accuracy detection result corresponding to the image processing network model to be cropped; determining whether the accuracy detection result is within the error allowable range threshold corresponding to the standard accuracy detection result; if so, determining the current image processing network model as the image processing network cropping model; if not, discarding the current image processing network model and feeding back an instruction that the image processing network model parameter cropping failed.

[0060] The standard accuracy test result can be the magnitude of the accuracy of a pre-set image processing network model.

[0061] In this embodiment, the accuracy detection result needs to be compared with the standard accuracy detection result. Assume the accuracy detection result is 94%; the standard accuracy detection result is 95%, and the error tolerance threshold is 0–0.5%, meaning the accuracy detection result needs to be between 94.5% and 95.5%. Since the accuracy detection result is 94%, the current image processing network model is discarded, and a message indicating that the image processing network model parameter cropping failed is sent.

[0062] Optionally, after performing parameter cropping on the image processing network model to be cropped according to the parameter cropping function and counting the current parameter cropping count, the method further includes: obtaining the model retraining frequency corresponding to the image processing network model, and calculating the model retraining count and the parameter cropping count sorting number matching the model retraining count based on the parameter cropping count; determining whether the current parameter cropping count matches the parameter cropping count sorting number; if so, retraining the image processing network model after parameter cropping using the obtained training parameter dataset and updating the model retraining count; after the image processing network model is retrained, performing parameter cropping on the retrained image processing network model according to the parameter cropping function, counting the current parameter cropping count, and returning to perform the operation of determining whether the current parameter cropping count matches the parameter cropping count sorting number, until the model retraining count is traversed.

[0063] Here, the model retraining frequency can be a pre-set frequency for model retraining. The number of model retraining iterations can be the number of times the model needs to be retrained. The parameter pruning iterations sorting number can be the sorting number corresponding to the number of model retraining iterations.

[0064] In this embodiment, it is necessary to match the current parameter cropping count with the parameter cropping count sorting number. If a match is found, the current image processing network model needs to be retrained. After retraining, parameter cropping is performed again until all retraining counts of the model have been traversed.

[0065] Optionally, the step of obtaining the model retraining frequency corresponding to the image processing network model, and calculating the model retraining count and the parameter cropping count ranking number matching the model retraining count based on the parameter cropping count, includes: obtaining the model retraining frequency and the parameter cropping count, and calculating the model retraining count and the parameter cropping count according to the formula... The number of model retraining attempts M is calculated; where N represents the number of parameter pruning attempts; f represents the model retraining frequency; M is a positive integer, and the quotient of (N-1) / f is taken; the sorting number of parameter pruning attempts is determined based on the number of model retraining attempts and the number of parameter pruning attempts.

[0066] In this embodiment, it is assumed that the model retraining frequency is 3 times per cycle (that is, each parameter is pruned 3 times, and the model is trained once). It is assumed that the number of parameter pruning cycles is 17.

[0067] Specifically, the model can be retrained 5 times, which corresponds to the parameter cropping count being 4, 7, 10, 13, or 16. If the current parameter cropping count matches the parameter cropping count, the image processing parameter model needs to be retrained before parameter cropping can be performed.

[0068] The advantage of this setup is that by cropping the parameters of the image processing network model, retraining the model, and then cropping the parameters again, the accuracy of the image processing network model can be better ensured, rather than simply cropping the parameters and causing the accuracy of the image processing network model to drop.

[0069] Preferably, an image processing method is executed by a target embedded terminal device having an image processing network model, comprising: inputting an image to be processed into the image processing network model in the target embedded terminal device, the image processing network model performing image processing on the image to be processed to obtain a target processed image, and displaying the target processed image in the target embedded terminal device to provide feedback to a user, or transmitting it to a storage device in the target embedded terminal device.

[0070] In this embodiment, the input of the image processing network model is the image to be processed, and the output is the target image to be processed. The image processing network model is obtained by applying the parameter cropping method of the image processing network model.

[0071] The technical solution of this invention obtains and parses the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and obtains the image model description information corresponding to the image processing network model; obtains the standard computing storage capacity corresponding to the target embedded terminal device; if the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then the image processing network model is determined to be an image processing network model to be parameterized; the image processing network model to be parameterized is parameterized according to a pre-built image model parameter pruning method to obtain a pruned image processing network model, thereby achieving correct installation of the pruned image processing network model on the target embedded terminal device. This solves the problem of installation failure of the image processing network model in the application software of the target embedded terminal device due to excessive computing storage capacity. By pruning the parameters of the image processing network model, correct installation of the image processing network model on the target embedded terminal device is achieved.

[0072] Example 2

[0073] Figure 2This is a schematic diagram of an application device for cropping image processing network model parameters according to Embodiment 2 of the present invention. The application device for cropping image processing network model parameters provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal device, especially an embedded terminal device, to implement the application method for cropping image processing network model parameters according to the embodiments of the present invention. Figure 2 As shown, the device includes: an image model description information parsing module 210, an image processing network model determination module 220 for parameter cropping, and an image processing network cropping model determination module 230.

[0074] The image model description information parsing module 210 is used to acquire and parse the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and to obtain the image model description information corresponding to the image processing network model.

[0075] The image processing network model determination module 220 is used to obtain the standard computing storage volume corresponding to the target embedded terminal device. If the computing storage volume of the image model in the image model description information is greater than the standard computing storage volume, then the image processing network model is determined as the image processing network model to be cropped.

[0076] The image processing network cropping model determination module 230 is used to perform parameter cropping on the image processing network model to be cropped according to a pre-built image model parameter cropping method, and determine the image processing network cropping model so as to ensure that the image processing network cropping model is correctly installed on the target embedded terminal device.

[0077] The technical solution of this invention obtains and parses the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and obtains the image model description information corresponding to the image processing network model; obtains the standard computing storage capacity corresponding to the target embedded terminal device; if the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then the image processing network model is determined to be an image processing network model to be parameterized; the image processing network model to be parameterized is parameterized according to a pre-built image model parameter pruning method to obtain a pruned image processing network model, thereby achieving correct installation of the pruned image processing network model on the target embedded terminal device. This solves the problem of installation failure of the image processing network model in the application software of the target embedded terminal device due to excessive computing storage capacity. By pruning the parameters of the image processing network model, correct installation of the image processing network model on the target embedded terminal device is achieved.

[0078] Optionally, the image model description information may also include the image model convergence threshold and the image model accuracy.

[0079] Optionally, the image processing network cropping model determination module 230 can be specifically used to: parse the image processing network model to be cropped, and obtain the number of parameters in each layer; determine the final sparsity and the number of parameter cropping steps of the image model based on the image model convergence threshold and image model accuracy, and determine the sparsity value based on the number of parameters in each layer; determine the parameter cropping function based on the final sparsity, the number of parameter cropping steps, and the sparsity value of the image model; and perform parameter cropping on the image processing network model to be cropped according to the parameter cropping function to obtain the image processing network cropping model.

[0080] Optionally, the image processing network cropping model determination module 230 can also be specifically used to: obtain the number of parameters of each layer corresponding to the image processing network model; sequentially match the number of parameters of each layer with a pre-built parameter sparsity value mapping table, and determine the sparsity value corresponding to the number of parameters of each layer respectively.

[0081] Optionally, the parameter pruning function is an exponential function.

[0082] Optionally, the image processing network cropping model determination module 230 can also be specifically used to: match the final sparsity of the image model, the number of parameter croppings, and the sparsity value with a preset cropping description parameter exponentiation function matching table to determine the target exponentiation function, and determine the target exponentiation function as the parameter cropping function.

[0083] Optionally, the image processing network cropping model determination module 230 can also be specifically used to: perform parameter cropping on the image processing network model to be cropped according to the parameter cropping function, and count the current parameter cropping count; determine whether the current cropping count has reached the parameter cropping count, and if so, detect the accuracy corresponding to the current image processing network model, and determine the image processing network cropping model according to the accuracy detection result; if not, return to execute the operation of performing parameter cropping on the image processing network model according to the parameter cropping function, until the parameter cropping count is completed.

[0084] Optionally, the image processing network cropping model determination module 230 can also be specifically used for: detecting the accuracy of the current image processing network model and obtaining the accuracy detection result; obtaining the standard accuracy detection result corresponding to the image processing network model to be cropped; determining whether the accuracy detection result is within the error allowable range threshold corresponding to the standard accuracy detection result; if so, determining the current image processing network model as the image processing network cropping model; if not, discarding the current image processing network model and feeding back an instruction that the image processing network model parameter cropping failed.

[0085] Optionally, the image processing network cropping model determination module 230 can also be specifically used for: after cropping the image processing network model to be cropped according to the parameter cropping function and counting the current parameter cropping count, obtaining the model retraining frequency corresponding to the image processing network model, and calculating the model retraining count and the parameter cropping count sorting number matching the model retraining count according to the parameter cropping count; determining whether the current parameter cropping count matches the parameter cropping count sorting number, and if so, retraining the image processing network model after the current parameter cropping using the obtained training parameter dataset, and updating the model retraining count; after the image processing network model is retrained, cropping the image processing network model after retraining according to the parameter cropping function, counting the current parameter cropping count, and returning to execute the operation of determining whether the current parameter cropping count matches the parameter cropping count sorting number, until the model retraining count is traversed.

[0086] Optionally, the image processing network cropping model determination module 230 can also be specifically used to: obtain the model retraining frequency and the number of parameter cropping times, and determine the cropping parameters according to the formula. The number of model retraining attempts M is calculated; where N represents the number of parameter pruning attempts; f represents the model retraining frequency; M is a positive integer, and the quotient of (N-1) / f is taken; the sorting number of parameter pruning attempts is determined based on the number of model retraining attempts and the number of parameter pruning attempts.

[0087] The image processing network model parameter clipping application device provided in the embodiments of the present invention can execute the image processing network model parameter clipping application method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0088] Example 3

[0089] Figure 3A schematic diagram of an electronic device 10, which can be used to implement Embodiment 3 of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0090] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0091] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0092] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the application method of image processing network model parameter pruning.

[0093] In some embodiments, the method for applying image processing network model parameter clipping can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for applying image processing network model parameter clipping described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method for applying image processing network model parameter clipping by any other suitable means (e.g., by means of firmware).

[0094] The method includes: acquiring and parsing the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and parsing the image model description information corresponding to the image processing network model; acquiring the standard computing storage capacity corresponding to the target embedded terminal device, and if the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then the image processing network model is determined as an image processing network model to be parameterized; performing parameter pruning on the image processing network model to be parameterized according to a pre-built image model parameter pruning method to determine an image processing network pruning model, so as to achieve correct installation of the image processing network pruning model on the target embedded terminal device.

[0095] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0099] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0100] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0101] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0103] Example 4

[0104] Embodiment 4 of the present invention also provides a computer-readable storage medium, wherein the computer-readable instructions, when executed by a computer processor, are used to perform an application method for parameter clipping of an image processing network model. The method includes: acquiring and parsing an image processing network model corresponding to an application software to be installed on a target embedded terminal device; parsing to obtain image model description information corresponding to the image processing network model; acquiring the standard computing storage capacity corresponding to the target embedded terminal device; if the computing storage capacity of the image model in the image model description information is greater than the standard computing storage capacity, then determining the image processing network model as the image processing network model to be parameter clipped; and clipping the image processing network model to be parameter clipped according to a pre-constructed image model parameter clipping method to obtain a clipped image processing network model, thereby enabling the correct installation of the image processing network model on the target embedded terminal device.

[0105] Alternatively, the method may include: receiving slave control commands and command conversion coordinates; mapping the command conversion coordinates using a preset coordinate mapping method to obtain command response content corresponding to the command conversion coordinates; and sending the command response content to the host system to update and display the command response content fed back by the slave system in the slave display window.

[0106] Of course, the computer-executable instructions provided in the embodiments of the present invention, which include a computer-readable storage medium, are not limited to the method operations described above, but can also perform related operations in the application method for image processing network model parameter clipping provided in any embodiment of the present invention.

[0107] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0108] It is worth noting that in the embodiments of the above-mentioned image processing network model parameter clipping application device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0109] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for cropping parameters of an image processing network model, characterized in that, include: Acquire and parse the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and parse the image model description information corresponding to the image processing network model; Obtain the standard computing storage volume corresponding to the target embedded terminal device. If it is determined that the computing storage volume of the image model running corresponding to the image model description information is greater than the standard computing storage volume, then the image processing network model is determined as the image processing network model to be clipped. The image processing network model to be cropped is cropped according to a pre-built image model parameter cropping method to obtain an image processing network cropping model, so as to ensure that the image processing network cropping model is correctly installed on the target embedded terminal device.

2. The method according to claim 1, characterized in that, The image model description information also includes the image model convergence threshold and the image model accuracy; The step of cropping the image processing network model to be cropped according to a pre-built image model parameter cropping method to obtain the image processing network cropping model includes: The image processing network model to be cropped is analyzed to obtain the number of parameters in each layer; The final sparsity and parameter clipping number of the image model are determined based on the convergence threshold and accuracy of the image model, and the sparsity value is determined according to the number of parameters in each layer. The parameter clipping function is determined based on the final sparsity of the image model, the number of parameter clipping steps, and the sparsity value. The image processing network model to be cropped is subjected to parameter cropping according to the parameter cropping function to obtain the image processing network cropping model.

3. The method according to claim 2, characterized in that, The process of determining the sparsity value based on the number of parameters in each layer includes: Obtain the number of parameters for each layer of the image processing network model; The number of parameters in each layer is matched with a pre-built parameter sparsity value mapping table in turn, and the sparsity value corresponding to the number of parameters in each layer is determined respectively.

4. The method according to claim 3, characterized in that, The parameter clipping function is an exponential function; The step of determining the parameter clipping function based on the final sparsity of the image model, the number of parameter clipping iterations, and the sparsity value includes: Based on the final sparsity of the image model, the number of parameter clippings, and the sparsity value, a matching table of preset clipping description parameter exponentiation functions is used to determine the target exponentiation function, and the target exponentiation function is determined as the parameter clipping function.

5. The method according to claim 4, characterized in that, The step of performing parameter cropping on the image processing network model to be cropped according to the parameter cropping function to obtain the image processing network cropping model includes: The image processing network model to be cropped is subjected to parameter cropping according to the parameter cropping function, and the current number of parameter croppings is counted. Determine whether the current parameter cropping count has reached the parameter cropping count. If so, detect the accuracy of the current image processing network model and determine the image processing network cropping model based on the accuracy detection result. If not, then return to the operation of cropping the image processing network model according to the parameter cropping function, until the parameter cropping count is completed.

6. The method according to claim 5, characterized in that, The step of detecting the accuracy of the current image processing network model and determining the image processing network cropping model based on the accuracy detection results includes: The accuracy of the current image processing network model is detected, and the accuracy detection result is obtained. Obtain the standard accuracy detection results corresponding to the image processing network model to be cropped; Determine whether the accuracy detection result is within the error allowable range threshold corresponding to the standard accuracy detection result. If so, determine the current image processing network model as the image processing network cropping model. If not, the current image processing network model is discarded, and an instruction indicating that the image processing network model parameter cropping failed is sent.

7. The method according to claim 5, characterized in that, After performing parameter cropping on the image processing network model to be cropped according to the parameter cropping function and counting the current number of parameter croppings, the method further includes: Obtain the model retraining frequency corresponding to the image processing network model, and calculate the model retraining count and the parameter cropping count sorting number that matches the model retraining count based on the parameter cropping count. Determine whether the current parameter cropping count matches the parameter cropping count sorting number. If so, retrain the image processing network model after current parameter cropping using the obtained training parameter dataset, and update the model retraining count. After the image processing network model is retrained, the parameters of the retrained image processing network model are cropped according to the parameter cropping function, the current parameter cropping count is counted, and the operation of judging whether the current parameter cropping count matches the parameter cropping count sorting number is performed is returned until the model retraining count is completed.

8. The method according to claim 7, characterized in that, The step of obtaining the model retraining frequency corresponding to the image processing network model, and calculating the model retraining count and the sorting number of parameter cropping counts matching the model retraining count based on the parameter cropping count, includes: Obtain the model retraining frequency and the number of parameter pruning times, and then apply the formula... The number of times the model needs to be retrained, M, is calculated. Where N represents the number of parameter pruning operations; f represents the model retraining frequency; and M is a positive integer, taking values ​​from 0 to 1. The business; The sorting number of parameter pruning times is determined based on the number of times the model is retrained and the number of times the parameters are pruned.

9. An application device for cropping parameters of an image processing network model, characterized in that, include: The image model description information parsing module is used to acquire and parse the image processing network model corresponding to the application software to be installed on the target embedded terminal device, and to obtain the image model description information corresponding to the image processing network model. The image processing network model determination module is used to obtain the standard computing storage volume corresponding to the target embedded terminal device. If it is determined that the computing storage volume of the image model corresponding to the image model description information is greater than the standard computing storage volume, then the image processing network model is determined as the image processing network model to be cropped. The image processing network cropping model determination module is used to crop the image processing network model to be cropped according to a pre-built image model parameter cropping method, and determine the image processing network cropping model so as to ensure that the image processing network cropping model is correctly installed on the target embedded terminal device.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the application method for cropping image processing network model parameters as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the application method for cropping image processing network model parameters as described in any one of claims 1-8.

Images