Design method, device and equipment based on deep learning general task framework model

Abstract

The application discloses a design method and device of a general task framework model based on deep learning, equipment and a storage medium, which comprises the following steps: constructing a general task framework model, wherein the general task framework model comprises an iterative task model and an index model; performing forward inference operation on input data inputted through the iterative task model to obtain a target result; performing forward inference operation on feature parameters outputted by the iterative task model as input through the index model to obtain an index result; and judging whether the iterative task model takes the target result as input to perform forward inference operation in the next round of iteration according to the index result. The application can flexibly ensure the best balance between computing power and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model design, in particular to a design method, device and equipment of a general task framework model based on deep learning. BACKGROUND

[0002] At present, in the field of artificial intelligence, the research and achievements in the aspect of deep learning are absolutely dominant, and at the same time, a large number of artificial intelligence companies are born. With the continuous expansion of application scenarios, various demand tasks emerge in an endless stream, but the requirements of software and hardware of devices are different in each task scenario, and the computing power and performance are also different for different people. At the same time, we know that the model of deep learning generally consumes a lot of computing power. The large model (referring to a large number of parameters) has good recognition effect, but requires high computing power cost. The small model has lower requirement for computing power, but the effect is worse than that of the large model. Therefore, how to balance the relationship between the two is a problem that cannot be avoided for each specific task. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a design method, device and equipment of a general task framework model based on deep learning, which aims to solve the problem that the existing model is difficult to balance the computing power and accuracy.

[0004] To achieve the above purpose, the present application provides a design method of a general task framework model based on deep learning, which comprises:

[0005] Constructing a general task framework model, wherein the general task framework model comprises an iterative task model and an index model;

[0006] Performing forward inference operation on the input data input by the iterative task model to obtain a target result;

[0007] Performing forward inference operation on the feature parameters output by the iterative task model as input by the index model to obtain an index result;

[0008] According to the index result, it is determined whether the iterative task model takes the target result as input to perform forward inference operation in the next round of iteration.

[0009] Preferably, according to the index result, it is determined whether the iterative task model takes the target result as input to perform forward inference operation in the next round of iteration, which comprises:

[0010] Cyclically performing forward inference operation on the target result obtained at present as input of the iterative task model, and performing forward inference operation on the feature parameters output by the iterative task model at present by the index model until a preset condition is met.

[0011] Preferably, the preset condition comprises:

[0012] the index result is lower than a set threshold; or,

[0013] a plurality of the index results obtained through iteration tend to be consistent; or,

[0014] the number of iterations of the iterative task model reaches a preset number of iterations.

[0015] Preferably, the network structure of the iterative task model comprises consistent dimensions of head and tail tensors.

[0016] Preferably, the index model is constructed through a convolution method or a full connection method.

[0017] Preferably, the index result output by the index model is set as a numerical value.

[0018] Preferably, the numerical value is set as a continuous numerical value in a specific interval.

[0019] To achieve the above object, the present application further provides a design device based on a deep learning general task framework model, which comprises:

[0020] a construction unit, configured to construct a general task framework model, wherein the general task framework model comprises an iterative task model and an index model;

[0021] a first operation unit, configured to perform forward inference operation on input data input through the iterative task model to obtain a target result;

[0022] a second operation unit, configured to perform forward inference operation on feature parameters output by the iterative task model as input through the index model to obtain an index result;

[0023] an iteration judgment unit, configured to judge whether the iterative task model takes the target result as input to perform forward inference operation in the next round of iteration according to the index result.

[0024] To achieve the above object, the present application further provides a device comprising a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a design method of a deep learning general task framework model as described in the above embodiments.

[0025] In order to achieve the above object, the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the design method of the deep learning general task framework model according to the above embodiment.

[0026] Advantages:

[0027] The above scheme constructs an iterative task model and an index model for evaluating the output result of the iterative task model, so that the best balance between computing power and accuracy can be ensured flexibly in use. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0029] Figure 1 A flowchart of the design method of the deep learning general task framework model according to an embodiment of the present application is provided.

[0030] Figure 2 A framework structure diagram of the general task framework model according to an embodiment of the present application is provided.

[0031] Figure 3 A structure diagram of the iterative task model according to an embodiment of the present application is provided.

[0032] Figure 4 A structure diagram of the design device of the deep learning general task framework model according to an embodiment of the present application is provided.

[0033] The implementation of the object of the application, functional features and advantages will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION

[0034] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0035] In the description of the present application, the terms "first", "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0036] The content of the present application will be described in detail below in combination with embodiments.

[0037] At present, the main researchers and developers generally optimize deep learning models to ensure that the recognition accuracy of the model is maximized under the premise that the computing power is met, but this consumes a lot of manpower. Common practices include: some researches use various lightweight model backbone feature extraction structures, some researches distill models, use teacher networks (large models) to train student networks (small models), and how to improve the accuracy of small models, such as optimizing the loss of specific tasks, and performing pruning, quantization and other operations after the model is trained. There is also a way to use different computing power large, medium and small models to complete the same task according to the current running hardware device during use. Therefore, the existing methods are difficult to achieve the balance between the accuracy and performance of the model according to the computing power of the current device.

[0038] Based on this, the present application proposes a design method based on a deep learning general task framework model, which mainly focuses on how to better maintain the balance between accuracy and performance according to the computing power of the current device and the accuracy of the current model in the process of studying how to miniaturize the model, so as to obtain better accuracy on devices with higher computing power and also obtain better accuracy on devices with lower computing power. The calculation amount of the model can be artificially controlled and adjusted to obtain the best result of the current scheme, which is flexible. At the same time, the present application proposes a relatively general model framework, which is a model design paradigm and has universality.

[0039] Reference Figure 1A flowchart of a design method of a general task framework model based on deep learning provided by an embodiment of the application is shown.

[0040] In this embodiment, the method comprises:

[0041] S11, a general task framework model is constructed, which comprises an iterative task model and an index model.

[0042] S12, the input data is input into the iterative task model for forward inference operation to obtain a target result.

[0043] S13, the feature parameters output by the iterative task model are input into the index model for forward inference operation to obtain an index result.

[0044] S14, it is judged whether the target result is input into the iterative task model for forward inference operation in the next round of iteration according to the index result.

[0045] As Figure 2 shown in the schematic diagram of the framework structure of the general task framework model. The general task framework model comprises an iterative task model and an index model. The input data is input into the iterative task model for forward inference operation to obtain the result required by the application scenario. Then, the current result is evaluated by the index model. The index model does not directly evaluate the result of the iterative task model (because there is no real label during testing), but indirectly uses the output feature parameters of each layer or part of important layers in the iterative task model as the input data of the index model (in this part, flexible selection can be made according to the specific situation: the output feature map of the entire stage of the iterative task model can be used, or a part or other conversion layers, attention layers, etc. can be used), the index result output by the forward operation of the index model is obtained, and then it is judged according to the good or bad level of the index result.

[0046] Further, the judging whether the target result is input into the iterative task model for forward inference operation in the next round of iteration according to the index result comprises:

[0047] the current target result is input into the iterative task model for forward inference operation, and the feature parameters output by the iterative task model are input into the index model for forward inference operation until the preset condition is met.

[0048] The preset condition comprises:

[0049] the index result is lower than a set threshold; or,

[0050] The multiple index results obtained through iteration tend to be consistent; or

[0051] The number of iterations of the iterative task model reaches a preset number of iterations.

[0052] Further, the index result output by the index model is set as a numerical value.

[0053] The numerical value is set as a continuous numerical value in a specific interval.

[0054] In the embodiment, the threshold value can be set artificially. Generally, if the current result accuracy of the iterative task model is higher than the threshold value, it is considered that the current result accuracy of the iterative task model is higher, and the iterative task model can be directly used. Otherwise, the last output result of the iterative task model is taken as the input of the iterative task model to perform forward inference operation again, and the output result of the iterative task model after iteration and the index result of the corresponding index model are obtained. The index result is judged again. This cycle is repeated until the index result reaches the threshold value requirement, or the number of iterations of the iterative task model reaches the maximum number of iterations set artificially or the index model output result changes little. At the same time, when the threshold value or the maximum number of iterations is set artificially, the computing power of the running device can be considered (that is, when the threshold value or the maximum number of iterations is reached, the task requirement is basically met, and the accuracy of the model based on the computing power of the device is optimal).

[0055] Further, the network structure of the iterative task model includes consistent dimensions of head and tail tensors.

[0056] Further, the index model is constructed by convolution or full connection.

[0057] In the embodiment, the iterative task model is iterative or a part of it is iterative. The requirement is that the model structure of the iterative part is consistent in dimensions of head and tail tensors or can be operated (such as full convolution with a reduction of 2 times). That is, the output of the part can be directly used as the input of the part, such as the classic hourglass module, which is generally used for pose key point coordinate detection. The common scheme is to stack multiple similar or identical modules in series. In the embodiment, the hourglass module is set to be operated multiple times, which can be referred to in Figure 3The index model needs to take the feature parameters of the intermediate output of the stage (all stages or part of the stages) of the iterative task model as input (the index model is connected with the iterative task model, but gradient back propagation is not performed during training, because it is regarded as input data), and the index model can be constructed in a convolution or full connection manner, that is, the index model can be a CNN structure or a full connection structure. In addition, the iterative task model can ignore the index model during training and testing, and the output of the index model is one or more numerical values or other (which can be set by humans), which is generally set as a continuous value in a specific interval. For example, the output of the iterative task model is binary classification, and whether there is a cat in the picture is identified, and the output is the probability of yes and no, and the sum is 1; the index model is set to output a value in the interval [0, 1], which represents the confidence of the output result of the iterative task model.

[0058] During the training phase of the general task framework model, the iterative task model needs to set the maximum number of iterations, that is, multiple output results can be obtained by multiple forward propagations in each training, and the loss is calculated respectively, and the gradient is back propagated together; at the same time, the index model sets the network structure parameters of the index model itself each time, and the output can be set as one or more consistent with the number of losses calculated by the index model, and the label data supervised by the index model is the loss calculated by the iterative task model in the current iteration, that is, the index model learns the current loss of the iterative task model, which is the learning of the loss (that is, the loss calculated by the iterative task model is fitted until the loss of the index model is very low (lower than the set threshold) before the back propagation of the iterative task model), that is, the index model is equivalent to learning the loss of the iterative task model, and the loss of the loss of the iterative task model is calculated. The principle is to fit the current loss with the output of the current feature parameters, because the loss size can reflect the goodness of the current iterative task model (generally, the lower the loss, the better), if the index model can fit the loss well, then it is the proxy of the loss, as long as the output of the index model is lower, that is, the true loss of the current output of the iterative task model is lower if there is a supervised label, it is indicated that the output result precision of the current iterative task model is higher, so that the output result of the index model can be used as a reference for whether to perform the next iteration forward reasoning of the iterative task model, and the controllability is high.

[0059] Referring to Figure 4 Fig. 1 shows a structure schematic diagram of a design device based on a deep learning general task framework model according to an embodiment of the present application.

[0060] In the embodiment, the device 40 comprises:

[0061] The construction unit 41 is configured to construct a general task framework model, which includes an iterative task model and an index model;

[0062] The first operation unit 42 is configured to perform forward inference operation on the input data by using the iterative task model to obtain a target result.

[0063] The second operation unit 43 is configured to perform forward inference operation on the feature parameters output by the iterative task model by using the index model as input to obtain an index result.

[0064] The iteration judgment unit 44 is configured to judge whether the iterative task model takes the target result as input to perform forward inference operation in the next iteration according to the index result.

[0065] Further, the iteration judgment unit 44 is configured to:

[0066] The current target result is taken as input to perform forward inference operation on the iterative task model, and the feature parameters output by the iterative task model are taken as input to perform forward inference operation by using the index model until a preset condition is met.

[0067] The preset condition includes:

[0068] The index result is lower than a set threshold; or,

[0069] The index results obtained through iteration tend to be consistent; or,

[0070] The number of iterations of the iterative task model reaches a preset iteration number.

[0071] Further, the network structure of the iterative task model includes consistent dimensions of head and tail tensors.

[0072] Further, the index model is constructed by using convolution or full connection.

[0073] Further, the index result output by the index model is set as a numerical value.

[0074] Further, the numerical value is set as a continuous numerical value in a specific interval.

[0075] The various unit modules of the device 40 can respectively perform the corresponding steps in the above method embodiments, and thus the unit modules will not be described here. For details, please refer to the above description of the corresponding steps.

[0076] The embodiment of the present application also provides a device comprising the design device based on the deep learning universal task framework model as described above, wherein the design device based on the deep learning universal task framework model can adopt Figure 4 The structure of the embodiment can correspondingly execute Figure 1 The technical solutions of the method embodiment shown achieve similar principles and technical effects, and details can be referred to the related descriptions in the above embodiments.

[0077] The device can include a mobile phone, a digital camera, a tablet computer or the like having a photographing function, or a device having an image processing function, or a device having an image display function. The device can include a memory, a processor, an input unit, a display unit, a power supply and the like.

[0078] The memory can be used to store software programs and modules, and the processor can execute various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs (such as an image playing function and the like) required by at least one function and the like; the data storage area can store data created according to the use of the device and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device or other volatile solid-state memory device. Correspondingly, the memory can also include a memory controller to provide access to the memory for the processor and the input unit.

[0079] The input unit can be used to receive inputted digital or character or image information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. Specifically, the input unit of the embodiment can include a camera in addition to a touch-sensitive surface (such as a touch display screen) and other input devices.

[0080] The display unit can be used to display information inputted by a user or provided to a user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, video and any combination thereof. The display unit can include a display panel, which can be configured in the form of an LCD (Liquid Crystal Display) or an OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface can cover the display panel, and when the touch-sensitive surface detects a touch operation thereon or in the vicinity thereof, transmits to the processor to determine the type of the touch event, and then the processor provides corresponding visual output on the display panel according to the type of the touch event.

[0081] The embodiment of the present application further provides a computer readable storage medium, which can be the computer readable storage medium contained in the memory in the above embodiment, or can exist separately and not be assembled into the device. The computer readable storage medium stores at least one instruction, which is loaded and executed by the processor to realize the method for designing the deep learning-based general task framework model as shown in the above embodiment. Figure 1 The computer readable storage medium can be a read-only memory, a disk or an optical disk, etc.

[0082] It should be noted that each embodiment in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between each embodiment can be referred to each other. For the device embodiment, the equipment embodiment and the storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0083] In addition, in the present specification, the term "comprising", "containing" or any other variant thereof is intended to cover the non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes the elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0084] The above description shows and describes the preferred embodiments of the present application, and it should be understood that the present application is not limited to the forms disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the present inventive concept by the above teaching or related art or knowledge. The modification and change made by the person skilled in the art without departing from the spirit and scope of the present application should be within the protection scope of the claims of the present application.

Claims

1. A method for designing a deep learning general task framework model, characterized in that, The method comprises: constructing a general task framework model, the general task framework model comprising an iterative task model and an index model; wherein the iterative task model is iterative, and the model structure of the iterative part is consistent in the dimensions of head and tail tensors; the iterative task model adopts an hourglass module for pose key point coordinate detection; the index model is used for evaluating the output result of the iterative task model; and the index model is constructed by convolution or full connection; performing forward inference operation on input data input by the iterative task model to obtain a target result; the input data comprises picture data; performing forward inference operation on feature parameters output by the iterative task model as input by the index model to obtain an index result; determining whether the target result is taken as input for forward inference operation in the next iteration of the iterative task model according to the index result; wherein the determination whether the target result is taken as input for forward inference operation in the next iteration of the iterative task model according to the index result comprises: performing forward inference operation on the target result obtained at present as input of the iterative task model in a loop, and performing forward inference operation on feature parameters output by the iterative task model at present as input by the index model, until a preset condition is met; in the training stage of the general task framework model, the index model learns the current loss of the iterative task model, and the current loss calculated by the iterative task model is fitted until the loss of the index model is lower than a set threshold, and then the back propagation of the iterative task model is performed.

2. The design method of a deep learning general task framework model according to claim 1, characterized in that, the preset condition comprises: the index result is lower than a set threshold; or a plurality of index results obtained through iteration tend to be consistent; or the number of iterations of the iterative task model reaches a preset number of iterations.

3. The design method of a general task framework model based on deep learning according to any one of claims 1 to 2, characterized in that, the index result output by the index model is set as a numerical value.

4. The design method of claim 3, wherein, the numerical value is set as a continuous numerical value in a specific interval. 5.A design device of a deep learning general task framework model, characterized by comprising: The device comprises: a construction unit configured to construct a general task framework model, the general task framework model comprising an iterative task model and an index model; wherein the iterative task model is iterative, and the model structure of the iterative part is consistent in the dimensions of head and tail tensors; the iterative task model adopts an hourglass module for pose key point coordinate detection; the index model is used for evaluating the output result of the iterative task model; and the index model is constructed by convolution or full connection; a first operation unit configured to perform forward inference operation on input data input by the iterative task model to obtain a target result; the input data comprises picture data; a second operation unit configured to perform forward inference operation on feature parameters output by the iterative task model as input by the index model to obtain an index result; An iteration judgment unit is configured to judge whether the target result is taken as input for the next round of iteration of the iterable task model according to the index result. The iteration judgment unit is further configured to: The current target result is taken as input for the forward inference operation of the iterable task model, and the feature parameters output by the iterable task model are obtained through the index model for the forward inference operation until a preset condition is met. In the training phase of the general task framework model, the index model learns the current loss of the iterable task model, and the current loss calculated by the iterable task model is fitted until the loss of the index model is lower than a set threshold, and then the back propagation of the iterable task model is performed.

6. An apparatus, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the design method of the deep learning-based general task framework model according to any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the design method of the deep learning-based general task framework model according to any one of claims 1 to 4.

Images