Method, apparatus and device for modifying fixed-length input neural network model

Abstract

This invention relates to a method, apparatus, and device for modifying a fixed-length input neural network model. The method includes: acquiring the original image of a neural network model that only supports fixed-length input; acquiring a mask processing subgraph, including a mask transformation subgraph and a mask clearing subgraph, for each operator of the neural network model; then generating a data clearing subgraph containing the mask processing subgraphs of each operator; upon receiving a mask dataset input by a user, performing a transformation operation on the masks in the mask dataset according to the mask transformation subgraph in the data clearing subgraph, and replacing the operators in the original image that need to be cleared with the mask clearing subgraphs of those operators, thereby merging the data clearing subgraph and the original image; finally, generating a target neural network model for calculating variable-length input data corresponding to the mask based on the merged image. Based on this, this application improves the efficiency and accuracy of adding mask and data clearing operations, and thus, this application can improve the efficiency and accuracy of modifying neural network models.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method, apparatus, and device for modifying a fixed-length input neural network model. Background Technology

[0002] Neural network models are widely used in scenarios such as speech recognition, image recognition, and semantic understanding. A neural network model that supports computation on variable-length input data can be trained using training samples of equal length. However, when used for actual inference, this type of trained neural network model only supports computation on fixed-length input data and cannot support computation on variable-length input data. Variable-length input data refers to data obtained by padding the input data with invalid data.

[0003] Currently, by manually adding a mask and clearing data to the aforementioned neural network model, the model can support computation of variable-length input data. However, manually adding masks and clearing data has problems such as low efficiency, high labor costs, and the risk of omissions. Summary of the Invention

[0004] To address the issues of low efficiency and accuracy in modifying fixed-length input neural network models to support variable-length inputs in existing technologies, this invention provides a method, apparatus, and device for modifying fixed-length input neural network models.

[0005] This invention provides a method for modifying a fixed-length input neural network model, including:

[0006] Obtain neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input;

[0007] Obtain the mask processing subgraphs of each operator in the neural network model; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraph is used to represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraph is used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph.

[0008] Generate a data clearing diagram containing each operator of the neural network model and the mask processing subgraph of each operator;

[0009] In response to a user's mask input, a mask dataset is received; the mask dataset contains at least one mask.

[0010] The mask in the mask dataset is transformed according to the mask transformation subgraph in the data clearing diagram, and the operator that needs to be cleared in the original diagram is replaced with the mask clearing subgraph of the operator, so as to realize the merging of the data clearing diagram and the original diagram.

[0011] A target neural network model is generated based on the model diagram formed by merging the data clearing diagram and the original diagram; the target neural network model is used to calculate the variable-length input data corresponding to the mask.

[0012] This invention also provides a device for modifying a fixed-length input neural network model, comprising:

[0013] A neural network model data acquisition module is used to acquire neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input.

[0014] The mask processing subgraph acquisition module is used to acquire mask processing subgraphs for each operator of the neural network model. The mask processing subgraph includes a mask transformation subgraph and a mask clearing subgraph. The mask transformation subgraph represents the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph. The mask clearing subgraph represents the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph.

[0015] The data clearing diagram generation module is used to generate a data clearing diagram that includes each operator of the neural network model and the mask processing subgraph of each operator;

[0016] The mask dataset receiving module is used to receive a mask dataset in response to a user's mask input operation; the mask dataset contains at least one mask;

[0017] The merging module is used to perform a transformation operation on the mask in the mask dataset according to the mask transformation subgraph in the data clearing diagram, and replace the operator that needs to be cleared in the original diagram with the mask clearing subgraph of the operator, so as to realize the merging of the data clearing diagram and the original diagram;

[0018] The target neural network model generation module is used to generate a target neural network model based on the model graph formed by merging the data clearing image and the original image; the target neural network model is used to calculate the variable-length input data corresponding to the mask.

[0019] This invention also provides a device for modifying a fixed-length input neural network model, comprising:

[0020] At least one processor; and,

[0021] A memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0023] Obtain neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input;

[0024] Obtain the mask processing subgraphs of each operator in the neural network model; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraph is used to represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraph is used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph.

[0025] Generate a data clearing diagram containing each operator of the neural network model and the mask processing subgraph of each operator;

[0026] In response to a user's mask input, a mask dataset is received; the mask dataset contains at least one mask.

[0027] The mask in the mask dataset is transformed according to the mask transformation subgraph in the data clearing diagram, and the operator that needs to be cleared in the original diagram is replaced with the mask clearing subgraph of the operator, so as to realize the merging of the data clearing diagram and the original diagram.

[0028] A target neural network model is generated based on the model diagram formed by merging the data clearing diagram and the original diagram; the target neural network model is used to calculate the variable-length input data corresponding to the mask.

[0029] This invention employs the above technical solution to obtain neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input; mask processing subgraphs of each operator of the neural network model are obtained; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraph is used to represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraph is used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph; generating data containing the original image of the neural network model; obtaining mask processing subgraphs of each operator of the neural network model; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraphs represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraphs represent the calculation process performed by the operator to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph; generating data containing the original image of the neural network model; obtaining mask processing subgraphs of each operator ... The process involves: a data clearing diagram of each operator and its mask processing subgraph in the network model; receiving a mask dataset in response to a user's mask input operation; the mask dataset containing at least one mask; performing transformation operations on the masks in the mask dataset according to the mask transformation subgraph in the data clearing diagram, and replacing the operator in the original diagram that needs data clearing with its mask clearing subgraph, thereby merging the data clearing diagram and the original diagram; generating a target neural network model based on the model diagram formed by merging the data clearing diagram and the original diagram; the target neural network model being used to calculate variable-length input data corresponding to the mask. Based on this, the present application can automatically add mask and data clearing operations to the neural network model, improving the efficiency and accuracy of adding mask and data clearing operations, and thus improving the modification efficiency and accuracy of the neural network model. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating a method for modifying a fixed-length input neural network model according to an embodiment of the present invention;

[0032] Figure 2 This invention provides a mask transformation subgraph and a mask clearing subgraph.

[0033] Figure 3 This is a schematic diagram of the structure of a device for modifying a fixed-length input neural network model provided in an embodiment of the present invention;

[0034] Figure 4This is a schematic diagram of the structure of a device for modifying a fixed-length input neural network model provided in an embodiment of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Cloud-based neural network models that support simultaneous input of multiple data streams and can compute those multiple inputs simultaneously are widely used in scenarios such as speech recognition, image recognition, and semantic understanding. These scenarios mostly involve variable-length input data. Specifically, during neural network inference, some of the input data being computed simultaneously is padded with invalid data to align the shapes of the input data. However, during neural network inference, the invalid padded data may gradually transform from initial values ​​that do not affect the computation result into valid values ​​that can influence the result; for example, 0 becomes non-zero after passing through the Add operator. Therefore, after padding the input data with invalid data, it is also necessary to clear the input data to zero.

[0037] In the training scenario of fixed-length input neural network models, neural network models that support variable-length input data computation classify and sort the training data, grouping data of equal length together. During model training, the length of the multiple data streams input to the prototype code of the neural network model supporting variable-length input data computation is always equal. However, fixed-length input neural network models cannot automatically perform data zeroing operations on variable-length input data, thus limiting their support to computation with fixed-length input data and preventing them from supporting computation with variable-length input data.

[0038] Currently, by manually adding a mask and clearing data to the aforementioned neural network model, the model can support computation of variable-length input data. However, manually adding masks and clearing data has problems such as low efficiency, high labor costs, and the risk of omissions.

[0039] Figure 1 This is a flowchart illustrating a method for modifying a fixed-length input neural network model according to an embodiment of the present invention. Figure 1 As shown, this process includes:

[0040] Step 101: Obtain neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input.

[0041] In the embodiments of this specification, a specific method for obtaining neural network model data may be to obtain neural network model data from a preset database in response to the user's original image acquisition operation.

[0042] Step 102: Obtain the mask processing subgraphs of each operator in the neural network model; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraph is used to represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraph is used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph.

[0043] In this embodiment of the specification, step 102: obtaining the mask processing subgraph of each operator of the neural network model, may specifically include:

[0044] Obtain the identification information of each operator in the neural network model.

[0045] Based on the identification information, the mask processing subgraph of the operator corresponding to the identification information is obtained from the preset database.

[0046] In this embodiment, for each operator, a zeroing operation class is pre-registered and stored in a preset database. This zeroing operation class includes two functions: one function generates the operator's mask transformation subgraph, and the other function generates the operator's mask zeroing subgraph. The mask transformation subgraph indicates how the mask changes after passing through this operator. The mask zeroing subgraph indicates how the operator's calculation process changes to achieve data zeroing after the corresponding operator inputs a mask.

[0047] In a specific example Figure 2 This is a mask transformation subgraph and a mask clearing subgraph provided in an embodiment of the present invention. For example... Figure 2 As shown, Figure 2 In the diagram, (a) is the original image of the Conv operator. (b) is the mask transformation sub-image corresponding to the Conv operator, which shows that the mask does not change after passing through the Conv operator. (c) is the mask clearing sub-image corresponding to the Conv operator, which shows that after the Conv operator inputs the mask and data, in order to clear the data, the Conv operator first multiplies the mask and data to obtain the data after the invalid padding data is cleared, and then performs a convolution operation on the data to obtain the output data.

[0048] Step 103: Generate a data clearing diagram containing each operator of the neural network model and the mask processing subgraph of each operator.

[0049] In this embodiment of the specification, after obtaining the mask processing subgraphs of each operator of the neural network model, a data clearing diagram containing each operator of the neural network model and its mask processing subgraph is generated based on the obtained mask processing subgraphs. Thus, the data clearing diagram includes the mask processing subgraphs of each operator of the neural network model, making it less likely to miss adding masks and data clearing operations when adding masks and data clearing operations to the neural network model based on the data clearing diagram.

[0050] Step 104: In response to the user's mask input operation, receive a mask dataset; the mask dataset contains at least one mask.

[0051] In the embodiments of this specification, the fixed-length input neural network model can support users to input multiple data simultaneously. In practical applications, if the user needs to use the fixed-length input neural network model to calculate multiple data containing variable-length input data, the user can set a corresponding mask according to the variable-length input data in the multiple data. If the multiple data needs to contain multiple variable-length input data, the number of masks is multiple.

[0052] Step 105: Perform a transformation operation on the mask in the mask dataset according to the mask transformation subgraph in the data clearing diagram, and replace the operator that needs to be cleared in the original diagram with the mask clearing subgraph of the operator, so as to merge the data clearing diagram and the original diagram.

[0053] In this embodiment of the specification, step 105, which involves transforming the masks in the mask dataset according to the mask transformation subgraph in the data clearing diagram, may specifically include:

[0054] For each mask in the mask dataset, determine the corresponding mask transformation subgraphs from the data clearing graph.

[0055] The mask is transformed using each of the mask transformation subgraphs.

[0056] In the embodiments of this specification, the calculation of a certain input data often involves multiple operators, and the mask transformation subgraph of these multiple operators is the mask transformation subgraph corresponding to the mask of the input data. To illustrate the above embodiments more clearly, the following examples are provided:

[0057] Assume an original image is H3(H2(H1(input))), where H1, H2, and H3 are operators, all of which require data clearing. The mask transformation subgraph for H1 is M1(mask), for H2 it is M2(M1), and for H3 it is M3(M2). Then, the mask transformation subgraphs corresponding to each mask are H1, H2, and H3. The merging process between the data-cleared image and the original image is as follows: The mask is transformed using the mask transformation subgraphs of H1, H2, and H3 to obtain... M3(M2(M1(mask))), then, from the data clearing diagram, determine the mask clearing sub-diagram MH1 of H1, the mask clearing sub-diagram MH2 of H2, and the mask clearing sub-diagram MH3 of H3, and replace H1 in the original diagram with MH1, H2 in the original diagram with MH2, and H3 in the original diagram with MH3. The data clearing diagram and the original diagram are successfully merged. The merged diagram is MH3(MH2(MH1(input,mask),M1(mask)),M2(M1)),M3(M2)).

[0058] The above describes the merging principle between the data clearing image and the original image when there is only one mask. The merging principle is the same when there are multiple masks.

[0059] Step 106: Generate a target neural network model based on the model diagram formed by merging the data clearing diagram and the original diagram; the target neural network model is used to calculate the variable-length input data corresponding to the mask.

[0060] In the embodiments described in this specification, a mask and a data clearing operation (i.e., a mask clearing subgraph) are added to the target neural network model, enabling it to be used to compute variable-length input data. However, the variable-length input data input into the target neural network model needs to match the corresponding mask in the user-preset mask dataset.

[0061] This specification's embodiments employ the above-described technical solution to obtain neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input; mask processing subgraphs for each operator of the neural network model are obtained; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraph is used to represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraph is used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into it; generating data containing the original image of the neural network model; obtaining mask processing subgraphs for each operator of the neural network model; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraphs represent the changes that occur to the mask after it passes through the operator corresponding to the mask clearing subgraph; the mask clearing subgraphs represent the calculation process performed by the operator to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph; generating data containing the original image of the neural network model; obtaining mask processing subgraphs for each operator of the neural network model; obtaining mask processing subgraphs for each operator; obtaining mask processing subgraphs for each operator corresponding to the mask clearing subgraph; obtaining mask processing subgraphs for each operator corresponding to the mask transformation ... The present invention relates to a neural network model, comprising operators and a data clearing subgraph of each operator; receiving a mask dataset in response to a user's mask input operation; the mask dataset containing at least one mask; performing a transformation operation on the masks in the mask dataset according to the mask transformation subgraph in the data clearing subgraph, and replacing the operators in the original image that need data clearing with the mask clearing subgraph of those operators, thereby merging the data clearing subgraph and the original image; generating a target neural network model based on the model graph formed by merging the data clearing subgraph and the original image; the target neural network model being used to calculate variable-length input data corresponding to the mask. Based on this, the present invention can automatically add mask and data clearing operations to the neural network model, improving the efficiency and accuracy of adding mask and data clearing operations, and thus improving the modification efficiency and accuracy of the neural network model.

[0062] In this embodiment of the specification, before step 105: performing a transformation operation on the mask in the mask dataset according to the mask transformation subgraph in the data clearing diagram, and replacing the operator that needs to be cleared in the original diagram with the mask clearing subgraph of that operator, the method in this embodiment of the specification may further include:

[0063] The data clearing graph is optimized; the optimization operation includes deleting operators in the data clearing graph that do not have corresponding mask clearing subgraphs.

[0064] In the embodiments of this specification, during the optimization of the data clearing diagram, if it is identified that an operator only has a corresponding mask transformation subgraph but no corresponding mask clearing subgraph, then the operator can be deleted from the data clearing diagram. In this way, useless mask transformation subgraphs in the data clearing diagram are deleted, reducing redundant calculations in the model diagram formed by merging the data clearing diagram and the original diagram.

[0065] In the embodiments of this specification, the optimization operation may further include merging specific operator pairs. The intermediate operators of the specific operator pair do not have corresponding mask clearing subgraphs, and both operators corresponding to the specific operator pair have corresponding mask transformation subgraphs. This solution can further remove useless mask transformation subgraphs from the data clearing graph, reducing redundant calculations in the model graph formed by merging the data clearing graph and the original graph.

[0066] In the embodiments described in this specification, the file format of the neural network model is OpenNeural Network Exchange (ONNX).

[0067] Based on a general inventive concept, embodiments of this specification also provide a device for modifying a fixed-length input neural network model. Figure 3 This is a schematic diagram of the structure of a device for modifying a fixed-length input neural network model provided in an embodiment of the present invention. Figure 3 As shown, this device includes:

[0068] The neural network model data acquisition module 31 is used to acquire neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input.

[0069] The mask processing subgraph acquisition module 32 is used to acquire the mask processing subgraphs of each operator of the neural network model; the mask processing subgraph includes a mask transformation subgraph and a mask clearing subgraph; the mask transformation subgraph is used to represent the changes that occur to the mask after the mask passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraph is used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph.

[0070] The data clearing diagram generation module 33 is used to generate a data clearing diagram that includes each operator of the neural network model and the mask processing subgraph of each operator.

[0071] The mask dataset receiving module 34 is used to receive a mask dataset in response to a user's mask input operation; the mask dataset contains at least one mask.

[0072] The merging module 35 is used to perform a transformation operation on the mask in the mask dataset according to the mask transformation subgraph in the data clearing diagram, and replace the operator that needs to be cleared in the original diagram with the mask clearing subgraph of the operator, so as to realize the merging of the data clearing diagram and the original diagram.

[0073] The target neural network model generation module 36 is used to generate a target neural network model based on the model graph formed by merging the data clearing image and the original image; the target neural network model is used to calculate the variable-length input data corresponding to the mask.

[0074] In the embodiments of this specification, the merging module 35 can be specifically used for:

[0075] For each mask in the mask dataset, determine the corresponding mask transformation subgraphs from the data clearing graph.

[0076] The mask is transformed using each of the mask transformation subgraphs.

[0077] In the embodiments of this specification, the apparatus may further include: a data clearing map optimization module, used for:

[0078] The data clearing graph is optimized; the optimization operation includes deleting operators in the data clearing graph that do not have corresponding mask clearing subgraphs.

[0079] In the embodiments described in this specification, the optimization operation may further include merging specific operator pairs.

[0080] The intermediate operators of the specific operator pair do not have corresponding mask clearing subgraphs, and both operators of the specific operator pair have corresponding mask transformation subgraphs.

[0081] In the embodiments of this specification, the mask processing sub-image acquisition module 32 can be specifically used for:

[0082] Obtain the identification information of each operator in the neural network model.

[0083] Based on the identification information, the mask processing subgraph of the operator corresponding to the identification information is obtained from the preset database.

[0084] In the embodiments described in this specification, the file format of the neural network model can be Open Neural Network Exchange.

[0085] Based on a general inventive concept, embodiments of the present invention also provide a device for modifying a fixed-length input neural network model. Figure 4 This is a schematic diagram of the structure of a device for modifying a fixed-length input neural network model provided in an embodiment of the present invention. For example... Figure 4 As shown, device 400 may include:

[0086] At least one processor 410; and,

[0087] Memory 430 communicatively connected to the at least one processor; wherein,

[0088] The memory 430 stores instructions 420 that can be executed by the at least one processor 410, the instructions being executed by the at least one processor 410 to enable the at least one processor 410 to:

[0089] Obtain neural network model data; the neural network model data includes the original image of the neural network model; the neural network model is a neural network model that only supports fixed-length input.

[0090] Obtain the mask processing subgraphs of each operator in the neural network model; the mask processing subgraphs include mask transformation subgraphs and mask clearing subgraphs; the mask transformation subgraphs are used to represent the changes that occur to the mask after it passes through the operator corresponding to the mask transformation subgraph; the mask clearing subgraphs are used to represent the calculation process performed by the operator corresponding to the mask clearing subgraph to clear the data after the mask is input into the operator corresponding to the mask clearing subgraph.

[0091] Generate a data clearing graph containing each operator of the neural network model and the mask processing subgraph of each operator.

[0092] In response to a user's mask input operation, a mask dataset is received; the mask dataset contains at least one mask.

[0093] The mask in the mask dataset is transformed according to the mask transformation subgraph in the data clearing diagram, and the operator that needs to be cleared in the original diagram is replaced with the mask clearing subgraph of the operator, so as to merge the data clearing diagram and the original diagram.

[0094] A target neural network model is generated based on the model diagram formed by merging the data clearing diagram and the original diagram; the target neural network model is used to calculate the variable-length input data corresponding to the mask.

[0095] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0096] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0097] The steps in the methods of the various embodiments of the present invention can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in the various embodiments can be replaced or combined.

[0098] The modules and sub-modules in the various embodiments of the present invention can be merged, divided, and deleted according to actual needs.

[0099] In the embodiments provided by this invention, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0100] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0101] Furthermore, the functional modules or sub-modules in the various embodiments of the present invention can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0102] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0103] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0104] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for modifying a fixed-length input neural network model, characterized in that, The method is applied to voice recognition and image recognition, and comprises the following steps: obtaining neural network model data; the neural network model data contains a graph of a neural network model; the neural network model is a neural network model supporting only fixed-length input; obtaining mask processing subgraphs of each operator of the neural network model from zero-clearing operation classes of each operator of the neural network model, which are pre-registered and stored in a preset database; the mask processing subgraph comprises a mask transformation subgraph and a mask zero-clearing subgraph; the mask transformation subgraph is used to represent changes of a mask after the mask passes through the corresponding operator of the mask transformation subgraph; the mask zero-clearing subgraph is used to represent a calculation process of an operator for realizing data zero-clearing after the mask inputs the corresponding operator of the mask zero-clearing subgraph; and a data zero-clearing graph containing each operator of the neural network model and the mask processing subgraph of the operator is generated; in response to a mask input operation of a user, receiving a mask data set; the mask data set contains at least one mask; transforming the mask in the mask data set according to the mask transformation subgraph in the data zero-clearing graph, and replacing an operator in the graph that needs to perform data zero-clearing with the mask zero-clearing subgraph of the operator, so as to realize merging of the data zero-clearing graph and the graph; wherein the transforming the mask in the mask data set according to the mask transformation subgraph in the data zero-clearing graph specifically comprises: for each mask in the mask data set, determining each mask transformation subgraph corresponding to the mask from the data zero-clearing graph; and transforming the mask by using the mask transformation subgraph; generating a target neural network model according to a model graph obtained by merging the data zero-clearing graph and the graph; the target neural network model is used to calculate variable-length input data corresponding to the mask.

2. The method of claim 1, wherein, Before the transforming the mask in the mask data set according to the mask transformation subgraph in the data zero-clearing graph, and replacing the operator in the graph that needs to perform data zero-clearing with the mask zero-clearing subgraph of the operator, the method further comprises: performing an optimization operation on the data zero-clearing graph; the optimization operation comprises deleting an operator in the data zero-clearing graph without a corresponding mask zero-clearing subgraph.

3. The method of claim 2, wherein, The optimization operation further comprises merging a specific operator pair; each operator in the middle of the specific operator pair does not have a corresponding mask zero-clearing subgraph, and both operators corresponding to the specific operator pair have corresponding mask transformation subgraphs.

4. The method of claim 1, wherein, The obtaining the mask processing subgraph of each operator of the neural network model specifically comprises: obtaining identification information of each operator of the neural network model; obtaining the mask processing subgraph of the operator corresponding to the identification information from the preset database according to the identification information.

5. The method of claim 1, wherein a file format of the neural network model is Open Neural Network Exchange.

6. An apparatus for modifying a fixed-length input neural network model, the apparatus comprising: The method is applied to voice recognition and image recognition, and comprises the following steps: The neural network model data acquisition module is configured to acquire neural network model data; the neural network model data contains an original graph of a neural network model; the neural network model is a neural network model that supports only fixed-length input; The mask processing subgraph acquisition module is configured to acquire mask processing subgraphs of each operator of the neural network model from zero-clearing operations of the respective operators that are pre-registered and stored in a preset database; the mask processing subgraphs include mask transformation subgraphs and mask zero-clearing subgraphs; the mask transformation subgraphs are used to represent changes in a mask after the mask passes through the corresponding operator of the mask transformation subgraphs; and the mask zero-clearing subgraphs are used to represent a calculation process performed by the operator to achieve data zero-clearing after the mask inputs the corresponding operator of the mask zero-clearing subgraphs. The data zero-clearing graph generation module is configured to generate a data zero-clearing graph containing each operator of the neural network model and the mask processing subgraph of the operator. The mask data set receiving module is configured to receive a mask data set in response to a mask input operation of a user; the mask data set contains at least one mask. The merging module is configured to perform a transformation operation on the mask in the mask data set according to the mask transformation subgraph in the data zero-clearing graph, and replace an operator in the original graph that needs to perform data zero-clearing with the mask zero-clearing subgraph of the operator, to achieve merging of the data zero-clearing graph and the original graph. The merging module is specifically configured to: for each mask in the mask data set, determine each mask transformation subgraph corresponding to the mask from the data zero-clearing graph; and perform a transformation operation on the mask by using the mask transformation subgraphs. The target neural network model generation module is configured to generate a target neural network model according to a model graph obtained by merging the data zero-clearing graph and the original graph; the target neural network model is used to calculate variable-length input data corresponding to the mask.

7. The apparatus of claim 6, wherein, The data zero-clearing graph optimization module is configured to: perform an optimization operation on the data zero-clearing graph; the optimization operation includes deleting an operator in the data zero-clearing graph that does not have a corresponding mask zero-clearing subgraph.

8. A device for modifying a fixed-length input neural network model, the device comprising: The application is applied to speech recognition and image recognition, and includes: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: acquire neural network model data; the neural network model data contains an original graph of a neural network model; the neural network model is a neural network model that supports only fixed-length input; obtaining a mask processing subgraph of each operator of the neural network model from a zero clearing operation class pre-registered and stored in a preset database for each operator of the neural network model; the mask processing subgraph includes a mask transformation subgraph and a mask zero clearing subgraph; the mask transformation subgraph is used to represent changes of a mask after the mask passes through the mask transformation subgraph corresponding to the operator; the mask zero clearing subgraph is used to represent a calculation process performed by the operator for data zero clearing after the mask inputs the mask zero clearing subgraph corresponding to the operator; generating a data zero clearing graph including each operator of the neural network model and the mask processing subgraph of the operator; in response to a mask input operation of a user, receiving a mask data set; the mask data set includes at least one mask; performing a transformation operation on the mask in the mask data set according to the mask transformation subgraph in the data zero clearing graph, and replacing an operator in the original graph that needs to perform data zero clearing with a mask zero clearing subgraph of the operator, to realize merging of the data zero clearing graph and the original graph; wherein the transformation operation on the mask in the mask data set according to the mask transformation subgraph in the data zero clearing graph specifically includes: for each mask in the mask data set, determining each mask transformation subgraph corresponding to the mask from the data zero clearing graph; and performing a transformation operation on the mask using the mask transformation subgraph; generating a target neural network model according to a model graph merged from the data zero clearing graph and the original graph; the target neural network model is used to calculate variable-length input data corresponding to the mask.

Images