Plug-in processing method and apparatus, data processing method and apparatus, and electronic device

By plug-in processing of the neural network model deployed on the AI ​​chip, the problem of inconsistency in processing logic caused by inconsistency in encoding language and data types is solved, the deployment process is simplified and efficiency is improved, and the model is efficiently reused.

WO2024230527A1PCT designated stage expired Publication Date: 2024-11-14NANJING HORIZON INFORMATION TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/090273
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-28
Filing Date
2024-04-28
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

When deploying neural network models using AI chips, the coding language and data types used on the deployment side and the algorithm side are inconsistent, resulting in inconsistent processing logic and inefficient model deployment.

Method used

By performing plug-in processing on the neural network model, determining the plug-in to be released and performing consistency verification, ensuring that the same set of code is used for data processing on the deployment side and the algorithm side, simplifying the deployment process and improving the efficiency of model deployment.

Benefits of technology

It realizes consistency of processing logic, simplifies the model deployment process, improves the model deployment efficiency, and supports the multiplexing of functionally decoupled neural network models in different application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024090273_14112024_PF_FP_ABST
    Figure CN2024090273_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present disclosure are a plug-in processing method and apparatus, a data processing method and apparatus, and an electronic device. The plug-in processing method comprises: determining a plug-in to be published corresponding to a neural network model; processing test input data on the basis of said plug-in and the neural network model, so as to obtain reference output data; determining test output data obtained by means of processing the test input data by a neural network processor on the basis of said plug-in; and publishing said plug-in on the basis of a consistency verification result determined on the basis of the test output data and the reference output data. In the present disclosure, when a consistency verification result indicates that verification is passed, a plug-in is published, such that when a neural network processor processes input feature data on the basis of the published plug-in, it can be ensured that a deployment side and an algorithm side use the same set of codes for data processing, and therefore it is guaranteed that processing logic is consistent; in addition, the published plug-in is used, such that the model deployment efficiency can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Plug-in processing method and device, data processing method and device, and electronic equipment

[0001] This disclosure claims priority to Chinese patent application number 202310533187.0, filed on May 11, 2023, entitled “Plug-in processing method and device, data processing method and device, electronic device”, and Chinese patent application number 202310780449.3, filed on June 28, 2023, entitled “Plug-in processing method and device, data processing method and device, electronic device”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of artificial intelligence technology, and in particular to a plug-in processing method and device, a data processing method and device, and an electronic device. Background Art

[0003] Currently, artificial intelligence (AI) chips are widely used in many application scenarios, including intelligent driving and image recognition. Typically, neural network models can be deployed in AI chips to process data in different application scenarios.

[0004] Summary of the Invention

[0005] When using a neural network model deployed on an AI chip for data processing, the coding languages ​​used by the deployment side and the algorithm side when writing the processing code are inconsistent in most cases, and the AI ​​chip on the deployment side only supports fixed-point models, while the neural network model on the algorithm side is generally a floating-point model, resulting in inconsistent data types processed by the deployment side and the algorithm side. The inconsistency in coding language and data type leads to inconsistent processing logic on the deployment side and the algorithm side. In order to solve the above technical problems, the present disclosure provides a plug-in processing method and device, a data processing method and device, and an electronic device.

[0006] According to one aspect of an embodiment of the present disclosure, a plug-in processing method for a neural network model is provided, including: determining a plug-in to be released corresponding to the neural network model; processing test input data based on the plug-in to be released and the neural network model to obtain reference output data; then determining test output data, which is obtained by a neural network processor processing the test input data based on the plug-in to be released; and finally, publishing the plug-in to be released based on a consistency check result determined based on the test output data and the reference output data.

[0007] According to another aspect of an embodiment of the present disclosure, a data processing method is provided, including: first determining the type of input feature data; based on the type of the input feature data, loading a published plug-in, where the published plug-in is a plug-in corresponding to a neural network model deployed on a neural network processor; and then processing the input feature data based on the published plug-in and the neural network model to obtain output feature data.

[0008] According to another aspect of the embodiments of the present disclosure, a plug-in processing device for a neural network model is provided, including: a first determination module for determining a plug-in to be released corresponding to the neural network model; a first processing module for processing test input data based on the plug-in to be released and the neural network model determined by the first determination module to obtain reference output data; a second determination module for determining test output data obtained by the neural network processor by processing the test input data based on the plug-in to be released; and a publishing module for publishing the plug-in to be released based on a consistency verification result determined by the test output data determined by the second determination module and the reference output data obtained by the processing module.

[0009] According to another aspect of an embodiment of the present disclosure, a data processing device is provided, including: a first determination module for determining the type of input feature data; a loading module for loading a published plug-in corresponding to a neural network model deployed on a neural network processor based on the type of input feature data determined by the first determination module; and a first processing module for processing the input feature data based on the published plug-in and the neural network model loaded by the loading module to obtain output feature data.

[0010] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program for executing the plug-in processing or data processing method of the neural network model of any of the above embodiments of the present disclosure.

[0011] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, which includes: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the plug-in processing or data processing method of the neural network model of any of the above embodiments of the present disclosure.

[0012] According to another aspect of the embodiments of the present disclosure, a computer program product is provided. When an instruction processor in the computer program product is executed, the plug-in processing or data processing method of the neural network model of any of the above embodiments of the present disclosure is performed.

[0013] Based on the solution provided by the embodiment of the present disclosure, a plug-in to be released is obtained by developing the pre- and post-processing of the neural network model as a plug-in, and the plug-in to be released is released when the consistency check is determined to have passed. When the neural network processor processes the input feature data based on the published plug-in, it can ensure that the deployment side and the algorithm side use the same set of code for data processing, thereby ensuring consistent processing logic. Moreover, the use of the published plug-in eliminates the need to compile the pre- and post-processing logic during the model deployment process, which can simplify the deployment process and thus improve the efficiency of model deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0015] FIG1 is a schematic diagram of the structure of a plug-in processing system provided by an exemplary embodiment of the present disclosure;

[0016] FIG2 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure;

[0017] FIG3 is a schematic structural diagram of an electronic device provided by another exemplary embodiment of the present disclosure;

[0018] FIG4 is a flow chart of a plug-in processing method for a neural network model provided by an exemplary embodiment of the present disclosure;

[0019] FIG5 is a flow chart of a plug-in processing method for a neural network model provided by another exemplary embodiment of the present disclosure;

[0020] FIG6 is a flow chart of a plug-in processing method for a neural network model provided by another exemplary embodiment of the present disclosure;

[0021] FIG7 is a flow chart of a plug-in processing method for a neural network model provided by yet another exemplary embodiment of the present disclosure;

[0022] FIG8 is a schematic diagram of an interaction flow of a plug-in processing system for a neural network model provided by an exemplary embodiment of the present disclosure;

[0023] FIG9 is a schematic diagram of an interaction flow of a plug-in processing system for a neural network model provided by another exemplary embodiment of the present disclosure;

[0024] FIG10 is a flow chart of a data processing method provided by an exemplary embodiment of the present disclosure;

[0025] FIG11 is a flow chart of a data processing method provided by another exemplary embodiment of the present disclosure;

[0026] FIG12 is a flow chart of a data processing method provided by yet another exemplary embodiment of the present disclosure;

[0027] FIG13 is a flow chart of a data processing method provided by yet another exemplary embodiment of the present disclosure;

[0028] FIG14 is a flow chart of a data processing method provided by yet another exemplary embodiment of the present disclosure;

[0029] FIG15 is a flow chart of a data processing method provided by yet another exemplary embodiment of the present disclosure;

[0030] FIG16 is a schematic diagram of the structure of a plug-in processing device for a neural network model provided by an exemplary embodiment of the present disclosure;

[0031] FIG17 is a schematic structural diagram of a data processing device provided by an exemplary embodiment of the present disclosure;

[0032] FIG18 is a schematic structural diagram of an electronic device provided by yet another exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0034] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0035] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meanings, nor do they indicate a necessary logical order between them.

[0036] It should also be understood that in the embodiments of the present disclosure, “a plurality of” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0037] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0038] In addition, the term "and / or" in this disclosure is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this disclosure generally indicates that the related objects are in an "or" relationship.

[0039] It should also be understood that the description of the various embodiments in this disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced with each other. For the sake of brevity, they will not be described one by one.

[0040] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0041] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0042] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0043] Overview of the Disclosure

[0044] After R&D personnel complete the development of a neural network model that implements a specific function on R&D equipment, they need to deploy it on an AI chip. Secondary developers then integrate the AI ​​chip to expand its functionality and apply it to various scenarios. The device where R&D personnel and developers write code can be called the algorithm-side device, and the AI ​​chip that runs the code can be called the deployment-side device. When developers use the neural network model deployed on the AI ​​chip to process data, the coding language they use may differ from the coding language written and deployed by the R&D personnel on the AI ​​chip. For example, the AI ​​chip uses C while the developer uses Matlab. This results in the deployment side (i.e., the deployment-side device) and the algorithm side (i.e., the algorithm-side device) using two different coding languages ​​for data processing. This inconsistency in coding languages ​​leads to inconsistencies in the processing code between the deployment side and the algorithm side. Furthermore, when deploying the neural network model on the AI ​​chip, to increase the AI ​​chip's data processing speed, R&D personnel need to compile the "floating-point model" into a "fixed-point model" and then deploy this "fixed-point model" on the AI ​​chip. This results in inconsistencies in the data types processed by the "floating-point model" on the algorithm side and the "fixed-point model" deployed on the AI ​​chip.

[0045] In addition, since different developers may use different coding languages, not only does it require developers and R&D personnel to communicate and translate the code, which is a tedious process, but the pre- and post-processing logic needs to be compiled when deploying the model on the AI ​​chip. The deployment process is complicated, resulting in low model deployment efficiency.

[0046] In order to solve the problem of inconsistent processing logic between the algorithm-side model and the deployment-side model due to inconsistent coding languages ​​and data types, as well as the problem of low model deployment efficiency, the disclosed embodiment provides a plug-in processing method for a neural network model, which develops a plug-in for pre- and post-processing of the neural network model on the algorithm side, and publishes the plug-in when it is determined that the consistency check has passed based on the plug-in to be released. During the application process, when processing the input feature data based on the published plug-in, it can ensure that the deployment side and the algorithm side use the same set of codes for data processing, thereby ensuring the consistency of the processing logic. In addition, by using the published plug-in, there is no need to compile the pre- and post-processing logic during the model deployment process, which can simplify the deployment process and thus improve the model deployment efficiency.

[0047] Exemplary Systems

[0048] Figure 1 is a schematic diagram of the structure of a plug-in processing system provided by an exemplary embodiment of the present disclosure. As shown in Figure 1, the system includes a plug-in development device 10 and a neural network processor 11. Plug-in development device 10, also known as an algorithm-side device, is used to develop plug-ins to be released and publish them when verification is confirmed. In some examples, plug-in development device 10 includes, but is not limited to, a personal computer, a server computer, a multiprocessor system, a microprocessor-based system, and the like.

[0049] The neural network processor 11, which may also be referred to as a deployment-side device, is used to process input feature data through a published plug-in published by the plug-in development device 10 to obtain output feature data.

[0050] FIG2 is a schematic structural diagram of a plug-in development device 10 provided in an exemplary embodiment of the present disclosure. As shown in FIG2 , the plug-in development device 10 includes at least one processor 21 and a memory 22 .

[0051] The processor 21 is used to execute the plug-in processing method of the neural network model in the embodiment of the present disclosure. The processor 21 can be a high-performance central processing unit (CPU), graphics processing unit (GPU) or other processor that supports plug-in publishing.

[0052] The memory 22 is used to store the plug-in to be published. The processor 21 can execute the operation of publishing the plug-in to be published to obtain the published plug-in to implement the following data processing method.

[0053] In some examples, as shown in FIG2 , the plug-in development device 10 may further include an input device 23 and an output device 24. The input device 23 is used to input the program code of the plug-in to be published, test input data, etc. The output device 24 is used to view the publishing results or intermediate data in the plug-in processing.

[0054] It should be noted that the present disclosure does not limit the specific structure of the plug-in development device 10, and the plug-in development device 10 may include more or fewer components than those shown in Figure 2. For example, the plug-in development device 10 may also include components such as a bus and an input / output interface.

[0055] FIG3 is a schematic diagram of the structure of a neural network processor 11 provided by an exemplary embodiment of the present disclosure. As shown in FIG3 , the neural network processor 11 is, for example, but not limited to, a processor that supports plug-in-based neural network-related calculations, such as a GPU, a brain processing unit (BPU), and a tensor processing unit (TPU). The neural network processor 11 may include: a control unit 31 and a computing unit 32. The control unit 31 is configured to execute instructions to control the computing unit 32 to perform the following data processing method. In some examples, the control unit 31 is configured to control the computing unit 32 to perform corresponding operations according to instructions executable by the neural network model deployed on the neural network processor.

[0056] The computing unit 32 includes an operation circuit, which can be configured to perform processing logic and neural network operation operations. The processing logic includes but is not limited to pre-processing logic and post-processing logic. The operation operation includes but is not limited to any one or more of convolution operations, pooling operations, activation function operations, etc.

[0057] In some examples, the neural network processor 11 may further include a buffer memory 33 and an off-chip memory 34. The buffer memory 33 may include one or more independent cache memories or processing units with data caching capabilities, and may access the off-chip memory 34 under the control of the control unit 31. In some examples, the buffer memory 31 may be a fast, small-capacity static access memory, such as a static random access memory (SRAM). Exemplarily, the buffer memory 33 may be configured to receive published plug-ins and input feature data from the off-chip memory 34 and cache them for use by the computing unit 32, and may also be configured to receive and cache output feature data output by the computing unit 32, and output the output feature data to the off-chip memory 34.

[0058] The off-chip memory 34 may include one or more independent memories or processing units with data storage capabilities, and can be accessed by the buffer memory 33 under the control of the control unit 31. In some examples, the off-chip memory 34 is a slow, large-capacity dynamic access memory, such as dynamic random access memory (DRAM). In some examples, the off-chip memory 34 is a double data rate (DDR) memory.

[0059] It should be noted that the embodiments of the present disclosure do not limit the specific structure of the neural network processor 11, and the neural network processor 11 may include more or fewer components than those shown in Figure 3. For example, the neural network processor 11 may also include components such as a bus.

[0060] Exemplary Methods

[0061] The methods of the following embodiments of the present disclosure can be applied to many fields that require neural network calculations, such as image processing and intelligent driving. For example, by executing the methods provided in the embodiments of the present disclosure, image data can be processed in the image processing field.

[0062] Figure 4 is a flow chart illustrating a method for processing a neural network model as a plug-in, according to an exemplary embodiment of the present disclosure. This method can be applied to plug-in development device 10. To distinguish the neural network model in plug-in development device 10 from the neural network model in neural network processor 11, in the following embodiments, the neural network model in plug-in development device 10 is represented by a first neural network model, and the neural network model in neural network processor 11 is represented by a second neural network model. As shown in Figure 4, this method includes steps 401 through 404.

[0063] Step 401: Determine the plug-in to be released corresponding to the first neural network model.

[0064] The embodiments of the present disclosure can be executed by a plug-in development device. The first neural network model is a model that requires pre-processing and / or post-processing in the plug-in development device. The embodiments of the present disclosure do not limit the data types that the first neural network model can process. In some examples, the first neural network model can be a floating-point model that can process floating-point data, where floating-point data includes single-precision floating-point numbers (float type) and double-precision floating-point numbers (double type).

[0065] During the development phase, developers define generic classes for pre- and post-processing. These generic classes, also known as generic classes, are classes that serve as templates. Then, based on the actual application scenario, the generic classes are inherited and rewritten, and processing codes suitable for the actual application scenario are customized. The processing codes include pre-processing code for pre-processing input data and post-processing code for post-processing data output by the first neural network model. The pre-processing code is used to implement pre-processing logic, and the post-processing code is used to implement post-processing logic. The plug-in development device generates a dynamic library based on the customized pre-processing logic and post-processing logic, and uses the dynamic library as a plug-in to be released. The plug-in to be released is the plug-in to be released corresponding to the first neural network model.

[0066] In some examples, after the plug-in development device determines the plug-in to be released, it can send the plug-in to the neural network processor. Exemplarily, the neural network processor includes but is not limited to a BPU, a TPU, a GPU, etc.

[0067] Step 402: Process the test input data based on the plug-in to be released and the first neural network model to obtain reference output data.

[0068] The plug-in development device receives input test input data, processes the test input data using the plug-in to be released and the first neural network model, and obtains reference output data.

[0069] In some examples, the plug-in to be released may include at least one processing plug-in. For example, when unifying the processing logic of the neural network processor and the plug-in development device, if a certain processing logic needs to be unified, the plug-in to be released includes a processing plug-in corresponding to the processing logic. If multiple processing logics need to be unified, the plug-in to be released includes multiple processing plug-ins corresponding to the multiple processing logics. The embodiment of the present disclosure does not limit the number of processing plug-ins included in the plug-in to be released, and the number of processing plug-ins can be determined according to the processing logic of the neural network processor.

[0070] Step 403: Determine test output data obtained by the neural network processor by processing the test input data based on the plug-in to be released.

[0071] In some examples, the plug-in development device receives test output data sent by the neural network processor, where the test output data is data obtained by the neural network processor processing test input data based on the plug-in to be released.

[0072] For example, after receiving the plug-in to be released and test input data from the plug-in development device, the neural network processor can process the test input data based on the plug-in to be released and the second neural network model deployed on the neural network processor to obtain test output data. The test output data is then sent to the plug-in development device.

[0073] Step 404: Publish the plug-in to be published based on the consistency check result determined between the test output data and the reference output data.

[0074] In some examples, the plug-in development device can perform a consistency check on the test output data and the reference output data, or the neural network processor can perform a consistency check on the test output data and the reference output data. It should be noted that if the neural network processor performs a consistency check on the test output data and the reference output data, the plug-in development device does not need to perform step 403 above, and the neural network processor sends the consistency check result to the plug-in development device. Figure 4 illustrates an example of a plug-in development device performing a consistency check on the test output data and the reference output data.

[0075] Exemplarily, after receiving the test output data, the plug-in development device calculates the similarity between the test output data and the reference output data, and determines a consistency check result based on the similarity and a preset similarity threshold.

[0076] If the similarity between the test output data and the reference output data is greater than the preset similarity threshold, it indicates that the result obtained by the neural network processor based on the plug-in to be released and the fixed-point model (the second neural network model) is basically consistent with the result obtained by the plug-in development device based on the plug-in to be released and the floating-point model (the first neural network model). The consistency check result is determined to be passed. When the consistency check result is passed, the plug-in development device can release the plug-in to be released. In actual applications, the published plug-in can be stored in an external memory after release, and the neural network processor processes the input feature data by loading the published plug-in stored in the external memory.

[0077] If the similarity between the test output data and the reference output data is less than or equal to a preset similarity threshold, the consistency check result is determined to be a failure, and the plugin to be published cannot be published. If the consistency check result is determined to be a failure, the plugin development device can adjust the processing code of the plugin to be published, and then re-execute steps 402 to 404 based on the adjusted plugin to be published until the consistency check result is a passing result, and then publish the adjusted plugin to be published.

[0078] In some examples, the plug-in development device can adjust the processing code of the plug-in to be released, including but not limited to modifying the processing logic of the plug-in to be released. For example, when performing image processing, the device can adjust image feature parameters or modify the extraction algorithm to re-extract image features to generate an adjusted plug-in to be released. For another example, the device can modify the output data type of the first neural network model to generate an adjusted plug-in to be released. The disclosed embodiments do not limit how the plug-in development device adjusts the plug-in to be released.

[0079] The plug-in processing method of the neural network model provided by the embodiment of the present disclosure is to obtain a plug-in to be released by developing the pre- and post-processing of the first neural network model as a plug-in, and to release the plug-in to be released when the consistency check is determined to have passed based on the plug-in to be released. During the application process, when the input feature data is processed based on the published plug-in, it can be ensured that the deployment side and the algorithm side use the same set of codes for data processing, thereby ensuring the consistency of the processing logic. In addition, by using the published plug-in, there is no need to compile the pre- and post-processing logic during the process of deploying the model, which can simplify the deployment process and thus improve the efficiency of model deployment.

[0080] In some examples, as shown in FIG. 5 , step 401 may include step 4011 and step 4012 .

[0081] Step 4011, determine the processing logic of the first neural network model.

[0082] When developers determine they need to develop plugins for pre- and post-processing of the first neural network model, they first define a general class containing a basic general model for parsing basic information. They then inherit and rewrite the general class based on the actual application scenario, customizing the processing code appropriate for the scenario. This processing code includes pre-processing code for pre-processing input data and post-processing code for post-processing data output by the first neural network model. The pre-processing code implements the pre-processing logic, while the post-processing code implements the post-processing logic.

[0083] The plug-in development device obtains the pre-processing code and the post-processing code to obtain the pre-processing logic and the post-processing logic corresponding to the first neural network model.

[0084] Step 4012: Based on the processing logic of the first neural network model, generate a plug-in to be released corresponding to the first neural network model.

[0085] The plug-in development device generates a dynamic library according to the customized pre-processing logic and post-processing logic, and uses the dynamic library as the plug-in to be released. In this way, the plug-in development device obtains the plug-in to be released corresponding to the first neural network model.

[0086] Since the input and output data of the first neural network model are feature matrix data, the image, audio or video to be processed input by the user needs to be pre-processed by feature extraction, data type conversion, etc. to obtain the data input to the first neural network model. Then the data is input to the first neural network model for processing, and the processed data is output. The processed data is feature data, and the processed data needs to be post-processed by feature recognition, data type conversion, etc. Based on this, the processing plug-in corresponding to the pre-processing is used as the first processing plug-in, and the processing plug-in corresponding to the post-processing is used as the second processing plug-in. The first processing plug-in and the second processing plug-in constitute the plug-in to be released. In one implementation, as shown in Figure 6, the step 4012 of generating the plug-in to be released may include steps 40121 and 40122.

[0087] Step 40121: Generate a first processing plug-in in the plug-in to be released based on the pre-processing logic in the processing logic of the first neural network model.

[0088] According to the input and pre-processing logic of the first neural network model, a first processing plug-in is generated, and the output data of the first processing plug-in serves as the input data of the first neural network model. In practical applications, plug-ins with different pre-processing logics can be developed for the first neural network model based on the difference in pre-processing logic or data, and these plug-ins can serve as first processing plug-ins. For example, when image data is converted into feature matrix data, a first processing plug-in for converting image data in .jpg format can be generated, or a first processing plug-in for converting image data in .png format can be generated. For another example, a first processing plug-in for converting single-frame image data can be generated, or a first processing plug-in for converting video data (multi-frame image data) can be generated. Therefore, the embodiment of the present disclosure does not limit the number of plug-ins for the first processing plug-in.

[0089] Step 40122: Generate a second processing plug-in in the plug-in to be released based on the post-processing logic in the processing logic of the first neural network model.

[0090] Based on the output of the first neural network model and the post-processing logic, a second processing plug-in is generated, and the output data of the first neural network model is used as the input data of the second processing plug-in. The embodiment of this disclosure does not limit the number of plug-ins of the second processing plug-in.

[0091] It should be noted that during the plug-in development phase, it is necessary to develop a first processing plug-in and a second processing plug-in for pre-processing and post-processing respectively. After the plug-in is released, during the application process, the first processing plug-in and / or the second processing plug-in can be selected for data processing according to actual conditions. For example, when the second neural network model deployed on the AI ​​chip already contains the pre-processing logic corresponding to the pre-processing, the pre-processing logic of the model can be used for data processing without using the first processing plug-in. Accordingly, there is no need to load the first processing plug-in, thus achieving plug-and-play of the plug-in and flexible loading on demand.

[0092] In some examples, as shown in FIG. 7 , step 402 may include steps 4021 to 4023 .

[0093] Step 4021: Process the test input data based on the first processing plug-in to obtain reference intermediate data.

[0094] In some examples, the test input data may be a test image, test audio, or test video. The test input data is pre-processed using the pre-processing logic corresponding to the first processing plug-in, such as format conversion, feature extraction, and data type conversion, to obtain reference intermediate data. The reference intermediate data is the data to be input into the first neural network model for processing.

[0095] Step 4022: Process the reference intermediate data based on the first neural network model to obtain model output reference data.

[0096] The reference intermediate data is input into the first neural network model, and is operated on by each network layer in the first neural network model. The operation result is the model output reference data. Exemplarily, the network layer includes but is not limited to a convolutional layer, a pooling layer, and a fully connected layer.

[0097] Step 4023: Process the model output reference data based on the second processing plug-in to obtain reference output data.

[0098] The model output reference data is post-processed using the post-processing logic corresponding to the second processing plug-in, such as feature recognition and data type conversion, to obtain reference output data. The reference output data is used to perform a consistency check on the obtained test output data, which is obtained by the neural network processor based on the test input data, the plug-in to be released, and the second neural network model.

[0099] The embodiment of the present disclosure determines reference output data based on test input data, the plug-in to be released, and the first neural network model, providing a comparison basis for consistency verification operations.

[0100] Figure 8 is a schematic diagram of the interaction flow of a plug-in processing system for a neural network model provided by an exemplary embodiment of the present disclosure. The method is executed by a plug-in development device 10 and a neural network processor 11. As shown in Figure 8, the method includes steps 801 to 807.

[0101] Step 801: The plug-in development device determines a plug-in to be released corresponding to the first neural network model.

[0102] The plug-in development device obtains a plug-in to be released developed by a developer for the first neural network model. During the development stage, the developer defines a general class for pre- and post-processing parsing, and then inherits and rewrites the general class according to the actual application scenario, and customizes the processing code suitable for the actual application scenario. The processing code includes a pre-processing code for pre-processing the input data and a post-processing code for post-processing the data output by the first neural network model. The pre-processing code is used to implement the pre-processing logic, and the post-processing code is used to implement the post-processing logic. The plug-in development device generates a dynamic library based on the customized pre-processing logic and post-processing logic, and uses the dynamic library as the plug-in to be released. The plug-in to be released is the plug-in to be released corresponding to the first neural network model.

[0103] In step 802, the plug-in development device processes the test input data based on the plug-in to be released and the first neural network model to obtain reference output data.

[0104] The plug-in to be released includes a first processing plug-in and a second processing plug-in. The first processing plug-in is a plug-in corresponding to the pre-processing logic, also called a pre-processing plug-in; the second processing plug-in is a plug-in corresponding to the post-processing logic, also called a post-processing plug-in.

[0105] The plug-in development device receives test input data from a developer and processes the test input data using the to-be-released plug-in and the first neural network model to obtain reference output data. In one implementation, the plug-in development device first processes the test input data based on the first processing plug-in to obtain reference intermediate data; then processes the reference intermediate data based on the first neural network model to obtain model output reference data; and finally, processes the model output reference data based on the second processing plug-in to obtain reference output data.

[0106] Step 803: The plug-in development device sends the plug-in to be released and test input data to the neural network processor.

[0107] In step 804, the neural network processor processes the test input data based on the plug-in to be released and the second neural network model to obtain test output data.

[0108] The second neural network model is a fixed-point model deployed on the neural network processor that requires pre-processing and / or post-processing and is capable of processing fixed-point data, including decimal data. The second neural network model implements the same functionality as the first neural network model on the plug-in development device, except that the first neural network model processes floating-point data, while the second neural network model processes fixed-point data. The first neural network model has higher accuracy than the second neural network model, and the second neural network model processes faster than the first neural network model.

[0109] The test input data is identical to the test input data processed by the plug-in development device. The neural network processor receives the plug-in to be released and the test input data from the plug-in development device, and processes the test input data using the plug-in to be released and the second neural network model to obtain test output data. In one implementation, the neural network processor first processes the test input data based on a first processing plug-in included in the plug-in to be released to obtain intermediate test data; then processes the intermediate test data based on a second neural network model to obtain model output test data; and finally, processes the model output test data based on a second processing plug-in included in the plug-in to be released to obtain test output data.

[0110] When determining the test output data, the first processing plug-in and the second processing plug-in used by the neural network processor for processing are respectively the same as the first processing plug-in and the second processing plug-in used by the plug-in development device for processing, and the processed data are also the same, both of which are test input data, but the data processed by the second neural network model is fixed-point data, and there is a certain loss of precision when the data type is converted, which makes the test output data have a certain error relative to the reference output data.

[0111] Step 805: The neural network processor sends test output data to the plug-in development device.

[0112] Step 806 : The plug-in development device determines a consistency check result based on the test output data and the reference output data.

[0113] After receiving the test output data sent by the neural network processor, the plug-in development device calculates the similarity between the test output data and the reference output data. If the similarity between the test output data and the reference output data is greater than a preset similarity threshold, the consistency check result is determined to be a pass, indicating that the error between the result obtained by the neural network processor based on the to-be-released plug-in and the second neural network model for processing the data and the result obtained by the plug-in development device based on the to-be-released plug-in and the first neural network model for processing the data is within the precision loss range. If the similarity between the test output data and the reference output data is less than or equal to the preset similarity threshold, the consistency check result is determined to be a fail.

[0114] Step 807: The plug-in development device publishes the plug-in to be published based on the consistency check result.

[0115] If the plug-in development device determines that the consistency check result is passed, the plug-in to be published is published. If the consistency check result is determined to be failed, the plug-in development device can adjust the processing logic of the plug-in to be published, for example, adjusting the processing method for different types of data, and then re-execute steps 802 to 806 based on the adjusted plug-in to be published until the consistency check result is passed, and then publish the adjusted plug-in to be published.

[0116] In practical applications, the published plug-in can be stored in an external memory after being published, and the neural network processor processes the input feature data by loading the published plug-in stored in the external memory.

[0117] In the plug-in processing system provided by the embodiment of the present disclosure, the neural network processor and the plug-in development device use the same plug-in to be released to respectively determine the test output data and the reference output data, and then the plug-in development device determines the consistency verification result based on the test output data and the reference output data, and releases the plug-in to be released based on the consistency verification result. When the neural network processor uses the published plug-in and the neural network model to process data, the neural network model and the plug-in are independent of each other. On the one hand, based on the plug-in, the deployment side and the algorithm side can use the same set of codes for data processing, which can ensure the consistency of the processing logic; on the other hand, there is no need to compile the pre- and post-processing logic in the process of deploying the model, which can simplify the deployment process, thereby improving the efficiency of model development and deployment; on the other hand, the neural network model with functional decoupling can be reused in different application scenarios, and the plug-in can be plug-and-play, which can simplify the steps of using the neural network model, thereby improving the reuse rate and usage simplification rate of the neural network model.

[0118] Figure 9 is a schematic diagram of the interaction flow of a plug-in processing system for a neural network model provided by another exemplary embodiment of the present disclosure. The method is executed by a plug-in development device 10 and a neural network processor 11. As shown in Figure 9, the method includes steps 901 to 907.

[0119] Step 901: The plug-in development device determines a plug-in to be released corresponding to the first neural network model.

[0120] In step 902, the plug-in development device processes the test input data based on the plug-in to be released and the first neural network model to obtain reference output data.

[0121] Step 903: The plug-in development device sends the plug-in to be released, reference output data, and test input data to the neural network processor.

[0122] In the embodiment of the present disclosure, based on step 803 of the embodiment shown in Figure 8, the plug-in development device also sends reference output data to the neural network processor so that the neural network processor determines the consistency verification result based on the reference output data.

[0123] In step 904, the neural network processor processes the test input data based on the plug-in to be released and the second neural network model to obtain test output data.

[0124] The implementation of the above steps 901, 902 and 904 can refer to the implementation of steps 801, 802 and 804 in the embodiment shown in Figure 8, and will not be repeated here.

[0125] In step 905 , the neural network processor determines a consistency check result based on the test output data and the reference output data.

[0126] After the neural network processor processes the test output data, it calculates the similarity between the test output data and the reference output data. If the similarity between the test output data and the reference output data is greater than a preset similarity threshold, the consistency check result is determined to be a pass, indicating that the error between the result obtained by the neural network processor of processing the data based on the to-be-released plug-in and the second neural network model and the result obtained by the plug-in development device of processing the data based on the to-be-released plug-in and the first neural network model is within the accuracy loss range. If the similarity between the test output data and the reference output data is less than or equal to the preset similarity threshold, the consistency check result is determined to be a fail.

[0127] Step 906: The neural network processor sends the consistency check result to the plug-in development device.

[0128] After the neural network processor determines that a consistency check result is obtained, it returns the consistency check result to the plug-in development device.

[0129] Step 907: The plug-in development device publishes the plug-in to be published based on the consistency check result.

[0130] After the plug-in development device receives the consistency check result sent by the neural network processor, if the consistency check result is determined to be passed, the plug-in to be released is released. If the consistency check result is determined to be failed, the plug-in development device can adjust the processing logic of the plug-in to be released, for example, adjusting the processing method for different types of data, and then re-execute steps 902 to 906 based on the adjusted plug-in to be released until the consistency check result is verified to be passed, and then release the adjusted plug-in to be released.

[0131] In practical applications, the published plug-in can be stored in an external memory after being published, and the neural network processor processes the input feature data by loading the published plug-in stored in the external memory.

[0132] Unlike the embodiment shown in FIG8 , in the embodiment shown in FIG9 , after the neural network processor obtains the test output data, it determines a consistency check result based on the test output data and the reference output data, and sends the consistency check result to the plug-in development device. In contrast, in the embodiment shown in FIG8 , after the neural network processor obtains the test output data, it directly sends the test output data to the plug-in development device, which then determines a consistency check result based on the test output data and the reference output data.

[0133] In the plug-in processing system provided by the embodiment of the present disclosure, the neural network processor and the plug-in development device use the same plug-in to be released to respectively determine the test output data and the reference output data, and then the neural network processor determines the consistency verification result based on the test output data and the reference output data, and the plug-in development device releases the plug-in to be released based on the consistency verification result. When the neural network processor uses the published plug-in and the neural network model to process data, the neural network model and the plug-in are independent of each other. On the one hand, based on the plug-in, the deployment side and the algorithm side can use the same set of codes for data processing, which can ensure the consistency of the processing logic; on the other hand, there is no need to compile the pre- and post-processing logic in the process of deploying the model, which can simplify the deployment process, thereby improving the efficiency of model development and deployment; on the other hand, the neural network model with functional decoupling can be reused in different application scenarios, and the plug-in can be plug-and-play, which can simplify the steps of using the neural network model, thereby improving the reuse rate and usage simplification rate of the neural network model.

[0134] FIG10 is a flow chart of a data processing method provided by an exemplary embodiment of the present disclosure. The method can be applied to a neural network processor 11 . As shown in FIG10 , the method includes steps 1001 to 1003 .

[0135] Step 1001: Determine the type of input feature data.

[0136] The embodiment of the present disclosure can be executed by a neural network processor. After the neural network processor obtains the input feature data to be processed, it determines the type of the input feature data. In the embodiment of the present disclosure, the type of the input feature data includes a first subtype and a second subtype. If the input feature data is raw data that has not been pre-processed, such as image, audio, or video data, the type of the input feature data is determined to be the first subtype. If the input feature data is feature matrix data that can be directly processed by the neural network model, the type of the input feature data is determined to be the second subtype.

[0137] Step 1002: Based on the type of input feature data, load the published plug-in corresponding to the second neural network model deployed on the neural network processor.

[0138] If the type of the input feature data is the first subtype, the input feature data needs to be pre-processed to obtain its corresponding feature matrix data; if the type of the input feature data is the second subtype, the input feature data does not need to be pre-processed and can be directly input into the second neural network model deployed on the neural network processor for processing.

[0139] In the disclosed embodiment, the published plug-in may include a first processing plug-in for pre-processing and a second processing plug-in for post-processing. When pre-processing of input feature data is required, the first processing plug-in and the second processing plug-in are loaded; when pre-processing of the input feature data is not required, only the second processing plug-in needs to be loaded. This can reduce the amount of data loaded and improve processing speed.

[0140] Step 1003: Process the input feature data based on the published plug-in and the second neural network model to obtain output feature data.

[0141] In one implementation, when the input feature data needs to be pre-processed, the input feature data is first pre-processed using the loaded first processing plug-in to obtain intermediate feature data; then the intermediate feature data is input into the second neural network model for processing to obtain model output feature data; finally, the model output feature data is post-processed using the second processing plug-in to obtain output feature data.

[0142] In another implementation, when pre-processing of the input feature data is not required, the input feature data is directly input into the second neural network model for processing to obtain model output feature data; then the model output feature data is post-processed using the second processing plug-in to obtain output feature data.

[0143] In the disclosed embodiment, when data processing is performed using a published plug-in and a neural network model, the neural network model and the plug-in are independent of each other. On the one hand, based on the plug-in, the deployment side and the algorithm side can use the same set of codes for data processing, which can ensure the consistency of the processing logic; on the other hand, there is no need to compile the pre- and post-processing logic during the deployment of the model, which can simplify the deployment process and thus improve the efficiency of model development and deployment; on the other hand, the neural network model with functional decoupling can be reused in different application scenarios, and the plug-in can be plug-and-play, which can simplify the steps of using the neural network model, thereby improving the reuse rate and usage simplification rate of the neural network model.

[0144] The method provided by the embodiment of the present disclosure allows the neural network processor to process input feature data by loading a published plug-in, ensuring that the deployment side and the algorithm side use the same set of codes for data processing, thereby ensuring the consistency of the processing logic; and using the published plug-in can improve the efficiency of model deployment.

[0145] In some examples, as shown in FIG. 11 , step 1002 may include step 1002 a 1 and step 1002 a 2 .

[0146] Step 1002a1: Based on the type of the input feature data, determine whether the input feature data needs to be pre-processed.

[0147] In the disclosed embodiment, whether pre-processing of the input feature data is required is determined based on the type of the input feature data. If the input feature data type is the first subtype, pre-processing of the input feature data is determined to be required; if the input feature data type is the second subtype, pre-processing of the input feature data is determined not to be required. The following describes two cases.

[0148] In the first case, when it is determined that the input feature data needs to be pre-processed:

[0149] In one implementation, the first processing plug-in and the second processing plug-in can be directly loaded, and the input feature data can be pre- and post-processed based on the first processing plug-in and the second processing plug-in. This method has the advantage that the neural network processor does not need to determine whether the second neural network model has its own pre- and post-processing logic.

[0150] In another implementation method, based on the inclusion relationship between the second neural network model and the pre-processing logic, as well as the inclusion relationship between the second neural network model and the post-processing logic, when it is determined that the first processing plug-in or the second processing plug-in needs to be loaded, the loading operation of the corresponding plug-in can be executed, thereby realizing flexible loading of the plug-in.

[0151] 1) When the inclusion relationship between the second neural network model and the pre-processing logic is that the second neural network model does not include the pre-processing logic, and the inclusion relationship between the second neural network model and the post-processing logic is that the second neural network model includes the post-processing logic, execute step 1002a2 in the embodiment shown in Figure 11.

[0152] 2) When the inclusion relationship between the second neural network model and the pre-processing logic is that the second neural network model does not include the pre-processing logic, and the inclusion relationship between the second neural network model and the post-processing logic is that the second neural network model does not include the post-processing logic, execute step 1002b2 in the embodiment shown in Figure 12.

[0153] 3) When the inclusion relationship between the second neural network model and the pre-processing logic is that the second neural network model includes the pre-processing logic, and the inclusion relationship between the second neural network model and the post-processing logic is that the second neural network model does not include the post-processing logic, execute step 1002c2 in the embodiment shown in Figure 13.

[0154] The second case is when it is determined that no pre-processing of the input feature data is required:

[0155] In one implementation, the second processing plug-in can be directly loaded, and the input feature data can be post-processed based on the second processing plug-in. This method has the advantage that the neural network processor does not need to execute retrieval logic. However, under the premise that the second neural network model has its own post-processing logic, the loaded second processing plug-in is still used for data processing, which will reduce the processing speed of the AI ​​chip.

[0156] In another implementation, based on the inclusion relationship between the second neural network model and the post-processing logic, when it is determined that the second processing plug-in needs to be loaded, the loading operation of the second processing plug-in can be executed, thereby realizing flexible loading of the plug-in.

[0157] 1) When the inclusion relationship between the second neural network model and the post-processing logic is that the second neural network model includes the post-processing logic, there is no need to load an external plug-in. The second neural network model can use its own post-processing logic to process the input feature data. In actual implementation, the input feature data is first processed based on the second neural network model to obtain model output feature data; then the model output feature data is processed based on the post-processing logic included in the second neural network model to obtain output feature data.

[0158] 2) When the inclusion relationship between the second neural network model and the post-processing logic is that the second neural network model does not include the post-processing logic, execute step 1002d2 in the embodiment shown in Figure 14.

[0159] Step 1002a2: Based on the inclusion relationship between the second neural network model and the pre-processing logic of the pre-processing, load the first processing plug-in in the published plug-ins.

[0160] In the embodiment of the present disclosure, the second neural network model does not include pre-processing logic, but includes post-processing logic. In this case, the step of loading the first processing plug-in is executed, and the step of loading the second processing plug-in is not required. After the first processing plug-in is loaded, step 1003a1 is executed.

[0161] In this embodiment, as shown in FIG11 , the above step 1003 may include steps 1003 a 1 to 1003 a 3 .

[0162] Step 1003a1: Based on the first processing plug-in, the input feature data is processed to obtain intermediate feature data.

[0163] In the embodiment of the present disclosure, the second neural network model does not include pre-processing logic, so it is necessary to load the first processing plug-in. After loading is completed, the first processing plug-in is used to process the input feature data to obtain intermediate feature data.

[0164] Step 1003a2: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0165] Step 1003a3: Based on the post-processing logic included in the second neural network model, the model output feature data is processed to obtain output feature data.

[0166] In the embodiment of the present disclosure, the second neural network model includes post-processing logic, so there is no need to load the second processing plug-in. The model output feature data is processed using the post-processing logic of the second neural network model to obtain the output feature data.

[0167] In the embodiment of the present disclosure, it is first determined whether the second neural network model deployed on the neural network processor includes pre-processing logic and post-processing logic, that is, it is determined whether the second neural network model has its own pre-processing logic and post-processing logic. If it only includes post-processing logic, there is no need to execute the step of loading the second processing plug-in, and only the first processing plug-in needs to be loaded. This can reduce the amount of loaded data, thereby improving the processing speed.

[0168] In some examples, as shown in FIG. 12 , step 1002 may include steps 1002 b 1 to 1002 b 3 .

[0169] Step 1002b1: Based on the type of the input feature data, determine whether the input feature data needs to be pre-processed.

[0170] For the implementation of step 1002b1, please refer to the detailed description of step 1002a1 in the embodiment shown in FIG11 .

[0171] Step 1002b2: Based on the inclusion relationship between the second neural network model and the pre-processing logic of the pre-processing, load the first processing plug-in in the published plug-ins.

[0172] Step 1002b3: Based on the inclusion relationship between the second neural network model and the post-processing logic corresponding to the second neural network model, load the second processing plug-in in the published plug-ins.

[0173] In the embodiment of the present disclosure, the second neural network model does not include pre-processing logic and does not include post-processing logic. At this time, the steps of loading the first processing plug-in and the second processing plug-in are performed. After the first processing plug-in and the second processing plug-in are loaded, step 1003b1 is performed.

[0174] In this embodiment, as shown in FIG12 , the above step 1003 may include steps 1003b1 to 1003b3.

[0175] Step 1003b1: Based on the first processing plug-in, the input feature data is processed to obtain intermediate feature data.

[0176] In the embodiment of the present disclosure, the second neural network model does not include pre-processing logic, so it is necessary to load the first processing plug-in. After loading is completed, the first processing plug-in is used to process the input feature data to obtain intermediate feature data.

[0177] Step 1003b2: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0178] Step 1003b3: Process the model output feature data based on the second processing plug-in to obtain output feature data.

[0179] In the embodiment of the present disclosure, the second neural network model does not include post-processing logic, so it is necessary to load a second processing plug-in. After loading is completed, when the second neural network model processes the intermediate feature data to obtain model output feature data, the second processing plug-in is used to process the model output feature data to obtain output feature data.

[0180] In the embodiment of the present disclosure, it is first determined whether the second neural network model deployed on the neural network processor contains pre-processing logic and post-processing logic, that is, it is determined whether the second neural network model has its own pre-processing logic and post-processing logic. When neither is contained, the loading operation of the first processing plug-in and the second processing plug-in is executed, which can realize flexible loading of plug-ins.

[0181] In some examples, as shown in FIG. 13 , step 1002 may include step 1002c1 and step 1002c2 .

[0182] Step 1002c1: Based on the type of the input feature data, determine whether the input feature data needs to be pre-processed.

[0183] For the implementation of step 1002c1, please refer to the detailed description of step 1002a1 in the embodiment shown in FIG11 .

[0184] Step 1002c2: Based on the inclusion relationship between the second neural network model and the post-processing logic corresponding to the second neural network model, load the second processing plug-in in the published plug-ins.

[0185] In the disclosed embodiment, the second neural network model includes pre-processing logic but not post-processing logic. Therefore, there is no need to execute the step of loading the first processing plug-in. Only the step of loading the second processing plug-in needs to be executed. After the second processing plug-in is loaded, step 1003c1 is executed.

[0186] In this embodiment, as shown in FIG13 , the above step 1003 may include steps 1003c1 to 1003c3.

[0187] Step 1003c1: Process the input feature data based on the pre-processing logic included in the second neural network model to obtain intermediate feature data.

[0188] In the embodiment of the present disclosure, the second neural network model includes pre-processing logic, so there is no need to load the first processing plug-in. The pre-processing logic of the second neural network model is used to process the input feature data to obtain intermediate feature data.

[0189] Step 1003c2: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0190] Step 1003c3: Process the model output feature data based on the second processing plug-in to obtain output feature data.

[0191] In the embodiment of the present disclosure, the second neural network model does not include post-processing logic, so it is necessary to load a second processing plug-in. After loading is completed, when the second neural network model processes the intermediate feature data to obtain model output feature data, the second processing plug-in is used to process the model output feature data to obtain output feature data.

[0192] In the embodiment of the present disclosure, it is first determined whether the second neural network model deployed on the neural network processor includes pre-processing logic and post-processing logic, that is, it is determined whether the second neural network model has its own pre-processing logic and post-processing logic. If it only includes pre-processing logic, there is no need to execute the step of loading the first processing plug-in, and only the second processing plug-in needs to be loaded. This can reduce the amount of loaded data, thereby improving the processing speed.

[0193] In some examples, as shown in FIG. 14 , step 1002 may include step 1002 d 1 and step 1002 d 2 .

[0194] Step 1002d1: Based on the type of the input feature data, determine that the input feature data does not need to be pre-processed.

[0195] For the implementation of step 1002d1, please refer to the detailed description of step 1002a1 in the embodiment shown in FIG11 .

[0196] Step 1002d2: Based on the inclusion relationship between the second neural network model and the post-processing logic corresponding to the second neural network model, load the second processing plug-in in the published plug-ins.

[0197] In the disclosed embodiment, the input feature data does not require pre-processing, and the second neural network model does not include post-processing logic. Therefore, there is no need to load the first processing plug-in, and only the step of loading the second processing plug-in needs to be performed. After the second processing plug-in is loaded, step 1003d1 is executed.

[0198] In this embodiment, as shown in FIG14 , the above step 1003 may include steps 1003 d 1 to 1003 d 3 .

[0199] Step 1003d1: Process the input feature data based on the second neural network model to obtain model output feature data.

[0200] Step 1003d2: Process the model output feature data based on the second processing plug-in to obtain output feature data.

[0201] In the embodiment of the present disclosure, the second neural network model does not include post-processing logic, so it is necessary to load a second processing plug-in. After loading is completed, when the second neural network model processes the input feature data to obtain model output feature data, the second processing plug-in is used to process the model output feature data to obtain output feature data.

[0202] In some examples, step 1002d2 can also be performed after step 1003d1, that is, after the second neural network model processes the input feature data to obtain model output feature data, the second processing plug-in is loaded. After loading is completed, the second processing plug-in is used to process the model output feature data to obtain output feature data.

[0203] In the embodiment of the present disclosure, first, it is determined whether the input feature data needs to be pre-processed based on the type of the input feature data. If the type of the input feature data is the second subtype, it is determined that the input feature data does not need to be pre-processed, and there is no need to execute the step of loading the first processing plug-in; then, it is determined whether the second neural network model deployed on the neural network processor contains post-processing logic, that is, it is determined whether the second neural network model has its own post-processing logic. If it contains post-processing logic, there is no need to execute the step of loading the second processing plug-in. This can reduce the amount of loaded data, thereby improving the processing speed.

[0204] Based on the above embodiment, a data processing method is provided below. FIG15 is a flow chart of a data processing method provided by another exemplary embodiment of the present disclosure, which can be applied to the neural network processor 11. As shown in FIG15 , the method includes steps 1501 to 1526.

[0205] Step 1501: Determine the type of input feature data.

[0206] In the disclosed embodiment, the type of input feature data includes a first subtype and a second subtype. If the input feature data is raw data that has not been pre-processed, such as image, audio, or video data, the type of the input feature data is determined to be the first subtype. If the input feature data is feature matrix data that can be directly processed by a neural network model, the type of the input feature data is determined to be the second subtype.

[0207] Step 1502: Based on the type of the input feature data, determine whether the input feature data needs to be pre-processed.

[0208] If the input feature data type is the first subtype, it is determined that the input feature data needs to be pre-processed, and the process goes to step 1503 ; if the input feature data type is the second subtype, the input feature data does not need to be pre-processed, and the process goes to step 1521 .

[0209] Step 1503: Determine whether the second neural network model includes pre-processing logic.

[0210] When the second neural network model includes pre-processing logic, the process proceeds to step 1504 ; when the second neural network model does not include pre-processing logic, the process proceeds to step 1512 .

[0211] Step 1504: Determine whether the second neural network model includes post-processing logic.

[0212] When the second neural network model includes post-processing logic, the second neural network model includes both pre-processing logic and post-processing logic, and there is no need to load the first processing plug-in and the second processing plug-in, and the process goes to step 1505 to process the input feature data using the pre- and post-processing logic of the second neural network model. When the second neural network model does not include post-processing logic, the second neural network model includes pre-processing logic but not post-processing logic, and the second processing plug-in needs to be loaded, and the process goes to step 1508.

[0213] Step 1505: Process the input feature data based on the pre-processing logic included in the second neural network model to obtain intermediate feature data.

[0214] Step 1506: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0215] Step 1507: Process the model output feature data based on the post-processing logic included in the second neural network model to obtain output feature data.

[0216] When it is determined that the input feature data needs to be pre-processed and the second neural network model contains pre-processing logic and post-processing logic, there is no need to load the first processing plug-in and the second processing plug-in. The input feature data can be directly processed using the pre-processing logic of the second neural network model, the second neural network model, and the post-processing logic of the second neural network model to obtain the output feature data.

[0217] Step 1508: Load the second processing plug-in in the published plug-ins.

[0218] Step 1509: Process the input feature data based on the pre-processing logic included in the second neural network model to obtain intermediate feature data.

[0219] Step 1510: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0220] Step 1511: Process the model output feature data based on the second processing plug-in to obtain output feature data.

[0221] When it is determined that the input feature data needs to be pre-processed, and the second neural network model contains pre-processing logic but does not contain post-processing logic, there is no need to load the first processing plug-in. Only the second processing plug-in needs to be loaded. Then, the input feature data is processed using the pre-processing logic of the second neural network model, the second neural network model and the loaded second processing plug-in to obtain the output feature data.

[0222] Step 1512: Determine whether the second neural network model includes post-processing logic.

[0223] When the second neural network model includes post-processing logic, the second neural network model does not include pre-processing logic but includes post-processing logic, and the first processing plug-in needs to be loaded, and step 1513 is entered; when the second neural network model does not include post-processing logic, the second neural network model includes neither pre-processing logic nor post-processing logic, and the first processing plug-in and the second processing plug-in need to be loaded, and step 1517 is entered.

[0224] Step 1513: Load the first processing plug-in among the published plug-ins.

[0225] Step 1514: Process the input feature data based on the first processing plug-in to obtain intermediate feature data.

[0226] Step 1515: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0227] Step 1516: Process the model output feature data based on the post-processing logic included in the second neural network model to obtain output feature data.

[0228] When it is determined that the input feature data needs to be pre-processed, and the second neural network model contains post-processing logic but does not contain pre-processing logic, there is no need to load the second processing plug-in. Only the first processing plug-in needs to be loaded, and then the input feature data is processed using the loaded first processing plug-in, the second neural network model, and the post-processing logic of the second neural network model to obtain the output feature data.

[0229] Step 1517: Load the first processing plug-in and the second processing plug-in in the published plug-ins.

[0230] Step 1518: Process the input feature data based on the first processing plug-in to obtain intermediate feature data.

[0231] Step 1519: Process the intermediate feature data based on the second neural network model to obtain model output feature data.

[0232] Step 1520: Process the model output feature data based on the second processing plug-in to obtain output feature data.

[0233] When it is determined that the input feature data needs to be pre-processed and the second neural network model does not contain pre- and post-processing logic, it is necessary to load the first processing plug-in and the second processing plug-in, and then use the loaded first processing plug-in, the second neural network model and the loaded second processing plug-in to process the input feature data to obtain the output feature data.

[0234] Step 1521: Determine whether the second neural network model includes post-processing logic.

[0235] When the second neural network model includes post-processing logic, there is no need to pre-process the input feature data, and the second neural network model includes post-processing logic, there is no need to load the first processing plug-in and the second processing plug-in, and the process goes to step 1522; when the second neural network model does not include post-processing logic, there is no need to pre-process the input feature data, and the second neural network model does not include post-processing logic, and the second processing plug-in needs to be loaded, and the process goes to step 1524.

[0236] Step 1522: Process the input feature data based on the second neural network model to obtain model output feature data.

[0237] Step 1523: Process the model output feature data based on the post-processing logic included in the second neural network model to obtain output feature data.

[0238] When it is determined that the input feature data does not need to be pre-processed and the second neural network model contains post-processing logic, there is no need to load the first processing plug-in and the second processing plug-in. The second neural network model and the post-processing logic of the second neural network model are directly used to process the input feature data to obtain the output feature data.

[0239] Step 1524: Load the second processing plug-in in the published plug-ins.

[0240] Step 1525: Process the input feature data based on the second neural network model to obtain model output feature data.

[0241] Step 1526: Process the model output feature data based on the second processing plug-in to obtain output feature data.

[0242] When it is determined that the input feature data does not need to be pre-processed and the second neural network model does not contain post-processing logic, there is no need to load the first processing plug-in. Only the second processing plug-in needs to be loaded, and then the second neural network model and the loaded second processing plug-in are used to process the input feature data to obtain the output feature data.

[0243] In the embodiment of the present disclosure, first, it is determined whether the input feature data needs to be pre-processed based on the type of the input feature data, and then it is determined whether the second neural network model contains pre-processing logic and post-processing logic. If the second neural network model does not contain processing logic, the corresponding processing plug-in is loaded, thereby achieving flexible loading of plug-ins. When there is no need to pre-process the input feature data, or the second neural network model has its own pre-processing logic or post-processing logic, there is no need to load the corresponding processing plug-in, which can reduce the amount of loaded data and thus improve the processing speed.

[0244] Exemplary devices

[0245] Figure 16 is a structural diagram of a plug-in processing device for a neural network model provided by an exemplary embodiment of the present disclosure. As shown in Figure 16, the plug-in processing device 1600 for a neural network model includes a first determination module 1601, a processing module 1602, a second determination module 1603 and a publishing module 1604.

[0246] A first determining module 1601 is used to determine a plug-in to be released corresponding to a neural network model;

[0247] The processing module 1602 is configured to process the test input data based on the plug-in to be released and the neural network model determined by the first determining module 1601 to obtain reference output data;

[0248] The second determining module 1603 is used to determine the test output data obtained by the neural network processor processing the test input data based on the plug-in to be released;

[0249] The publishing module 1604 is configured to publish the plug-in to be published based on the consistency check result determined by the test output data determined by the second determining module 1603 and the reference output data obtained by the processing module 1602 .

[0250] In some embodiments, the first determining module 1601 may include a determining unit and a generating unit.

[0251] A determination unit, used to determine the processing logic of the neural network model;

[0252] The generating unit is used to generate a plug-in to be released corresponding to the neural network model based on the processing logic of the neural network model determined by the determining unit.

[0253] In some embodiments, the generation unit is further used to: generate a first processing plug-in in the plug-in to be released based on the pre-processing logic in the processing logic of the neural network model determined by the determination unit; and generate a second processing plug-in in the plug-in to be released based on the post-processing logic in the processing logic of the neural network model determined by the determination unit.

[0254] In some embodiments, the processing module 1602 may include a first processing unit, a second processing unit, and a third processing unit.

[0255] a first processing unit, configured to process the test input data based on the first processing plug-in generated by the generating unit to obtain reference intermediate data;

[0256] a second processing unit, configured to process the reference intermediate data obtained by the first processing unit based on the neural network model to obtain model output reference data;

[0257] The third processing unit is configured to process the model output reference data obtained by the second processing unit based on the second processing plug-in generated by the generating unit to obtain reference output data.

[0258] On the basis of the above embodiments, another exemplary embodiment of the present disclosure provides a plug-in processing device for a neural network model, which includes a determination module, a processing module, a sending module, a receiving module and a publishing module.

[0259] A determination module is used to determine the plug-in to be released corresponding to the neural network model;

[0260] A processing module is used to process the test input data based on the plug-in to be released and the neural network model determined by the determination module to obtain reference output data;

[0261] a sending module, configured to send the plug-in to be released, the test input data, and the reference output data obtained by the processing module to the neural network processor, so that the neural network processor processes the test input data based on the plug-in to be released to obtain test output data, and determines a consistency check result based on the test output data and the reference output data;

[0262] A receiving module, configured to receive a consistency check result sent by the neural network processor;

[0263] The publishing module is used to publish the plug-in to be published based on the received consistency check result.

[0264] FIG17 is a schematic structural diagram of a data processing device provided by an exemplary embodiment of the present disclosure. As shown in FIG17 , the data processing device 1700 includes a first determining module 1701 , a loading module 1702 , and a first processing module 1703 .

[0265] A first determining module 1701 is used to determine the type of input feature data;

[0266] A loading module 1702 is configured to load a published plug-in corresponding to a neural network model deployed on the neural network processor based on the type of input feature data determined by the first determining module 1701;

[0267] The first processing module 1703 is used to process the input feature data based on the published plug-in and neural network model loaded by the loading module 1702 to obtain output feature data.

[0268] In some embodiments, the loading module 1702 may include: a determination unit and a loading unit.

[0269] a determining unit, configured to determine whether pre-processing of the input feature data is required based on the type of the input feature data determined by the first determining module 1701;

[0270] A loading unit is used to load the first processing plug-in in the published plug-in based on the inclusion relationship between the neural network model and the pre-processing logic of the pre-processing; and / or to load the second processing plug-in in the published plug-in based on the inclusion relationship between the neural network model and the post-processing logic corresponding to the neural network model.

[0271] In some embodiments, the first processing module 1703 may include: a first processing unit, a second processing unit, and a third processing unit.

[0272] a first processing unit, configured to process the input feature data based on the first processing plug-in loaded by the loading unit to obtain intermediate feature data;

[0273] a second processing unit, configured to process the intermediate feature data obtained by the first processing unit based on a neural network model to obtain model output feature data;

[0274] The third processing unit is configured to process the model output feature data obtained by the second processing unit based on the second processing plug-in loaded by the loading unit to obtain output feature data.

[0275] In some embodiments, the determining unit is further configured to determine that no pre-processing is required for the input feature data based on the type of the input feature data determined by the first determining module 1701;

[0276] The loading unit is also used to load the second processing plug-in in the published plug-in based on the inclusion relationship between the neural network model and the post-processing logic corresponding to the neural network model.

[0277] In some embodiments, the second processing unit is further configured to process the input feature data based on the neural network model to obtain model output feature data;

[0278] The third processing unit is further configured to process the model output feature data obtained by the second processing unit based on the second processing plug-in loaded by the loading unit to obtain output feature data.

[0279] In some embodiments, the data processing device 1700 may further include: a second determining module, a second processing module, and a first sending module.

[0280] The second determination module is used to determine the plug-in to be released corresponding to the neural network model;

[0281] A second processing module is configured to process the test input data based on the plug-in to be released and the neural network model determined by the second determining module to obtain test output data;

[0282] The first sending module is configured to send the test output data obtained by the second processing module to the plug-in development device, so that the plug-in development device publishes the plug-in to be published based on the test output data.

[0283] In some embodiments, the data processing device 1700 may further include: a third determining module, a third processing module, a fourth determining module, a fifth determining module, and a second sending module;

[0284] The third determination module is used to determine the plug-in to be released corresponding to the neural network model;

[0285] A third processing module is used to process the test input data based on the plug-in to be released and the neural network model determined by the third determination module to obtain test output data;

[0286] A fourth determining module is used to determine reference output data obtained by the plug-in development device through processing the test input data based on the plug-in to be released;

[0287] a fifth determining module, configured to determine a consistency check result based on the test output data obtained by the third processing module and the reference output data determined by the fourth determining module;

[0288] The second sending module is configured to send the consistency check result determined by the fifth determining module to the plug-in development device, so that the plug-in development device publishes the plug-in to be published based on the consistency check result.

[0289] It should be noted that the description of the above exemplary device embodiments is similar to the description of the above method embodiments and has the same beneficial effects as the corresponding exemplary method embodiments. For technical details and corresponding beneficial technical effects not disclosed in the exemplary device embodiments of the present disclosure, those skilled in the art should refer to the description of the exemplary method embodiments of the present disclosure for understanding, and will not be repeated here.

[0290] Exemplary electronic devices

[0291] FIG18 is a schematic diagram of the structure of an electronic device 1800 provided by an exemplary embodiment of the present disclosure. As shown in FIG18 , the electronic device 1800 includes at least one processor 1801 and a memory 1802 .

[0292] The processor 1801 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1800 to perform desired functions.

[0293] The memory 1802 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 1801 may execute one or more computer program instructions to implement the plug-in processing method or data processing method of the neural network model of each embodiment of the present disclosure and / or other desired functions.

[0294] In one example, the electronic device 1800 may further include an input device 1803 and an output device 1804 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0295] The input device 1803 may also include, for example, a keyboard, a mouse, and the like.

[0296] The output device 1804 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output devices, etc.

[0297] Of course, for simplicity, FIG18 only shows some of the components related to the present disclosure in the electronic device 1800, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1800 may further include any other appropriate components according to specific application scenarios.

[0298] Exemplary computer program products and computer-readable storage media

[0299] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the plug-in processing method or data processing method of the neural network model of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0300] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the disclosed embodiments. Programming languages ​​include object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0301] In addition, embodiments of the present disclosure may also provide a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps of the plug-in processing method or data processing method of the neural network model of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0302] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0303] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0304] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0305] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0306] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0307] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0308] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0309] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A plug-in processing method for a neural network model, comprising: Determine the plug-in to be released corresponding to the neural network model; Based on the plug-in to be released and the neural network model, the test input data is processed to obtain reference output data; Determine test output data obtained by the neural network processor processing the test input data based on the plug-in to be released; The plug-in to be released is released based on a consistency check result determined between the test output data and the reference output data.

2. The method according to claim 1, wherein: The step of determining the plug-in to be released corresponding to the neural network model includes: Determining the processing logic of the neural network model; Based on the processing logic of the neural network model, a plug-in to be released corresponding to the neural network model is generated.

3. The method according to claim 2, wherein: The processing logic based on the neural network model generates a plug-in to be released corresponding to the neural network model, including: Generate a first processing plug-in in the plug-in to be released based on the pre-processing logic in the processing logic of the neural network model; Based on the post-processing logic in the processing logic of the neural network model, a second processing plug-in in the plug-in to be released is generated.

4. The method according to claim 3, wherein: The step of processing the test input data based on the plug-in to be released and the neural network model to obtain reference output data includes: Processing the test input data based on the first processing plug-in to obtain reference intermediate data; Processing the reference intermediate data based on the neural network model to obtain model output reference data; The model output reference data is processed based on the second processing plug-in to obtain the reference output data.

5. A data processing method, comprising: Determine the type of input feature data; Based on the type of the input feature data, loading a published plug-in corresponding to the neural network model deployed on the neural network processor; Based on the published plug-in and the neural network model, the input feature data is processed to obtain output feature data.

6. The method according to claim 5, wherein: The step of loading a published plug-in corresponding to the neural network model deployed on the neural network processor based on the type of the input feature data includes: Based on the type of the input feature data, determining that pre-processing of the input feature data is required; Based on the inclusion relationship between the neural network model and the pre-processing logic of the pre-processing, the first processing plug-in in the published plug-in is loaded; and / or, based on the inclusion relationship between the neural network model and the post-processing logic corresponding to the neural network model, the second processing plug-in in the published plug-in is loaded.

7. The method according to claim 6, wherein: The step of processing the input feature data based on the published plug-in and the neural network model to obtain output feature data includes: Based on the first processing plug-in, the input feature data is processed to obtain intermediate feature data; Processing the intermediate feature data based on the neural network model to obtain model output feature data; The model output feature data is processed based on the second processing plug-in to obtain the output feature data.

8. The method according to claim 5, wherein: The step of loading a published plug-in corresponding to the neural network model deployed on the neural network processor based on the type of the input feature data includes: Based on the type of the input feature data, determining that no pre-processing is required for the input feature data; Based on the inclusion relationship between the neural network model and the post-processing logic corresponding to the neural network model, a second processing plug-in in the published plug-ins is loaded.

9. The method according to claim 8, wherein: The step of processing the input feature data based on the published plug-in and the neural network model to obtain output feature data includes: Processing the input feature data based on the neural network model to obtain model output feature data; The model output feature data is processed based on the second processing plug-in to obtain the output feature data.

10. The method according to any one of claims 5 to 9, further comprising: Determine a plug-in to be released corresponding to the neural network model; Based on the plug-in to be released and the neural network model, the test input data is processed to obtain test output data; The test output data is sent to a plug-in development device, so that the plug-in development device publishes the plug-in to be published based on the test output data.

11. A plug-in processing device for a neural network model, comprising: A first determination module is used to determine the plug-in to be released corresponding to the neural network model; A processing module, configured to process the test input data based on the plug-in to be released determined by the first determining module and the neural network model to obtain reference output data; A second determination module is used to determine the test output data obtained by the neural network processor processing the test input data based on the plug-in to be released; A publishing module is used to publish the plug-in to be published based on the test output data determined by the second determining module and the consistency verification result determined by the reference output data obtained by the processing module.

12. A data processing device, comprising: A first determination module, used to determine the type of input feature data; A loading module, used to load a published plug-in corresponding to the neural network model deployed on the neural network processor based on the type of input feature data determined by the first determining module; The first processing module is used to process the input feature data based on the published plug-in loaded by the loading module and the neural network model to obtain output feature data.

13. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the plug-in processing method of the neural network model described in any one of claims 1 to 4, or to execute the data processing method described in any one of claims 5 to 10.

14. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the instructions from the memory and execute the instructions to implement the plug-in processing method of the neural network model described in any one of claims 1-4, or to execute the data processing method described in any one of claims 5-10.

Citation Information

Patent Citations

  • Chip verification method and device and storage medium

    CN113408240A

  • Machine learning processing pipeline optimization

    CN113924582A

  • Chip verification method, system and device and storage medium

    CN114139475A

  • Machine learning method and device based on assembly line, electronic equipment and storage medium

    CN115904359A

  • Plug-in processing method and device, data processing method and device and electronic equipment

    CN116700830A