A method and system for automatically optimizing configuration generation
By generating data operation process description information and optimizing functional module logic, the chip board configuration is automatically optimized, which solves the problem of low configuration efficiency and realizes efficient configuration generation.
Patent Information
- Application Number
- CN202010167803.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-03-11
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2040-03-11
AI Technical Summary
Before using the chip board, users need to repeatedly adjust the configuration parameters to achieve data interaction and scheduling of multi-chip boards, resulting in inefficient configuration.
By generating data operation process description information, determining the module description information and overall logic of the functional module, optimizing the basic functional modules to generate configuration files, supporting the order of API calls, and realizing automatic optimization of configuration generation.
This reduces configuration time and improves configuration efficiency. Users can define business needs in different forms to obtain the optimal chip board hardware platform parameter configuration.
Smart Images
Figure CN113391798B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of chip development, and in particular to an automatic optimization configuration generation method and system. Background Art
[0002] At present, before using chip boards, users generally directly call the SDK of chip boards to familiarize themselves with the internal operation logic of the boards. Users must not only consider the logical order of the specific implementation steps of each function, but also consider the data interaction of operations not supported by the chip and the use scheduling of multi-chip boards. Therefore, they must repeatedly adjust the configuration parameters, which takes a lot of time and has low configuration efficiency. Summary of the invention
[0003] To solve the above problems, the purpose of the present invention is to provide an automatic optimization configuration generation method and system, where users can define the data operation process of business needs in different forms. The chip board device can obtain the API corresponding to the operation node and the API calling sequence by referring to the operation node support list. The user can call the API combination to obtain the optimal supporting chip board hardware platform parameter configuration and automatic optimization plan, thereby reducing configuration time and improving configuration efficiency.
[0004] The present invention provides a method for automatically optimizing configuration generation, comprising:
[0005] Generate data operation process description information according to user business requirements, wherein the process description information is used to describe the process of the data operation to be processed, and the data operation process description information includes each functional module of the data operation and the execution logic between each functional module, and each functional module includes at least each basic functional module;
[0006] Determine the module description information of each functional module of the data operation based on the data operation process description information;
[0007] Based on the data operation process description information, determine the overall logic between the functional modules;
[0008] According to the overall logic between each functional module, optimize the module description information of each basic functional module;
[0009] Generate a configuration file based on the optimized module description information of each functional module.
[0010] As a further improvement of the present invention, when generating the data operation process description information, it includes:
[0011] Generate data operation process description information through one or more of a graphical interface configuration wizard, a UI interactive interface, a command line mode, a text format file, and a table file.
[0012] As a further improvement of the present invention, the basic function module is a description diagram based on operation nodes, and the module description information of the basic function module includes: the operation sequence between the operation nodes within the module, the input and output data of each operation node, the input and output data type of the module to the outside, and the required computing device type.
[0013] As a further improvement of the present invention, each functional module also includes each custom functional module.
[0014] As a further improvement of the present invention, the module description information of the custom function module includes: the module's external input and output data types and the required computing device type.
[0015] As a further improvement of the present invention, the overall logic between the functional modules includes: sequence, selection and loop execution between the functional modules.
[0016] As a further improvement of the present invention, the sequential execution is determined according to the natural order between the functional modules, and the selection and loop execution is determined according to the specific expression of the input selection and loop execution conditions.
[0017] As a further improvement of the present invention, according to the overall logic of each functional module, the module description information of each basic functional module is optimized, including: one or more of sequential execution optimization, parallel execution optimization and power consumption optimization; wherein,
[0018] The optimization of sequential execution includes: equivalently replacing multiple sequential operation nodes in the basic function module with a single operation node;
[0019] The optimization of parallel execution includes: allocating basic functional modules that can be logically calculated in parallel to different computing devices;
[0020] The optimization of power consumption includes: controlling the computing power of the chip board device according to parameters specified by the user.
[0021] As a further improvement of the present invention, the format of the generated configuration file is one of yaml, html, xml and text formats.
[0022] The present invention also provides an automatic optimization configuration generation system, using the method, the system comprising:
[0023] A data operation process generation module, which is used to generate data operation process description information according to user business requirements, wherein the process description information is used to describe the process of the data operation to be processed, and the data operation process description information includes various functional modules of the data operation and the execution logic between the functional modules, and each functional module includes at least various basic functional modules;
[0024] A function module determination module, which is used to determine module description information of each function module of the data operation based on the data operation process description information;
[0025] An overall logic determination module, which is used to determine the overall logic between each functional module based on the process description information;
[0026] An optimization module, which is used to optimize the module description information of each basic functional module according to the overall logic between each functional module;
[0027] The configuration generation module is used to generate a configuration file according to the optimized module description information of each functional module.
[0028] As a further improvement of the present invention, the data operation process generation module, when generating the data operation process description information, includes:
[0029] Generate data operation process description information through one or more of the following methods: graphical interface configuration wizard, UI interactive interface, command line method and text format file.
[0030] As a further improvement of the present invention, the basic function module is a description diagram based on operation nodes, and the module description information of the basic function module includes: the operation sequence between the operation nodes within the module, the input and output data of each operation node, the input and output data type of the module to the outside, and the required computing device type.
[0031] As a further improvement of the present invention, each functional module also includes each custom functional module.
[0032] As a further improvement of the present invention, the module description information of the custom function module includes: the module's external input and output data types and the required computing device type.
[0033] As a further improvement of the present invention, the overall logic between the functional modules in the overall logic determination module includes: sequence, selection and loop execution between the functional modules.
[0034] As a further improvement of the present invention, the sequential execution is based on the natural order between the functional modules, and the selection and loop execution are based on the specific expression of the input selection and loop execution conditions.
[0035] As a further improvement of the present invention, when the optimization module optimizes each basic functional module according to the overall logic between each functional module, it includes: one or more of sequential execution optimization, parallel execution optimization and power consumption optimization; wherein,
[0036] The optimization of sequential execution includes: equivalently replacing multiple sequential operation nodes in the basic function module with a single operation node;
[0037] The optimization of parallel execution includes: allocating basic functional modules that can be logically calculated in parallel to different computing devices;
[0038] The optimization of power consumption includes: controlling the computing power of the chip board device according to parameters specified by the user.
[0039] As a further improvement of the present invention, the format of the configuration file generated by the configuration generation module is one of yaml, html, xml and text formats.
[0040] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the described method.
[0041] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the method.
[0042] The beneficial effects of the present invention are:
[0043] Users can define the data operation process of business needs in different forms, including various functional modules and their execution logic. By defining the description information of each functional module, the chip board device can obtain the API corresponding to the operation node by comparing the operation node support list. By defining the overall logic of each functional module, the calling sequence of the API corresponding to all operation nodes can be obtained. The basic functional modules are optimized through the defined overall logic to obtain the optimized API combination. Users can obtain the optimal parameter configuration and automatic optimization plan of the supporting chip board hardware platform by calling the API combination, effectively reducing the configuration time and thus improving the configuration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.
[0045] Figure 1 A schematic diagram of a flow chart of an automatic optimization configuration generation method according to an exemplary embodiment of the present disclosure;
[0046] Figure 2 A basic function module data operation diagram described in an exemplary embodiment of the present disclosure;
[0047] Figure 3 It is an overall logic diagram of the basic function modules and the custom function modules described in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.
[0049] It should be noted that if the embodiments of the present disclosure involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0050] In addition, in the description of the present disclosure, the terms used are only for illustrative purposes and are not intended to limit the scope of the present disclosure. The terms "include" and / or "comprise" are used to specify the existence of the elements, steps, operations and / or components, but do not exclude the existence or addition of one or more other elements, steps, operations and / or components. The terms "first", "second" and the like may be used to describe various elements, do not represent the order, and do not limit these elements. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two and more than two. These terms are only used to distinguish one element from another element. In conjunction with the following drawings, these and / or other aspects become apparent, and it is easier for those of ordinary skill in the art to understand the description of the embodiments described in the present disclosure. The accompanying drawings are used to describe the embodiments described in the present disclosure for illustrative purposes only. Those skilled in the art will easily recognize from the following description that, without departing from the principles described in the present disclosure, alternative embodiments of the structures and methods shown in the present disclosure can be adopted.
[0051] An automatic optimization configuration generation method described in an embodiment of the present disclosure is as follows: Figure 1 As shown, including:
[0052] According to the user's business needs, data operation process description information is generated. The process description information is used to describe the process of the data operation to be processed. The data operation process description information includes each functional module of the data operation and the execution logic between each functional module. Each functional module at least includes each basic functional module. In this step, the user can define the process of the chip board device for the data operation to be processed, the execution logic between each functional module and modules, and some necessary parameters.
[0053] Each functional module has at least one step to operate on data. When the data operation process description information is generated, the chip board device can refer to the operation node support list to determine which functional modules are basic functional modules and which functional modules are custom functional modules. Basic functional modules are functional modules that are supported by the chip board device to realize user services, and custom functional modules are functional modules that are not supported by the chip board device but are necessary to realize user services. By dividing the functional modules into basic functional modules and custom functional modules, the optimization can be achieved when optimizing the basic functional modules.
[0054] In an optional implementation, the user can use a text format file to define the data operation flow description information, or use a table format file (such as an Excel file) to define the data operation flow description information. In another optional implementation, in order to make the user-defined data operation flow description information have structured characteristics, the data operation flow description information is generated through one or more of a graphical interface configuration wizard, a UI interactive interface, and a command line method.
[0055] Based on the data operation process description information, the module description information of each functional module is determined. This step is to instantiate the abstract, high-level process description information generated in the previous step into multiple operation instructions related to the chip board device, which is to further refine the data operation process description information in the previous step and determine the module description information of each functional module.
[0056] Among them, the basic function module is a description diagram of the operation node executed in the chip board to realize a specific function, and the operation node is an operation performed on the data. In an optional implementation, the module description information of the basic function module includes: the operation sequence between the operation nodes inside the module, the input and output data of each operation node, the input and output data type of the module to the outside, and the required computing device type. The module description information of the custom function module includes: the input and output data type of the module to the outside, and the required computing device type. Based on the chip's operation node support list, each operation node corresponds to an API. After determining the module description information of the basic function module and the custom function module, the APIs of all operation nodes can be obtained accordingly.
[0057] Based on the data operation process description information, determine the overall logic between each functional module. The first step defines the execution logic between each functional module. This step is to determine the overall logic between each functional module. After determining the overall logic between each functional module, the calling sequence of the API corresponding to all operation nodes can be obtained.
[0058] The overall logic of each functional module includes: the order, selection and loop execution between each functional module. The order execution is determined according to the natural order of each functional module, and the selection and loop execution are determined according to the specific expression of the input selection and loop execution conditions. The execution conditions are input by the pre-configured application when the chip board device executes the execution conditions.
[0059] According to the overall logic between each functional module, the module description information of each basic functional module is optimized. This step can optimize the API corresponding to the operation node by optimizing each basic functional module to obtain an optimized API combination.
[0060] In an optional implementation, when optimizing each basic functional module, it includes: one or more of sequential execution optimization, parallel execution optimization and power consumption optimization; wherein,
[0061] The optimization of sequential execution includes: replacing multiple sequential operation nodes in the basic function module with a single operation node. For example, the three operation instructions of "image size normalization", "image color space conversion", and "image boundary filling" are merged into the "image size normalization-color space conversion-boundary filling" operation instruction with equivalent results. For another example, the two operations of "classification neural network reasoning" and "maximum probability item selection" can be merged into the "classification neural network reasoning and maximum probability item selection" operation instruction.
[0062] Optimization of parallel execution includes: allocating basic functional modules that can be logically calculated in parallel to different computing devices. For example, if the user's logic requires multiple neural network inferences on a certain image area, and multiple AI chip board resources have been installed, the parallel operation can be distributed to multiple computing devices for execution.
[0063] Power consumption optimization includes: controlling the computing power of chip board devices according to user-specified parameters. For example, if a user uses a battery-powered terminal device and wants the device to have a longer battery life, the configuration items that reduce the computing power of the main functional components can be configured to achieve the requirement of reducing power consumption.
[0064] Generate a configuration file based on the optimized module description information of each functional module. This can be understood as outputting the module description information of the custom functional module and the configuration information of the optimized basic functional module in a specific format, so that the subsequent compilation module can parse and generate the corresponding SDK call relationship. The SDK call relationship is the call relationship corresponding to the optimized API combination.
[0065] In an optional implementation, the format of the generated configuration file is one of yaml, html, xml and text formats.
[0066] An automatic optimization configuration generation system according to an embodiment of the present disclosure adopts the method described above, and the system includes:
[0067] The data operation process generation module is used to generate data operation process description information according to user business needs. The process description information is used to describe the process of the data operation to be processed. The data operation process description information includes each functional module of the data operation and the execution logic between each functional module. Each functional module at least includes each basic functional module. This module is used by the user to define the process of the chip board device for the data operation to be processed, the execution logic between each functional module and modules, and some necessary parameters.
[0068] Each functional module has at least one step to operate on data. When the data operation process description information is generated, the chip board device can refer to the operation node support list to determine which functional modules are basic functional modules and which functional modules are custom functional modules. Basic functional modules are functional modules that are supported by the chip board device to realize user services, and custom functional modules are functional modules that are not supported by the chip board device but are necessary to realize user services. By dividing the functional modules into basic functional modules and custom functional modules, the optimization can be achieved when optimizing the basic functional modules.
[0069] In an optional implementation, the user can use a text format file to define the data operation flow description information, or use a table format file (such as an Excel file) to define the data operation flow description information. In another optional implementation, in order to make the user-defined data operation flow description information have structured characteristics, the data operation flow description information is generated through one or more of a graphical interface configuration wizard, a UI interactive interface, and a command line method.
[0070] The function module determination module is used to determine the module description information of each function module based on the data operation process description information. This step is to instantiate the abstract, high-level data operation process description information generated in the previous step into multiple operation instructions related to the chip board device, and further refine the data operation process description information in the data operation process generation module to determine the module description information of each function module.
[0071] Among them, the basic function module is a description diagram of the operation node executed in the chip board to realize a specific function, and the operation node is an operation performed on the data. In an optional implementation, the module description information of the basic function module includes: the operation sequence between the operation nodes inside the module, the input and output data of each operation node, the input and output data type of the module to the outside, and the required computing device type. The module description information of the custom function module includes: the input and output data type of the module to the outside, and the required computing device type. Based on the chip's operation node support list, each operation node corresponds to an API. After determining the module description information of the basic function module and the custom function module, the APIs of all operation nodes can be obtained accordingly.
[0072] The overall logic determination module is used to determine the overall logic between each functional module based on the data operation process description information. After this module determines the overall logic between each functional module, the calling sequence of the API corresponding to all operation nodes can be obtained.
[0073] The overall logic of each functional module includes: sequence, selection and loop execution between each functional module. Sequential execution is determined by the natural sequence between each basic functional module, and selection and loop execution are determined by the specific expression of the input selection and loop execution conditions. The execution conditions are input by the pre-configured application when the chip board device executes the execution conditions.
[0074] The optimization module is used to optimize the module description information of each basic functional module according to the overall logic between each functional module. This module can optimize the API corresponding to the operation node by optimizing each basic functional module to obtain an optimized API combination.
[0075] In an optional implementation, when optimizing each basic functional module, it includes: one or more of sequential execution optimization, parallel execution optimization and power consumption optimization; wherein,
[0076] The optimization of sequential execution includes: replacing multiple sequential operation nodes in the basic function module with a single operation node. For example, the three operation instructions of "image size normalization", "image color space conversion", and "image boundary filling" are merged into the "image size normalization-color space conversion-boundary filling" operation instruction with equivalent results. For another example, the two operations of "classification neural network reasoning" and "maximum probability item selection" can be merged into the "classification neural network reasoning and maximum probability item selection" operation instruction.
[0077] Optimization of parallel execution includes: allocating basic functional modules that can be logically calculated in parallel to different computing devices. For example, if the user's logic requires multiple neural network inferences on a certain image area, and multiple AI chip board resources have been installed, the parallel operation can be distributed to multiple computing devices for execution.
[0078] Power consumption optimization includes: controlling the computing power of chip board devices according to user-specified parameters. For example, if a user uses a battery-powered terminal device and wants the device to have a longer battery life, the configuration items that reduce the computing power of the main functional components can be configured to achieve the requirement of reducing power consumption.
[0079] The configuration generation module is used to generate configuration files based on the module description information of each optimized functional module. This module outputs the module description information of the custom functional module and the configuration information of the module description information of the optimized basic functional module in a specific format, so that the subsequent compilation module can parse and generate the corresponding SDK call relationship. The SDK call relationship is the call relationship corresponding to the optimized API combination.
[0080] In an optional implementation, the format of the generated configuration file is one of yaml, html, xml and text formats.
[0081] The chip board device in the above implementation manner may be, for example, an AI chip board device.
[0082] The present disclosure also relates to an electronic device, including a server, a terminal, etc. The electronic device includes: at least one processor; a memory connected to the at least one processor; and a communication component connected to the storage medium, the communication component receiving and sending data under the control of the processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the automatic optimization configuration generation method in the above embodiment.
[0083] In an optional embodiment, the memory is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory, that is, realizing the above-mentioned automatic optimization configuration generation method.
[0084] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0085] One or more modules are stored in the memory, and when executed by one or more processors, the automatic optimization configuration generation method in any of the above method embodiments is executed.
[0086] The above-mentioned product can execute the automatic optimization configuration generation method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the automatic optimization configuration generation method provided in the embodiment of the present application.
[0087] The present disclosure also relates to a computer-readable storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above-mentioned automatic optimization configuration generation method embodiments.
[0088] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0089] A specific embodiment is used below to illustrate an automatic optimization configuration generation method involved in the present disclosure.
[0090] Description of the application scenario of user business requirements: Assume that the application needs to detect targets in video streams. When a motor vehicle target appears in the video, the area where the motor vehicle target is located is captured, and then the captured image is deblurred and sent to a neural network for classification. Similarly, when an animal target appears in the video, the area where the animal target is located is captured, and then the captured image is deblurred and sent to another neural network for classification. On this basis, it is required to output the captured image of the discovered target.
[0091] It is assumed that the user uses multiple chip board devices to realize the business requirements of the above application scenario.
[0092] In the first step, the user can input the business requirements in the form of Excel tables or text files (the user can also generate business requirements through the graphical interface configuration wizard), and then generate the operation process description information. Taking the above application scenario description as an example, the following functional modules can be generated: 1) decoding module, 2) target detection module, 3) classification network 1 module (motor vehicle), 4) classification network 2 module (animal), 5) image encoding module, 6) image deblurring module 1 (motor vehicle), 7) image deblurring module 2 (animal). There is at least one step in each module to operate on the data. The execution logic of each functional module is: the output of the decoding module is connected to the input of the target detection module and the image encoding module, the output of the target detection module is connected to the input of the image deblurring module 1 and the image deblurring module 2, and the output of the image deblurring module 1 and the image deblurring module 2 is connected to the input of the classification network 1 module and the classification network 2 module.
[0093] The second step is to determine the module description information of the basic function module and the custom function module. Figure 2 As shown, taking 1) the decoding module (basic functional module) as an example, the module description information is as follows:
[0094] Decoding module:
[0095] Module input: video stream
[0096] Module output: normalized original video frame
[0097] Computing equipment: AI chip board Number: 01
[0098] Operation Node:
[0099] Video decoding:
[0100] Step ID: 01
[0101] Input: Video stream
[0102] Output: YUV original frame image
[0103] Parameters: parameter 1, parameter 2, ...
[0104] Image color space conversion:
[0105] Step ID: 02
[0106] Input: YUV original frame image
[0107] Output: BGR image
[0108] Parameters: parameter 1, parameter 2, ...
[0109] Image resizing:
[0110] Step ID: 03
[0111] Input: BGR image
[0112] Output: Normalized original video frame
[0113] Taking the image deblurring module (custom function module) as an example, the module description information is as follows:
[0114] Image deblurring module:
[0115] Module input: original image of target area ROI
[0116] Module output: clear image of target area ROI
[0117] Computing device: host CPU
[0118] The third step is to determine the overall logic between the functional modules, including the connection relationship between the basic functional modules and the custom functional modules, such as Figure 3 As shown. In the first step, the execution logic of each functional module has been defined. In this step, the branch logic of each functional module needs to be clarified to determine the overall logic between each functional module. Among them, the branch logic definition of each functional module is:
[0119] Switch found the target:
[0120] Case target is motor vehicle:
[0121] Execute the classification network 1 module;
[0122] Break;
[0123] Case target is animal:
[0124] Execute the classification network 2 module;
[0125] Break;
[0126] Default:
[0127] Do not execute the branch and continue to the next frame target detection
[0128] The fourth step is to optimize each basic functional module according to the overall logic. The branch logic in the third step is to judge all discovered targets. Since the two targets of "motor vehicle" and "animal" may exist at the same time in the same video frame, and the user device has multiple AI chip board devices, the classification network 1 module and the classification network 2 module can be executed in parallel logically, and the encoding module is logically independent of other modules. Therefore, the module description information of the classification network 1 module, the classification network 2 module and the encoding module is modified, and multiple AI chip board devices are used to calculate the data of the three modules in parallel.
[0129] The fifth step is to generate a configuration file. The parameter configuration that meets the user's needs is output as a configuration file in a specific format. The format of the output configuration file can be specified by the user, including yaml, html, xml and other formats. Text files can also be used to facilitate subsequent compilation modules to parse and generate process-optimized SDK call combinations.
[0130] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0131] In addition, it will be understood by those skilled in the art that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present disclosure and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.
[0132] It will be appreciated by those skilled in the art that, although the present disclosure has been described with reference to exemplary embodiments, various changes may be made and elements thereof may be replaced with equivalents without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt specific circumstances or materials to the teachings of the present disclosure without departing from the substantive scope of the present disclosure. Therefore, the present disclosure is not limited to the specific embodiments disclosed, but the present disclosure will include all embodiments falling within the scope of the appended claims.
Claims
1. A method for automatically generating an optimized configuration, characterized in that: include: Generate data operation process description information according to user business requirements, wherein the data operation process description information is used to describe the process of the data operation to be processed, and the data operation process description information includes each functional module of the data operation and the execution logic between each functional module, and each functional module includes at least each basic functional module; Determine the module description information of each functional module based on the data operation process description information; Based on the data operation process description information, determine the overall logic between the functional modules; According to the overall logic between the functional modules, the module description information of each basic functional module is optimized, including the optimization of parallel execution, wherein the optimization of parallel execution includes: allocating the basic functional modules that can be calculated in parallel logically to different chip board computing devices; Generate a configuration file based on the optimized module description information of each functional module.
2. The method according to claim 1, characterized in that Generate data operation process description information, including: Generate data operation process description information through one or more of a graphical interface configuration wizard, a UI interactive interface, a command line mode, a text format file, and a table file.
3. The method according to claim 1, characterized in that The basic function module is a description diagram based on the operation node. The module description information of the basic function module includes: the operation sequence between the operation nodes inside the module, the input and output data of each operation node, the input and output data type of the module to the outside, and the required computing device type.
4. The method according to claim 1, characterized in that: Each functional module also includes each custom functional module.
5. The method according to claim 4, characterized in that The module description information of the custom function module includes: the module's external input and output data types and the required computing device type.
6. The method according to claim 1, characterized in that The overall logic between each functional module includes: the order, selection and loop execution between each functional module.
7. The method according to claim 6, characterized in that The sequential execution is determined according to the natural order between the functional modules, and the selection and loop execution are determined according to the specific expression of the input selection and loop execution conditions.
8. The method according to claim 1, characterized in that According to the overall logic between each functional module, the module description information of each basic functional module is optimized, and it also includes: one or more of sequential execution optimization and power consumption optimization; the sequential execution optimization includes: equivalently replacing multiple sequential operation nodes in the basic functional module with a single operation node; the power consumption optimization includes: controlling the computing power of the chip board device according to the parameters specified by the user.
9. The method according to claim 1, characterized in that: The format of the generated configuration file is at least one of yaml, html, xml, and text.
10. An automatic optimization configuration generation system, characterized in that: According to any one of claims 1 to 9, the system comprises: A data operation process generation module, which is used to generate data operation process description information according to user business requirements, wherein the process description information is used to describe the process of the data operation to be processed, and the data operation process description information includes various functional modules of the data operation and the execution logic between the functional modules, and each functional module includes at least various basic functional modules; A function module determination module, which is used to determine module description information of each function module based on the data operation process description information; An overall logic determination module, which is used to determine the overall logic between various functional modules based on the data operation process description information; An optimization module, which is used to optimize the module description information of each basic functional module according to the overall logic between each functional module, including the optimization of parallel execution, wherein the optimization of parallel execution includes: allocating the basic functional modules that can be calculated in parallel logically to different chip board computing devices; The configuration generation module is used to generate a configuration file according to the optimized module description information of each functional module.
11. The system according to claim 10, characterized in that When generating the data operation process description information, the data operation process generation module includes: Generate data operation process description information through one or more of the following methods: graphical interface configuration wizard, UI interactive interface, command line method and text format file.
12. The system according to claim 10, characterized in that The basic function module is a description diagram based on the operation node. The module description information of the basic function module includes: the operation sequence between the operation nodes inside the module, the input and output data of each operation node, the input and output data type of the module to the outside, and the required computing device type.
13. The system according to claim 10, characterized in that Each functional module also includes each custom functional module.
14. The system according to claim 13, characterized in that The module description information of the custom function module includes: the module's external input and output data types and the required computing device type.
15. The system according to claim 10, characterized in that The overall logic between the functional modules in the overall logic determination module includes: the order, selection and loop execution between the functional modules.
16. The system according to claim 15, characterized in that The sequential execution is based on the natural order between the functional modules, and the selection and loop execution are based on the specific expression of the input selection and loop execution conditions.
17. The system according to claim 10, characterized in that When the optimization module optimizes each basic functional module according to the overall logic between each functional module, it also includes: one or more of sequential execution optimization and power consumption optimization; The optimization of sequential execution includes: equivalently replacing multiple sequential operation nodes in the basic function module with a single operation node; The optimization of power consumption includes: controlling the computing power of the chip board device according to parameters specified by the user.
18. The system according to claim 10, characterized in that The configuration file generated by the configuration generation module is in one of yaml, html, xml and text formats.
19. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 9.
20. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the method according to any one of claims 1 to 9.