Model packaging method, device, storage medium and electronic device

Abstract

The embodiment of the present invention provides a model packaging method, device, storage medium and electronic device, wherein the method includes: obtaining a target algorithm basic package from an algorithm warehouse based on a received selection instruction, wherein the target algorithm basic package is the algorithm basic package specified in the selection instruction; generating a training task based on the target algorithm basic package, and sending the training task to a training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, and perform model training according to the target training scheme in the target algorithm basic package to obtain a target model; receiving the target model sent by the training platform, and packaging the target model according to the model upgrade scheme to obtain an algorithm software package. Through the present invention, the problem of low model training efficiency existing in the related art is solved, and the effect of improving model training efficiency is achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence model training, and specifically, to a model packaging method, device, storage medium, and electronic device. Background Art

[0002] In related technologies, when requirements for existing AI applications change, the original AI application developer is typically tasked with rebuilding and training a new model. This results in long development cycles and inefficient adaptation. Users can also use model training platforms to train new models for specific device types, but simulation training platforms are limited in their versatility, as they can only train for specific device types and training scenarios.

[0003] From this, we can see that there is a problem of low model training efficiency in related technologies.

[0004] Currently, no effective solution has been proposed to the above-mentioned problems existing in the related technologies. Summary of the Invention

[0005] Embodiments of the present invention provide a model packaging method, device, storage medium, and electronic device to at least solve the problem of low model training efficiency existing in related technologies.

[0006] According to one embodiment of the present invention, a model packaging method is provided, comprising: obtaining a target algorithm basic package from an algorithm warehouse based on a received selection instruction, wherein the target algorithm basic package is the algorithm basic package specified in the selection instruction; generating a training task based on the target algorithm basic package, and sending the training task to a training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, and performing model training according to the target training scheme in the target algorithm basic package to obtain a target model; receiving the target model sent by the training platform, and packaging the target model according to the model upgrade scheme to obtain an algorithm software package.

[0007] According to another embodiment of the present invention, a model packaging device is provided, including: an acquisition module, used to obtain a target algorithm basic package from an algorithm warehouse based on a received selection instruction, wherein the target algorithm basic package is the algorithm basic package specified in the selection instruction; a generation module, used to generate a training task based on the target algorithm basic package, and send the training task to a training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, perform model training according to the target training scheme in the target algorithm basic package, and obtain a target model; a receiving module, used to receive the target model sent by the training platform, and package the target model according to the model upgrade scheme to obtain an algorithm software package.

[0008] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0009] According to another embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0010] Through the present invention, the target algorithm basic package is obtained from the algorithm warehouse according to the received selection instruction, a training task is generated according to the target algorithm basic package, and the training task is sent to the training platform. The training platform obtains the training data of the model in the target algorithm basic package, and performs model training according to the target training scheme included in the target algorithm basic package to obtain the target model. The target model sent by the training platform is received, and the target model is packaged according to the model upgrade scheme to obtain the algorithm software package. Since when the user's needs change, the target algorithm basic package specified by the user can be obtained from the algorithm warehouse according to the selection instruction input by the user, and a training task is generated according to the target algorithm basic package and handed over to the training platform for training. The training platform performs model training according to the training scheme in the target algorithm basic package. There is no need to build a new model. It is only necessary to train and package the existing algorithm basic package to obtain the algorithm software package. Therefore, the problem of low model training efficiency in the related art can be solved, and the effect of improving model training efficiency can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a hardware structure block diagram of a mobile terminal for a model packaging method according to an embodiment of the present invention;

[0012] Figure 2 is a flowchart of a method for packaging a model according to an embodiment of the present invention;

[0013] Figure 3 is a schematic diagram of model packaging according to an exemplary embodiment of the present invention;

[0014] Figure 4 2. It is a structural diagram of a packaging device according to a specific embodiment of the present invention;

[0015] Figure 5 is a flow chart of a basic package generation algorithm according to a specific embodiment of the present invention;

[0016] Figure 6 is a flow chart of a model packaging method according to a specific embodiment of the present invention;

[0017] Figure 7 4 is a structural block diagram of a model packaging device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with embodiments.

[0019] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0020] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure diagram of a mobile terminal of a model packaging method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0021] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the packaging method of the model in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0023] In this embodiment, a model packaging method is provided. Figure 2 is a flow chart of a method for packaging a model according to an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0024] Step S202: acquiring a target algorithm basic package from the algorithm warehouse based on the received selection instruction, wherein the target algorithm basic package is the algorithm basic package specified in the selection instruction;

[0025] Step S204: Generate a training task based on the target algorithm basic package, and send the training task to the training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, perform model training according to the target training scheme in the target algorithm basic package, and obtain a target model;

[0026] Step S206: receiving the target model sent by the training platform, and packaging the target model according to the model upgrade plan to obtain an algorithm software package.

[0027] In the above embodiment, the selection instruction can be a command input by a user from the central platform. The user can be a user who manages the device, and the device can include an intelligent monitoring device, such as a smart camera. The device can be installed with a network model for detecting, identifying, and performing other tasks on images collected or acquired by the device through the network model. In other words, all network models installed in the device can perform target tasks, and the user-managed device can perform target tasks. Target tasks can include detection tasks, identification tasks, tracking tasks, and the like. Detection tasks can include detecting objects included in the collected image, such as vehicles, pedestrians, and animals. Identification tasks can include identifying the type of object included in the collected image, such as vehicle type identification, gender identification, and animal species identification. Tracking tasks can include tracking the target object after detecting it in the image. The target task performed by the device can be determined based on the user's needs. When the user requires the device to perform a tracking task, the network model that performs the tracking task can be installed in the device.

[0028] When user needs change, the user can input selection instructions through the central platform and select the target algorithm software package that meets the current needs. The change in user needs can be that the current task performed by the device is to identify pedestrians in images, but the user now wants the device to identify pedestrians and animals in images. This can be considered a change in user needs.

[0029] In the above embodiment, the target algorithm software package can be stored in the algorithm warehouse, that is, the algorithm warehouse can include multiple algorithm software packages, and each algorithm software package can include a training scheme. The training scheme can be a scheme composed of some network models, such as object detection (Object Detection), object classification / recognition (Object Classification), object tracking (Object Tracking), face recognition (Face Detection), etc. Each network model can be trained using some neural network frameworks. The algorithm basic package can also include an inference scheme. Compared with the training scheme, the inference scheme can use the model of the training scheme to process the input. Usually, in addition to the model, some professional algorithms and business logic (such as rules, alarm events, etc.) will be added according to specific application scenarios.

[0030] In the above embodiment, after the user determines the requirements, they can select a target algorithm base package from the algorithm bin via a selection command and specify the training data used to train the target algorithm base package via a selection command. Specifically, if the device is currently performing the task of identifying pedestrians, and the requirements are changed so that the device's task is to identify pedestrians and animals, the training scheme included in the target algorithm base package selected by the user via the selection command will be the target recognition scheme, and the training data selected via the selection command will include images of animals.

[0031] In the above embodiment, after determining the algorithm basic package, the central platform can generate a training task based on the target algorithm basic package, and send the training task to the training platform. The user selects a training plan and can import and label the training materials according to the requirements of the plan. Different training plans have different material and labeling requirements. For example, target detection does not require manual labeling, and only requires that the target to be detected occupies most of the area of ​​the screen (such as more than 70%). The central platform applies for training resources from the training platform, initiates a training task, and stores the trained model in the algorithm warehouse (the same material and training plan do not need to be trained repeatedly). That is, import training materials, select a model training plan, label according to the requirements of the plan, execute training tasks, and obtain the training model and model computing power.

[0032] It should be noted that the training platform can train the training schemes of the models included in different algorithm basic packages to obtain trained models, that is, the general training platform can produce special models for different devices.

[0033] In the above embodiment, after model training, the training platform obtains a target model and sends it to the central platform. The central platform receives the target model from the training platform and packages it according to the model upgrade plan to obtain an algorithm software package. Once the algorithm software package is obtained, it can be installed on the device to enable the device to perform the task after the requirement change.

[0034] The execution entity of the above steps may be a central platform, etc., but is not limited thereto.

[0035] Through the present invention, the target algorithm basic package is obtained from the algorithm warehouse according to the received selection instruction, a training task is generated according to the target algorithm basic package, and the training task is sent to the training platform. The training platform obtains the training data of the model in the target algorithm basic package, and performs model training according to the target training scheme included in the target algorithm basic package to obtain the target model. The target model sent by the training platform is received, and the target model is packaged according to the model upgrade scheme to obtain the algorithm software package. Since when the user's needs change, the target algorithm basic package specified by the user can be obtained from the algorithm warehouse according to the selection instruction input by the user, and a training task is generated according to the target algorithm basic package and handed over to the training platform for training. The training platform performs model training according to the training scheme in the target algorithm basic package. There is no need to build a new model. It is only necessary to train and package the existing algorithm basic package to obtain the algorithm software package. Therefore, the problem of low model training efficiency in the related art can be solved, and the effect of improving model training efficiency can be achieved.

[0036] In an exemplary embodiment, the target model is packaged according to the model upgrade plan to obtain the algorithm software package, including: decompressing the target algorithm base package to obtain the base package description information of the target algorithm base package; determining the information of the first device that supports the installation of the algorithm base package based on the base package description information; converting the target model into a format supported by the first device to obtain a conversion model; packaging the conversion model and the base package description information according to the model upgrade plan to obtain the model upgrade package. In this embodiment, when packaging the target model, the algorithm base package can be decompressed, and the content can be organized according to the content specification of the upgrade package, the base package description information can be converted into the corresponding upgrade package description information, and the corresponding optimizer can be selected according to the hardware platform information of the base package to convert the target model into a model available for the hardware unit under the hardware platform. That is, the target model is converted into a format supported by the first device.

[0037] In the above embodiment, when the demand changes, you can select the same algorithm basic package as the aforementioned model training solution, select a model replacement solution (such as inheritance, replacement), decompress the algorithm basic package, and convert the basic package description into an upgrade package description in a certain format; according to the hardware platform information in the basic package description information, optimize the aforementioned training model and convert it into a model that can be used on the hardware platform; the platform for executing the model is different, and the model needs to be converted; and execute the model replacement solution on the optimized model.

[0038] In the above embodiment, the basic package description information may include information such as the model training scheme, model capability set, hardware platform, etc. Among them, the hardware platform information may be the information of the first device that supports the installation of the algorithm basic package. The hardware platform information may include the system information and hardware information of the device. The system information may include the operating system information; the hardware information may include the memory information, CPU information, and external device information of the device (such as pan-tilt information, infrared module information, fill light information, audio module information, radar module information, etc.). The basic package description information can identify the model training scheme, model capability set, and hardware platform information of the algorithm basic package that the algorithm basic package allows to be installed. When the user's needs change, the user can select the target algorithm basic package according to the basic package description information and generate a selection instruction.

[0039] In one exemplary embodiment, packaging the conversion model and the base package description information according to the model upgrade scheme to obtain the model upgrade package includes: if the model upgrade scheme is a merge scheme for merging the original model, merging the configuration file of the conversion model with the configuration file of the original model to obtain a target configuration file, and packaging the target configuration file and the base package description information to obtain the algorithm software package; if the model upgrade scheme is a replace scheme for the original model, packaging the configuration file of the conversion model and the base package description information to obtain the algorithm software package. In this embodiment, according to the replacement scheme, i.e., the model upgrade scheme, the models included in the original base package can be merged or replaced. If merging, the two model capability information are merged (e.g., if the original model capability is "target cat, white color, breed ragdoll" and the new model capability is "target dog, golden color, breed teddy", the merged model capability is "target cat, white color, breed ragdoll; target dog, golden color, breed teddy"). If replacing, the original model is deleted, and only the configuration file of the conversion model and the base package description information are packaged. Among them, the model capabilities mainly correspond to target detection and target classification, and also correspond to the calibration information of training materials.

[0040] In the above embodiment, when the model upgrade solution is to merge the original model, the new model can be merged with the model in the original base package, and the union of the model capability set of the new model and the model capability set of the original model is used as the model capability set of the merged model and placed in the package description of the upgrade package; the model capability set is trained according to the model training solution; in the case of homology, the model capability sets are compared to determine the homology; when the model replacement solution is to replace the original model, the model in the original base package can be deleted, and the model capability set of the new model is placed in the package description of the upgrade package. When the model upgrade solution is to replace the original model, the original model capabilities described in the upgrade package can be replaced with the new model capabilities, and other information (such as signatures, certificates, etc.) can be added as needed, and the new organization files are packaged into an algorithm software package. Among them, homology means determining the product compatibility of the algorithm product. Even if there are multiple algorithm product delivery parties, the products delivered according to certain requirements can be compatible with each other. Only one homology model can exist on the device, and multiple models of different series can exist. Homologous series means determining the business compatibility of homology models. During the upgrade process, homology ensures that the business configuration remains unchanged and can be directly upgraded and used, while homology models of non-homologous series require configuration changes to continue to use. The model upgrade diagram can be found in the attached Figure 3 .

[0041] In an exemplary embodiment, before obtaining the target algorithm base package from the algorithm warehouse based on the received selection instruction, the method further includes: receiving the target algorithm base package sent by the software publishing platform; verifying the legitimacy of the target algorithm base package; if the target algorithm base package is legal, determining whether the target algorithm base package exists in the algorithm warehouse; if the target algorithm base package does not exist, determining whether the training platform supports the training scheme included in the target algorithm base package; and if so, storing the target algorithm base package in the algorithm warehouse. In this embodiment, before obtaining the target algorithm base package from the algorithm warehouse, the software publishing platform can send the target algorithm base package to the central platform. After receiving the target algorithm base package, the central platform can verify the legitimacy of the target algorithm base package. That is, the central platform can verify the base package: verify the legitimacy of the target algorithm base package. If the target algorithm base package is determined to be legal, a duplicate push check is performed on the base package. If the target algorithm base package is determined to be non-duplicate push, a platform availability verification of the base package is performed. That is, the hardware platform of the base package is analyzed to check whether the current platform supports training the model of the platform (i.e., optimizer support). Parse the model training scheme of the target algorithm base package and query whether the training scheme exists on the central platform. If not, check whether the training of the basic models that make up the training scheme is supported. If so, create the training scheme on the central platform; otherwise, the central platform does not support the basic package. The basic model refers to a network model trained with a certain neural network framework (for example, the OD model trained with YOLO. Of course, these frameworks themselves also have versions, specifically: a certain network model trained with a certain version of the neural network framework).

[0042] In one exemplary embodiment, receiving the target algorithm base package sent by the software publishing platform includes: receiving the target algorithm base package sent by the software publishing platform after obtaining the target algorithm base package in the following manner: verifying the legitimacy of the received algorithm product, wherein the algorithm product includes an algorithm library, a model, and a product description file of the algorithm product; if the algorithm product is legal, assigning a target identifier to the algorithm product; and determining the target algorithm base package based on the product description file and the target identifier. In this embodiment, after receiving the algorithm product, the software publishing platform may verify the algorithm product and generate the target algorithm base package. Verifying the legitimacy of the received algorithm product may include identifying the submitted product and performing verification based on the identified type, such as developer legitimacy, product legitimacy, and duplicate product submission check. If the algorithm product is legal, a target identifier, such as a source ID, may be assigned to the algorithm product. The target algorithm base package is then determined based on the product description file and the target identifier. In the above embodiment, the algorithm product may include files such as the algorithm library, model, and product description. The algorithm library: a dynamic library that loads the model and performs inference on the input according to a specific inference scheme; a binary file, an executable file, which is bound to the business (an executable file, the algorithm loads the model into the library for operation). Model: a model file obtained according to a specific training scheme; the model is related to the hardware platform, that is, different models can be applied to different hardware platforms. Product description file: contains information such as the algorithm library version (mainly characterizing the inference scheme and including information on whether it is forward compatible), the algorithm library API version, the model training scheme, the model capability set, the hardware platform, etc. Among them, the hardware platform information may include the system information and hardware information of the device, the system information may include the operating system information; the hardware information may include the memory information, CPU information, and external device information of the device (such as gimbal information, infrared module information, fill light information, audio module information, radar module information, etc.).

[0043] In an exemplary embodiment, the software publishing platform is used to determine the target algorithm basic package based on the product description file and the target identifier in the following manner: determine the information and device model of the second device that supports the algorithm product based on the product description file; convert the format of the product description file into a predetermined format to obtain a basic package description file; package the target identifier, the information of the second device, the device model, and the basic package description file to obtain the target algorithm basic package. In this embodiment, the product description file can be converted into package description information in a predetermined format to obtain a basic package description file. It is also possible to put all product model information of the same hardware platform that supports the algorithm product, that is, the information and device model of the second device, into the basic package description file. The information and device model of the second device, the target identifier, and the basic package description file are packaged to obtain the algorithm basic package.

[0044] In an exemplary embodiment, the software publishing platform is used to implement the assignment of a target identifier for the algorithm product in the following manner: determining whether the first hardware platform information included in the product description file is the same as the second hardware platform information stored in the product warehouse included in the software publishing platform; if they are the same, obtaining the algorithm product information corresponding to the second hardware platform information from the product warehouse; determining whether the model training scheme included in the product description file is the same as the model training scheme included in the algorithm product information; if they are the same, determining whether the algorithm library version included in the algorithm product is compatible with the algorithm library version included in the algorithm product information; if they are compatible, determining the target product identifier corresponding to the algorithm library version included in the algorithm product information as the target identifier of the algorithm product; if they are not compatible, assigning the target identifier to the algorithm product, wherein the target identifier is different from the target product identifier. In this embodiment, a unique source ID, i.e., a target identifier, can be assigned to the algorithm base package based on the algorithm library version, algorithm library API version, model training scheme, and hardware platform information, and placed in the package description. The source ID changes when any of the algorithm library API version, model training scheme, or hardware platform changes. The source ID changes only when the algorithm library version changes and does not support forward compatibility; otherwise, the source ID remains unchanged. You can query the product warehouse for information about algorithm products on the same hardware platform and compare whether the API version and model training scheme are the same. If they are, continue to check whether the algorithm library version is compatible. If so, extract the source ID of the queried product as the source ID of the current product. Otherwise, assign a unique source ID to the current product.

[0045] In an exemplary embodiment, after the software publishing platform assigns a target identifier to the algorithm product, the method further includes: saving the algorithm product and the target identifier into the product warehouse according to the type of the algorithm product and the hardware platform. In this embodiment, the software publishing platform can store the submitted algorithm product and the obtained source ID (i.e., target identifier) ​​into the product warehouse by type and hardware platform, run the hardware platform information according to the algorithm described in the product, query the product database, obtain all product models of the same hardware platform that support the algorithm, put the source ID, product model table, the algorithm library version, API version, model training scheme, model capability set, hardware platform and other information in the original algorithm product description into the basic package description file, organize the basic package description file and the algorithm product file according to the specifications, and package them into an algorithm basic package. And push the algorithm basic package to the central platform.

[0046] In an exemplary embodiment, after receiving the target model sent by the training platform and packaging the target model according to the model upgrade plan to obtain an algorithm software package, the method further includes: sending the algorithm software package to the target device; if the original model of the same type as the target model is not installed in the target device, instructing the target device to install the algorithm software package to install the target model in the target device; if the original model is already installed in the target device, using the algorithm software package to upgrade the original model. In this embodiment, after packaging the target model to obtain the algorithm software package, the algorithm software package can also be sent to the target device to install the target model corresponding to the algorithm software package in the target device. When the original model of the same type as the target model exists in the target device, the original model can be upgraded. When the original model is not installed in the target device, the target model can be installed in the target device so that the target device can perform tasks corresponding to the target model to meet the needs of the user. During the model installation or upgrade process, you only need to train the training plan in the selected algorithm basic package to obtain the trained model, package the model, generate an upgrade package, that is, the algorithm software package, and install the algorithm software package to realize the installation or upgrade of the model, thereby improving the efficiency of model installation or upgrade when user needs change and the model needs to be installed or upgraded, thereby improving the user experience.

[0047] In the above embodiment, when the model is upgraded or installed, the devices in the product model table that supports the installation of the target model carried in the algorithm software package can be upgraded in batches to improve the upgrade efficiency of the model.

[0048] In an exemplary embodiment, when the original model has been installed in the target device, the user wants to update the original model but continue to use the original model. The user can select the algorithm base package corresponding to the original model from the algorithm warehouse, package the algorithm base package to obtain the algorithm software package, and install the algorithm software package in the target device to update the model.

[0049] The following describes the model packaging method in conjunction with specific implementation methods:

[0050] Figure 4 : is a schematic diagram of the packaging device structure of the model according to a specific embodiment of the present invention, such as Figure 4 As shown, the device includes a software release platform, a central platform, an algorithm repository, and a training platform. The software release platform is used to receive and manage various software delivery products, packaging them as required, and includes a product repository. The central platform connects to the software release platform, manages the algorithm repository, training platform, and access devices. The algorithm repository stores and manages various algorithm packages. The training platform manages training server resources and executes training tasks. Based on model training requests initiated by the central platform, the training platform selects a relatively idle server from its managed cluster of training servers (general hardware platforms) to execute training tasks. The upgrade device pushes the upgrade package to the target device based on the product model table in the upgrade package description and user selection. The algorithm vendor can be the algorithm department of the entire system developer or a third-party developer that adheres to certain development specifications. The algorithm developer and the central platform can agree on the network model types and combinations of these network models. The algorithm developer uses a specific combination as the model training solution based on this agreement. The algorithm product deliverer provides information such as the algorithm library version (primarily corresponding to the inference solution and its compatibility), external API version, model training solution, model capabilities, and hardware platform. The product publisher determines which specific products and devices the algorithm product is applicable to based on the product information provided by the algorithm deliverer, and performs version management on the algorithm product to ensure product compatibility. The model trainer uses the same model training scheme as the algorithm deliverer to ensure that the trained model can be used by the algorithm deliverer's algorithm library. Even if the algorithm library is the same or the version is compatible, different models will lead to business incompatibility, requiring management of model capabilities to ensure business compatibility.

[0051] Figure 5 is a flow chart of a basic package generation algorithm according to a specific embodiment of the present invention, such as Figure 5 As shown in the figure, the algorithm vendor delivers the algorithm product (algorithm library + model + product description) to the software publishing platform. The software publishing platform verifies the product, packages it into an algorithm base package, adds product signal support information, assigns a source ID, and uploads the algorithm base package to the central platform. The central platform verifies the algorithm base package, creates a model training plan, and stores the new plan in the algorithm warehouse.

[0052] Specifically, the algorithm developer delivers the algorithm product: mainly including algorithm library, model, product description and other files.

[0053] 1) Algorithm library: A dynamic library that loads models and performs inference on inputs according to a specific inference scheme; binary files and executable files are bound to the business (executable files, algorithms load models into the library for operation).

[0054] 2) Model: The model file obtained according to a specific training scheme; the model is related to the hardware platform.

[0055] 3) Product description: This includes the algorithm library version (mainly representing the inference solution and including information on whether it is forward compatible), algorithm library API version, model training solution, model capability set, hardware platform, and other information.

[0056] Generate algorithm basic package: package the algorithm product (add device information to the algorithm basic package) intermediate product.

[0057] 1) Convert product description information into package description information in a certain format;

[0058] 2) Put all product model information of the same hardware platform that supports the algorithm product into the package description;

[0059] 3) Assign a unique source ID to the basic package based on the algorithm library version, algorithm library API version, model training scheme, and hardware platform information, and put it into the package description: When any of the algorithm library API version, model training scheme, or hardware platform changes, the source ID changes; the source ID changes only when the algorithm library version changes and does not support forward compatibility; otherwise, the source ID remains unchanged;

[0060] Perform model training: import training materials, select a model training plan, annotate according to the plan requirements, execute training tasks, and obtain the training model and model computing power.

[0061] Generate an algorithm software package (i.e., upgrade package): which can be imported into the device.

[0062] 1) Select the same algorithm base package as the previous model training solution and choose the model replacement solution (inheritance or replacement);

[0063] 2) Decompress the algorithm base package and convert the base package description into an upgrade package description in a certain format;

[0064] 3) Based on the hardware platform information in the basic package description information, the aforementioned training model is optimized and converted into a model that can be used on the hardware platform; if the platform on which the model is executed is different, the model needs to be converted;

[0065] 4) Execute the model replacement scheme for the optimized model:

[0066] ① Inherit the original model: merge the new model with the model in the original base package, take the union of the new model's model capability set and the original model's model capability set as the merged model's model capability set, and put it into the package description of the upgrade package; the model capability set is trained according to the model training plan; in the case of the same source, compare the model capability sets to determine the same system;

[0067] ②Replace the original model: Delete the model in the original base package and put the model capability set of the new model into the package description of the upgrade package.

[0068] Figure 6 is a flow chart of a packaging method for a model according to a specific embodiment of the present invention, such as Figure 6 As shown, the user selects an algorithm base package, chooses a training plan, imports training materials, annotates them according to the plan requirements, and selects a model replacement plan. The central platform retrieves the algorithm base package from the algorithm warehouse and submits the training task to the training platform. The training platform allocates training resources, trains and verifies the model, and selects an optimizer based on the hardware platform for model conversion. The converted model is then sent to the central platform, which replaces the model in the base package according to the replacement plan, updates the model capabilities, packages the algorithm software package (i.e., the model upgrade package), and stores it in the algorithm warehouse.

[0069] Specifically, 1. Algorithm vendors submit their developed algorithm products to the software publishing platform.

[0070] 2. Software Release Platform:

[0071] 1) Identify submitted products and perform verification based on the identified types: developer legitimacy, product legitimacy, and duplicate product submission checks, etc.

[0072] 2) Assign source ID: Query the product warehouse for information on algorithm products on the same hardware platform, compare the API version and model training scheme to see if they are the same, and if so, check whether the algorithm library version is compatible. If so, extract the source ID of the queried product as the source ID of the current product, otherwise assign a unique source ID to the current product.

[0073] 3) The submitted products and the source ID obtained in 2) are stored in the product warehouse according to type and hardware platform.

[0074] 4) Run the hardware platform information according to the algorithm described in the product, query the product database, and obtain all product models with the same hardware platform that support the algorithm.

[0075] 5) Put the source ID, product model table, algorithm library version, API version, model training scheme, model capability set, hardware platform and other information in the original algorithm product description into the basic package description file, that is, the basic package description file includes the source ID, product model table, algorithm library version, API version, model training scheme, model capability set, hardware platform and other information in the original algorithm product description. Among them, the hardware platform information may include the system information and hardware information of the device, and the system information may include the operating system information; the hardware information may include the memory information, CPU information, and external device information of the device (such as gimbal information, infrared module information, fill light information, audio module information, radar module information, etc.). Organize the basic package description file and the algorithm product file according to the specifications and package them into an algorithm basic package.

[0076] 6) Push the algorithm basic package to the central platform.

[0077] 3. Central platform:

[0078] 1) Verify the algorithm basic package:

[0079] ① Basic package legality;

[0080] ② Check for repeated push of basic packages;

[0081] ③ Basic package platform availability: Analyze the hardware platform of the basic package and check whether the current platform supports training the model of the platform (i.e., optimizer support); Analyze the model training plan of the basic package and query whether the training plan exists on the central platform. If not, check whether the training of each basic model that constitutes the training plan is supported. If supported, create the training plan on the central platform. Otherwise, the central platform does not support the basic package.

[0082] 2) Store the basic package in the algorithm warehouse

[0083] 3) Model training:

[0084] ① The user selects a model training plan, imports training materials, and annotates them according to the plan's requirements. Different training plans have different materials and annotation requirements. For example, object detection can eliminate the need for manual annotation and only requires that the object to be detected occupy a large portion of the image (e.g., more than 70%).

[0085] ② The central platform applies for training resources from the training platform, initiates training tasks, and stores the trained model in the algorithm warehouse (the same materials and training plans do not need to be trained repeatedly).

[0086] 4) Generate algorithm software package (upgrade package).

[0087] ①The user selects the algorithm base package and model replacement plan (continue use, merge, replace).

[0088] ② Unzip the base package and organize the contents according to the content specifications of the upgrade package, and convert the base package description information into the corresponding upgrade package description information.

[0089] ③ According to the hardware platform information of the basic package, select the corresponding optimizer and convert the above 3) training model into a model that can be used by the hardware unit under the hardware platform.

[0090] ④ According to the replacement plan, merge or replace the models that come with the original basic package. If merging, merge the two model capability information (for example, if the original model capability is "target cat, white color, ragdoll breed", and the new model capability is "target dog, golden color, teddy breed", the merged model capability is "target cat, white color, ragdoll breed; target dog, golden color, teddy breed"). If replacing, delete the original model.

[0091] ⑤ Replace the original model capabilities described in the upgrade package in ② above with the new model capabilities in ④ above, and append other information (such as signature, certificate, etc.) as needed.

[0092] ⑥Package the new organization files into an algorithm software package (upgrade package).

[0093] 4. Training platform: Based on the model training request initiated by the central platform, a relatively idle server is selected from the managed training server cluster (general hardware platform, such as x86 CPU + NVIDIA GPU) to perform the training task.

[0094] 5. Upgrade device: Push the upgrade package to the target device based on the product model table in the upgrade package description and user selection.

[0095] It should be noted that when the model replacement plan is to continue using, there is no need to import material training, but directly repackage the basic package in the upgrade package format.

[0096] In the aforementioned embodiment, the upgrade package packaging is divided into two stages. The first stage is the basic package packaging, which mainly converts the algorithm product into a standard format and adds version management and product model support information. The second stage is the upgrade package packaging, which mainly performs model replacement. Based on the basic package, the new model and the original model are processed according to the model replacement plan (use, merge, replace), and the capability information of the processed model is used as the model capability information of the upgrade package. The upgrade package description contains at least the following information: source ID, algorithm library version, API version, model training plan, model capability set, hardware platform, product model table, etc. Among them, "source ID, algorithm library version, API version, model capability set" provides a decision basis for the device to install the upgrade package, and "product model table" provides a device screening basis for the central platform to remotely upgrade the device. By assigning a source ID to the algorithm product, the kinship relationship between upgrade packages is characterized. The source ID is calculated based on "algorithm library version, API version, model training plan, hardware platform, etc.". When the source ID is the same and the "model capability set" is in a containment or equivalence relationship, it is considered to be in the same series. The service configuration of the same series packages is fully compatible during the upgrade. That is, it can determine the operational compatibility and business compatibility of different device-specific algorithm software packages produced by the training platform, such as whether the model can run normally after installation, that is, whether installation and upgrade are allowed; it can also determine whether the original configured business needs to be reconfigured after the upgrade, that is, the source ID allocation and the generation process of various configuration files.

[0097] The model training scheme is a combination of some specific network models. The inference scheme of the algorithm library and the model training scheme form a complete set. The material is trained on a general hardware platform according to the training scheme to obtain a general model, and then converted into a model that can be used on a specific hardware platform. The model training scheme is not fixed, so that the algorithm developer and the model trainer can obtain more flexible business and scenario adaptability; the newly trained model and the original model of the algorithm developer are not mutually exclusive and can be merged, so as to facilitate the expansion of new detection types based on the original model and reduce repeated training; the product of the algorithm developer is version managed to determine the relationship between different algorithm software packages. When upgrading the device, the device can judge whether the received package is an upgrade or a new installation of an already installed package. Through the description of the above implementation method, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0098] This embodiment also provides a model packaging device, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0099] Figure 7 is a structural block diagram of a packaging device for a model according to an embodiment of the present invention, such as Figure 7 As shown, the device includes:

[0100] An acquisition module 72 is configured to acquire a target algorithm basic package from the algorithm warehouse based on the received selection instruction, wherein the target algorithm basic package is the algorithm basic package specified in the selection instruction;

[0101] A generation module 74 is configured to generate a training task based on the target algorithm basic package and send the training task to the training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, perform model training according to the target training scheme in the target algorithm basic package, and obtain a target model;

[0102] The receiving module 76 is used to receive the target model sent by the training platform and package the target model according to the model upgrade plan to obtain an algorithm software package.

[0103] In an exemplary embodiment, the receiving module 76 can package the target model according to the model upgrade plan to obtain the algorithm software package in the following manner: decompress the target algorithm basic package to obtain the basic package description information of the target algorithm basic package; determine the information of the first device that supports the installation of the algorithm basic package based on the basic package description information; convert the target model into a format supported by the first device to obtain a conversion model; package the conversion model and the basic package description information according to the model upgrade plan to obtain the algorithm software package.

[0104] In an exemplary embodiment, the receiving module 76 can package the conversion model and the basic package description information according to the model upgrade plan to obtain the algorithm software package in the following manner: when the model upgrade plan is a plan of merging the original model, the configuration file of the conversion model is merged with the configuration file of the original model to obtain the target configuration file, and the target configuration file and the basic package description information are packaged to obtain the algorithm software package; when the model upgrade plan is a plan of replacing the original model, the configuration file of the conversion model and the basic package description information are packaged to obtain the algorithm software package.

[0105] In an exemplary embodiment, the device can be used to receive the target algorithm base package sent by the software publishing platform before obtaining the target algorithm base package from the algorithm warehouse based on the received selection instruction; verify the legitimacy of the target algorithm base package; if the target algorithm base package is legal, determine whether the target algorithm base package exists in the algorithm warehouse; if the target algorithm base package does not exist, determine whether the training platform supports the training scheme included in the target algorithm base package; if supported, store the target algorithm base package in the algorithm warehouse.

[0106] In an exemplary embodiment, the device can implement receiving the target algorithm basic package sent by the software publishing platform in the following manner: receiving the target algorithm basic package sent by the software publishing platform after obtaining the target algorithm basic package in the following manner: verifying the legitimacy of the received algorithm product, wherein the algorithm product includes an algorithm library, a model, and a product description file of the algorithm product; if the algorithm product is legal, assigning a target identifier to the algorithm product; determining the target algorithm basic package based on the product description file and the target identifier.

[0107] In an exemplary embodiment, the software publishing platform is used to determine the target algorithm basic package based on the product description file and the target identifier in the following manner: determine the information and device model of the second device that supports the algorithm product based on the product description file; convert the format of the product description file into a predetermined format to obtain a basic package description file; package the target identifier, the information of the second device, the device model and the basic package description file to obtain the target algorithm basic package.

[0108] In an exemplary embodiment, the software publishing platform is used to assign a target identifier to the algorithm product in the following manner: determine whether the first hardware platform information included in the product description file is the same as the second hardware platform information stored in the product warehouse included in the software publishing platform; if they are the same, obtain the algorithm product information corresponding to the second hardware platform information from the product warehouse; determine whether the model training scheme included in the product description file is the same as the model training scheme included in the algorithm product information; if they are the same, determine whether the algorithm library version included in the algorithm product is compatible with the algorithm library version included in the algorithm product information; if they are compatible, determine the target product identifier corresponding to the algorithm library version included in the algorithm product information as the target identifier of the algorithm product; if they are incompatible, assign the target identifier to the algorithm product, wherein the target identifier is different from the target product identifier.

[0109] In an exemplary embodiment, the software publishing platform may be configured to save the algorithm product and the target identifier into the product warehouse according to the type of the algorithm product and the hardware platform after the software publishing platform assigns a target identifier to the algorithm product.

[0110] In an exemplary embodiment, the device can be used to receive the target model sent by the training platform, package the target model according to the model upgrade plan, and after obtaining the algorithm software package, send the algorithm software package to the target device; if the original model of the same type as the target model is not installed in the target device, instruct the target device to install the algorithm software package to install the target model in the target device; if the original model is already installed in the target device, use the algorithm software package to upgrade the original model.

[0111] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0112] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0113] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0114] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0115] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0116] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0117] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0118] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A model packaging method, characterized in that: include: Obtaining a target algorithm base package from the algorithm warehouse based on the received selection instruction, wherein the target algorithm base package is the algorithm base package specified in the selection instruction; Generate a training task based on the target algorithm basic package, and send the training task to the training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, perform model training according to the target training scheme in the target algorithm basic package, and obtain a target model; Receiving the target model sent by the training platform, and packaging the target model according to the model upgrade plan to obtain an algorithm software package; Before obtaining the target algorithm basic package from the algorithm warehouse based on the received selection instruction, the method also includes: receiving the target algorithm basic package sent by the software publishing platform; wherein, receiving the target algorithm basic package sent by the software publishing platform includes: receiving the target algorithm basic package sent by the software publishing platform after obtaining the target algorithm basic package in the following manner: verifying the legitimacy of the received algorithm product, wherein the algorithm product includes an algorithm library, a model and a product description file of the algorithm product; if the algorithm product is legal, assigning a target identifier to the algorithm product, the target identifier being used to indicate the kinship relationship between target algorithm basic packages; and determining the target algorithm basic package based on the product description file and the target identifier.

2. The method according to claim 1, characterized in that The target model is packaged according to the model upgrade plan to obtain an algorithm software package including: Decompressing the target algorithm base package to obtain base package description information of the target algorithm base package; Determining information of a first device that supports installation of the target algorithm base package based on the base package description information; Converting the target model into a format supported by the first device to obtain a converted model; The conversion model and the basic package description information are packaged according to the model upgrade plan to obtain the algorithm software package.

3. The method according to claim 2, characterized in that The conversion model and the basic package description information are packaged according to the model upgrade plan to obtain the algorithm software package including: In the case where the model upgrade solution is a solution of merging the original models, merging the configuration file of the conversion model with the configuration file of the original model to obtain a target configuration file, and packaging the target configuration file and the basic package description information to obtain the algorithm software package; In the case where the model upgrade solution is a solution of replacing the original model, the configuration file of the conversion model and the basic package description information are packaged to obtain the algorithm software package.

4. The method according to claim 1, wherein Before obtaining the target algorithm base package from the algorithm warehouse based on the received selection instruction, the method further includes: Verify the legitimacy of the target algorithm base package; If the target algorithm base package is legal, determining whether the target algorithm base package exists in the algorithm warehouse; In the absence of the target algorithm base package, determining whether the training platform supports the training scheme included in the target algorithm base package; If supported, the target algorithm base package is stored in the algorithm warehouse.

5. The method according to claim 1, characterized in that The software publishing platform is used to determine the target algorithm base package based on the product description file and the target identifier in the following manner: Determining information and a device model of a second device that supports the algorithm product based on the product description file; Converting the format of the product description file into a predetermined format to obtain a basic package description file; The target identifier, the information of the second device, the device model, and the basic package description file are packaged to obtain the target algorithm basic package.

6. The method according to claim 1, characterized in that The software publishing platform is used to allocate a target identifier to the algorithm product in the following manner: Determining whether the first hardware platform information included in the product description file is the same as the second hardware platform information stored in the product warehouse included in the software publishing platform; In the same situation, obtaining algorithm product information corresponding to the second hardware platform information from the product warehouse; Determining whether the model training scheme included in the product description file is the same as the model training scheme included in the algorithm product information; In the same case, determining whether the algorithm library version included in the algorithm product is compatible with the algorithm library version included in the algorithm product information; In the case of compatibility, the target product identifier corresponding to the algorithm library version included in the algorithm product information is determined as the target identifier of the algorithm product; In case of incompatibility, the target identifier is assigned to the algorithm product, wherein the target identifier is different from the target product identifier.

7. The method according to claim 6, characterized in that After the software publishing platform assigns a target identifier to the algorithm product, the method further includes: The algorithm product and the target identifier are saved in the product warehouse according to the type of the algorithm product and the hardware platform.

8. The method according to claim 1, characterized in that After receiving the target model sent by the training platform and packaging the target model according to the model upgrade plan to obtain an algorithm software package, the method further includes: Sending the algorithm software package to the target device; In a case where an original model of the same type as the target model is not installed in the target device, instructing the target device to install the algorithm software package to install the target model in the target device; In the case that the original model has been installed in the target device, the original model is upgraded using the algorithm software package.

9. A model upgrading device, characterized in that: include: an acquisition module, configured to acquire a target algorithm base package from the algorithm warehouse based on the received selection instruction, wherein the target algorithm base package is the algorithm base package specified in the selection instruction; A generation module is used to generate a training task based on the target algorithm basic package, and send the training task to the training platform to instruct the training platform to obtain training data for training the model in the target algorithm basic package, perform model training according to the target training scheme in the target algorithm basic package, and obtain a target model; A receiving module is used to receive the target model sent by the training platform and package the target model according to the model upgrade plan to obtain an algorithm software package; The device is used to: receive the target algorithm basic package sent by the software publishing platform before obtaining the target algorithm basic package from the algorithm warehouse based on the received selection instruction; wherein, receiving the target algorithm basic package sent by the software publishing platform includes: receiving the target algorithm basic package sent by the software publishing platform after obtaining the target algorithm basic package in the following manner: verifying the legitimacy of the received algorithm product, wherein the algorithm product includes an algorithm library, a model and a product description file of the algorithm product; if the algorithm product is legal, assigning a target identifier to the algorithm product, the target identifier being used to indicate the kinship relationship between target algorithm basic packages; and determining the target algorithm basic package based on the product description file and the target identifier.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 8 when executed.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 8.

Images