A model deployment method, device, equipment and computer readable storage medium

Abstract

The embodiment of the application provides a model deployment method and device, equipment and a computer readable storage medium, relates to the visual technical field, and the method is applied to a visual algorithm development platform, the visual algorithm development platform comprises a visual task module, which is used for building a visual task scheme; the visual task module comprises a first inference module for performing inference by using a model, and the method comprises the following steps: in response to an adding model operation, a first model is acquired; the first model is trained by using sample images to obtain a second model; and the second model is deployed to the first inference module. In this way, the efficiency of model deployment can be improved.

Description

Technical Field

[0001] This application relates to the field of vision technology, and in particular to a model deployment method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of computer technology, visual task solutions can be executed based on visual algorithm development platforms. For example, a visual task workflow can be built within the interface of the visual algorithm development platform, and the built visual task workflow can be run to process images. The visual task workflow includes an inference module for image detection, and a model is deployed in the inference module.

[0003] Currently, model training needs to be completed on a platform used for training the model (called the training platform), and the user must manually import the trained model into the visual algorithm development platform. Only then can the user deploy the corresponding model for the inference module in the visual algorithm development platform. Furthermore, if a model is already deployed in the inference module, and it is determined that the deployed model does not meet the requirements, or if it needs to be iterated, the user needs to manually export the images required for training from the visual algorithm development platform, manually import the images into the training platform to train a new model, and then manually import the new model into the visual algorithm development platform. Throughout this process, the user needs to operate back and forth between the visual algorithm development platform and the training platform, resulting in high operational complexity and reduced model deployment efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a model deployment method, apparatus, device, and computer-readable storage medium to improve the efficiency of model deployment. The specific technical solution is as follows:

[0005] The first aspect of this application provides a model deployment method applied to a visual algorithm development platform, wherein the visual algorithm development platform includes a visual task module for building visual task schemes; the visual task module includes a first inference module that uses a model for inference; the method includes:

[0006] In response to the add model operation, retrieve the first model;

[0007] The first model is trained using sample images to obtain the second model;

[0008] The second model is deployed to the first inference module.

[0009] In an optional embodiment, obtaining the first model in response to the add model operation includes:

[0010] In response to receiving a model creation instruction, create the first model;

[0011] or,

[0012] In response to receiving a model import instruction, the first model indicated by the model import instruction is obtained.

[0013] In an optional embodiment, the step of obtaining the first model indicated by the model import instruction in response to receiving the model import instruction includes:

[0014] In response to receiving a local model import command, retrieve the first model from the local machine;

[0015] or,

[0016] In response to receiving a cloud model import command, the first model is retrieved from the cloud.

[0017] or,

[0018] In response to receiving a shared model import instruction, the first model is retrieved from the models already deployed in the vision task pipeline.

[0019] In one optional embodiment, the first model is obtained in the following manner:

[0020] Models are categorized and displayed according to their type.

[0021] In response to receiving the first selection instruction, the first model is retrieved from the displayed models.

[0022] In one optional embodiment, the first model is obtained in the following manner:

[0023] Displays a model that is compatible with the functionality of the current module;

[0024] In response to receiving a second selection instruction, the first model is retrieved from the displayed models.

[0025] In an optional embodiment, the step of obtaining the first model from the cloud in response to receiving a cloud model import instruction includes:

[0026] Upon receiving a cloud model import command, the cloud login interface is displayed;

[0027] After successful login, the model in the cloud will be displayed;

[0028] In response to receiving a third selection instruction, the first model is retrieved from the displayed models.

[0029] In an optional embodiment, the step of obtaining the first model from the models already deployed in the vision task pipeline in response to receiving a shared model import instruction includes:

[0030] In response to receiving a shared model import instruction, the model already deployed in the vision task workflow is displayed;

[0031] In response to receiving the fourth selection instruction, the first model is retrieved from the displayed models.

[0032] In an optional embodiment, before training the first model using sample images to obtain the second model, the method further includes:

[0033] In response to receiving a sample append instruction, the image indicated by the sample append instruction is appended as a sample image.

[0034] In an optional embodiment, training the first model using sample images to obtain the second model includes:

[0035] In response to receiving the sample append instruction, after appending the image indicated by the sample append instruction as a sample image, the first model is trained using the sample image to obtain the second model;

[0036] or,

[0037] When the number of additional sample images reaches a preset number, the first model is trained using the sample images to obtain the second model.

[0038] or,

[0039] In response to receiving a model training instruction, the first model is trained using sample images to obtain a second model.

[0040] In an optional embodiment, before appending the image indicated by the sample append instruction as a sample image in response to receiving the sample append instruction, the method further includes:

[0041] Deploy the first model to the first inference module;

[0042] The first inference module is used to detect the image to be detected, and the detection result is obtained.

[0043] The step of responding to receiving a sample append instruction by appending the image indicated by the sample append instruction as a sample image includes:

[0044] In response to receiving a sample append instruction, the detected image indicated by the sample append instruction is appended as a sample image; wherein the sample append instruction is triggered based on the detection result.

[0045] In an optional embodiment, the method further includes:

[0046] This displays the training status of the model.

[0047] In an optional embodiment, the training state includes at least one of the following:

[0048] The training progress of the model;

[0049] This indicates the training result, whether the training was successful or unsuccessful.

[0050] Reasons for training failure.

[0051] In an optional embodiment, the method further includes:

[0052] The second model is deployed in batches to at least one second inference module included in the vision task scheme.

[0053] In an optional embodiment, the detection type of the second inference module is consistent with the detection type of the first inference module.

[0054] In an optional embodiment, before batch deploying the second model to at least one second inference module included in the vision task scheme, the method further includes:

[0055] Display the inference module in the visual task scheme that has the same detection type as the first inference module;

[0056] In response to the fourth selection instruction, a second inference module is determined from the displayed inference modules.

[0057] In an optional embodiment, before training the first model using sample images to obtain the second model, the method further includes:

[0058] Images used to train the first model are determined from the real-time acquired images and used as sample images.

[0059] In an optional embodiment, determining the images used for training the first model from the real-time acquired images, as sample images, includes:

[0060] In response to receiving a single image addition instruction, the currently acquired single image is determined as a sample image;

[0061] or,

[0062] In response to receiving a batch image addition instruction, the acquired multiple images are identified as sample images.

[0063] In an optional embodiment, training the first model using sample images to obtain the second model includes:

[0064] The training settings interface is displayed; wherein, the training settings interface displays: sample images and label setting areas;

[0065] Obtain the labels that are used to annotate the sample image in the label setting area;

[0066] The first model is trained using sample images and acquired labels to obtain the second model.

[0067] In an optional embodiment, the first model and the second model are models with adjustable parameters.

[0068] A second aspect of this application provides a model deployment apparatus applied to a visual algorithm development platform, the visual algorithm development platform including a visual task module for building visual task schemes; the visual task module including a first inference module for inference using a model; the apparatus comprising:

[0069] The first model acquisition unit is used to acquire the first model in response to the add model operation;

[0070] The training unit is used to train the first model using sample images to obtain the second model;

[0071] The deployment unit is used to deploy the second model to the first inference module.

[0072] In an optional embodiment, the first model acquisition unit is specifically configured to create a first model in response to receiving a model creation instruction;

[0073] or,

[0074] In response to receiving a model import instruction, the first model indicated by the model import instruction is obtained.

[0075] In an optional embodiment, the first model acquisition unit is specifically configured to acquire a first model from the local source in response to receiving a local model import instruction;

[0076] or,

[0077] In response to receiving a cloud model import command, the first model is retrieved from the cloud.

[0078] or,

[0079] In response to receiving a shared model import instruction, the first model is retrieved from the models already deployed in the vision task pipeline.

[0080] In an optional embodiment, the first model acquisition unit is specifically configured to acquire the first model in the following manner:

[0081] Models are categorized and displayed according to their type.

[0082] In response to receiving the first selection instruction, the first model is retrieved from the displayed models.

[0083] In an optional embodiment, the first model acquisition unit is specifically configured to acquire the first model in the following manner:

[0084] Displays a model that is compatible with the functionality of the current module;

[0085] In response to receiving a second selection instruction, the first model is retrieved from the displayed models.

[0086] In an optional embodiment, the first model acquisition unit is specifically used to display the cloud login interface in response to receiving a cloud model import instruction;

[0087] After successful login, the model in the cloud will be displayed;

[0088] In response to receiving a third selection instruction, the first model is retrieved from the displayed models.

[0089] In an optional embodiment, the first model acquisition unit is specifically configured to display the models deployed in the visual task process in response to receiving a shared model import instruction;

[0090] In response to receiving the fourth selection instruction, the first model is retrieved from the displayed models.

[0091] In an optional embodiment, the apparatus further includes:

[0092] An appending unit is configured to, in response to receiving a sample appending instruction, append the image indicated by the sample appending instruction as a sample image before training the first model using the sample images to obtain the second model.

[0093] In an optional embodiment, the training unit is specifically configured to, in response to receiving the sample append instruction, append the image indicated by the sample append instruction as a sample image, and then train the first model using the sample image to obtain the second model;

[0094] or,

[0095] When the number of additional sample images reaches a preset number, the first model is trained using the sample images to obtain the second model.

[0096] or,

[0097] In response to receiving a model training instruction, the first model is trained using sample images to obtain a second model.

[0098] In an optional embodiment, the apparatus further includes:

[0099] The detection unit is configured to deploy the first model to the first inference module before appending the image indicated by the sample appending instruction to the sample image in response to receiving the sample appending instruction;

[0100] The first inference module is used to detect the image to be detected, and the detection result is obtained.

[0101] The appending unit is specifically configured to append the detected image indicated by the sample appending instruction to the sample image in response to receiving the sample appending instruction; wherein the sample appending instruction is triggered based on the detection result.

[0102] In an optional embodiment, the apparatus further includes:

[0103] The training status display unit is used to display the training status of the model.

[0104] In an optional embodiment, the training state includes at least one of the following:

[0105] The training progress of the model;

[0106] This indicates the training result, whether the training was successful or unsuccessful.

[0107] Reasons for training failure.

[0108] In an optional embodiment, the apparatus further includes:

[0109] A batch deployment unit is used to batch deploy the second model to at least one second inference module included in the vision task scheme.

[0110] In an optional embodiment, the detection type of the second inference module is consistent with the detection type of the first inference module.

[0111] In an optional embodiment, the apparatus further includes:

[0112] An inference module determination unit is used to display, before deploying the second model in batches to at least one second inference module included in the vision task scheme, an inference module whose detection type is consistent with that of the first inference module.

[0113] In response to the fourth selection instruction, a second inference module is determined from the displayed inference modules.

[0114] In an optional embodiment, the apparatus further includes:

[0115] The sample image determination unit is used to determine, as sample images, images from real-time acquired images before training the first model using sample images to obtain the second model.

[0116] In an optional embodiment, the sample image determination unit is specifically configured to determine the currently acquired single image as a sample image in response to receiving a single image addition instruction;

[0117] or,

[0118] In response to receiving a batch image addition instruction, the acquired multiple images are identified as sample images.

[0119] In an optional embodiment, the training unit is specifically used to display a training settings interface; wherein the training settings interface displays: sample images and a label setting area;

[0120] Obtain the labels that are used to annotate the sample image in the label setting area;

[0121] The first model is trained using sample images and acquired labels to obtain the second model.

[0122] In an optional embodiment, the first model and the second model are models with adjustable parameters.

[0123] A third aspect of this application provides an electronic device, comprising:

[0124] Memory, used to store computer programs;

[0125] When a processor executes a program stored in memory, it implements any of the methods described in the first aspect above.

[0126] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described in the first aspect above.

[0127] The fifth aspect of this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described in the first aspect above.

[0128] Beneficial effects of the embodiments in this application:

[0129] The model deployment method provided in this application embodiment is applied to a visual algorithm development platform. The visual algorithm development platform includes a visual task module for building a visual task scheme. The visual task module includes a first inference module that uses a model for inference. The method includes: in response to an add model operation, obtaining a first model; training the first model using sample images to obtain a second model; and deploying the second model to the first inference module.

[0130] Based on the model deployment method provided in this application, the visual algorithm development platform can acquire a first model, train the first model based on sample images, and deploy the trained second model. That is, the visual algorithm development platform can achieve integrated processing of model acquisition, training, and deployment, eliminating the need for users to operate back and forth between the visual algorithm development platform and the training platform, reducing operational complexity, and improving the efficiency of model deployment.

[0131] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0132] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0133] Figure 1 A flowchart illustrating the first model deployment method provided in this application embodiment;

[0134] Figure 2 A schematic diagram of a first type of process management interface provided in an embodiment of this application;

[0135] Figure 3 A schematic diagram of a second process management interface provided in an embodiment of this application;

[0136] Figure 4 A schematic diagram illustrating a login interface provided in an embodiment of this application;

[0137] Figure 5 A schematic diagram illustrating a model for displaying cloud storage, provided as an embodiment of this application;

[0138] Figure 6 A schematic diagram of a third process management interface provided in an embodiment of this application;

[0139] Figure 7 A schematic diagram of a fourth process management interface provided in an embodiment of this application;

[0140] Figure 8 A schematic diagram of the fifth process management interface provided in the embodiments of this application;

[0141] Figure 9 A schematic diagram of the sixth process management interface provided in the embodiments of this application;

[0142] Figure 10 A schematic diagram illustrating a batch deployment interface provided in an embodiment of this application;

[0143] Figure 11 A schematic diagram of a training settings interface provided in an embodiment of this application;

[0144] Figure 12 A flowchart illustrating the second model deployment method provided in this application embodiment;

[0145] Figure 13 This is a schematic diagram of the structure of a model deployment device provided in an embodiment of this application;

[0146] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0147] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0148] Currently, model training typically requires the user to manually import the trained model into the visual algorithm development platform. Only then can the user deploy the corresponding model for the inference module within the visual algorithm development platform. This process involves users switching back and forth between the visual algorithm development platform and the training platform, resulting in high operational complexity and reduced model deployment efficiency.

[0149] To address the aforementioned technical problems, embodiments of this application provide a model deployment method, apparatus, device, and computer-readable storage medium.

[0150] The model deployment method provided in this application embodiment will be described below. This model deployment method can be applied to a visual algorithm development platform. For example, the visual algorithm development platform is a software client installed in an electronic device. Users can build visual task flows in the interface (which can be called the process management interface) of the visual algorithm development platform according to business needs, and run the built visual task flows to process images. The visual task flow includes an inference module for image detection, and a model is deployed in the inference module.

[0151] In one embodiment, the model deployment method is applied to a visual algorithm development platform. The visual algorithm development platform includes a visual task module for building visual task schemes. The visual task module includes a first inference module that uses models for inference. The method includes: in response to an add model operation, acquiring a first model; training the first model using sample images to obtain a second model; and deploying the second model to the first inference module. Based on the model deployment method provided in this application embodiment, the visual algorithm development platform can acquire a first model, train the first model based on sample images, and deploy the trained second model. That is, the acquisition, training, and deployment of models can be integrated in the visual algorithm development platform, eliminating the need for users to operate back and forth between the visual algorithm development platform and the training platform, reducing operational complexity, and improving the efficiency of model deployment.

[0152] In one embodiment, see Figure 1 The model deployment method provided in this application includes the following steps:

[0153] S101: In response to the add model operation, retrieve the first model.

[0154] S102: Train the first model using sample images to obtain the second model.

[0155] S103: Deploy the second model to the first inference module.

[0156] The first inference module can be any inference module in the visual task flow. That is, based on the method provided in this application embodiment, the trained model can be deployed to any inference module in the visual task flow. It is understood that the visual task flow may also include non-inference modules, such as modules for acquiring images and modules for outputting image detection results.

[0157] In one embodiment, any inference module in the visual task flow has a corresponding visual detection function, and each inference module indicates a detection type. For example, the user creates an inference module for the required detection type in the visual task flow. If an inference module is used to segment objects in an image, the detection type indicated by the inference module is image segmentation, and the model to be deployed in the inference module is an image segmentation model; if an inference module is used to recognize characters in an image, the detection type indicated by the inference module is character recognition, and the model to be deployed in the inference module is a character recognition model; if an inference module is used to recognize targets in an image, the detection type indicated by the inference module is target detection, and the model to be deployed in the inference module is a target detection model.

[0158] A vision task flow represents a complete process of visual processing of an image. Besides the vision task module, a vision task flow can also include other modules, which may also deploy corresponding models. For example, a vision task flow might include: an image acquisition module, an image visual inspection module (i.e., an inference module), and an output module for the inspection results. It's understandable that multiple vision task flows can exist simultaneously in the same scenario, forming a vision task scheme. For instance, in a scenario involving the inspection of printed circuit boards (PCBs), the vision task scheme might include multiple vision task flows, each inspecting the PCB from different dimensions. One vision task flow might detect whether components are correctly mounted on the PCB, another might detect soldering defects, and yet another might detect anomalies in the information codes on the PCB.

[0159] In one embodiment, in the process management interface, each module in the visual task process can be represented in the form of a block diagram, and the data flow direction of the visual task process can be represented by arrows between the block diagrams. See also Figure 2 The process management interface includes a task process setup area 201 and a module settings area 202. Figure 2 In the task flow setup area 201, four visual task flows (flow one, flow two, flow three, and flow four) have been created. Currently, the modules in flow one are displayed, including the image acquisition module, image matching module, and image segmentation module. For any module currently displayed in task flow setup area 201, if the module is selected, its information is displayed in module settings area 202. Users can configure modules through operations in module settings area 202. For example, if an inference module (image segmentation module) currently displayed in task flow setup area 201 is selected, a "Select Model" component is provided in module settings area 202, allowing users to configure the model deployed in that inference module.

[0160] In some embodiments, the Add Model operation can also indicate the detection type. For example, if the Add Model operation is triggered for a first inference module, then in response to the Add Model operation, the visual algorithm development platform determines the detection type of the first inference module and obtains a first model for that detection type.

[0161] Alternatively, the model addition operation may not specify a detection type. In this case, the first model obtained can be the default detection type. Subsequently, after training the second model, a first inference module is determined that matches the detection type of the first model, and the second model is deployed to the first inference module. The default detection type can be determined based on the current vision task scheme.

[0162] In one embodiment, the first model and the second model are models with adjustable parameters. That is, the first model and the second model have model structures and model parameters. The process of training the model is also the process of adjusting the model parameters.

[0163] For example, the first and second models can be artificial intelligence (AI) models, such as deep learning models. Alternatively, they can be traditional machine learning models, such as random forest models and logistic regression models. In different application scenarios, the first and second models can be object detection models, semantic segmentation models, instance segmentation models, etc.

[0164] It is understandable that some useful models in existing technologies may involve a "model training" process. However, the "model training" they refer to essentially does not involve adjusting model parameters; it merely increases the mapping relationship between the features extracted by the model and the inference results. For example, the character recognition model used previously was essentially a mapping library of extracted graphic features and character recognition results. The "training process" of this model included: extracting graphic features using conventional methods such as extracting grayscale features and texture features, and then having the user label the characters corresponding to those graphic features, thus completing the "training." This training did not involve optimizing the feature extraction process; the extracted features were fixed, and only the extracted features and characters were mapped. This approach itself does not have a model with model parameters, and correspondingly, it does not adjust the model parameters.

[0165] The model in this application is trained using sample images, meaning that the model parameters are adjusted using sample images. For example, gradients are calculated via backpropagation, and model parameters are optimized via gradient descent. In other words, the visual algorithm development platform in this application can handle all stages of adjusting the model parameters.

[0166] Since the second model is trained based on the first model, the detection types of the second model are the same as those of the first model. The second model and the first model can be understood as different versions of the same model.

[0167] A model is deployed to an inference module, meaning the visual algorithm development platform records the model used by that inference module. Correspondingly, during the execution of the visual task flow belonging to that inference module, when the execution progress reaches that inference module, the visual algorithm development platform can use the recorded model used by that inference module to perform inference on the image.

[0168] In one embodiment, the visual algorithm development platform's interface includes a unified acquisition component for obtaining models. That is, in this embodiment, the "add model" operation that triggers the acquisition of the first model is not targeted at the first inference module. In this case, when the user triggers the acquisition component, the visual algorithm development platform can also display a detection type setting option. The user can trigger this setting option to specify the detection type of the model to be acquired, and the visual algorithm development platform acquires the first model of the detection type set by the user. Correspondingly, after training the second model, the visual algorithm development platform can deploy the second model to the first inference module according to the user's selection. Alternatively, in this case, when the user triggers the acquisition component, the visual algorithm development platform acquires the model of the default detection type as the first model.

[0169] In one embodiment, the model addition operation is performed on the first inference module. If the first inference module is a first detection type, the visual algorithm development platform can acquire a first model of that first detection type. For example, the interface of the visual algorithm development platform may include a corresponding acquisition component for acquiring the first model for the first inference module. When the user triggers this acquisition component, the visual algorithm development platform can acquire a first model that matches the detection type of the first inference module. In this case, the visual algorithm development platform can automatically determine the detection type of the first inference module, eliminating the need for the user to manually set the required detection type of the first model, thus simplifying user operation. Correspondingly, after training the second model, the visual algorithm development platform automatically deploys the second model to the first inference module; alternatively, after obtaining the second model, the visual algorithm platform can deploy the second model to the first inference module according to the user's instructions.

[0170] Visual algorithm development platforms can obtain the first model through at least one of the following methods:

[0171] One method (referred to as Method 1):

[0172] Step S101 above includes: creating a first model in response to receiving a model creation instruction. Thus, the visual algorithm development platform supports the creation of new models, meeting different business needs of users.

[0173] In one embodiment, the model creation command may be triggered for the first inference module. When the user triggers the model creation command, since the visual algorithm development platform can automatically identify the detection type (i.e., the first detection type) of the first inference module, a first model of the first detection type can be automatically created.

[0174] like Figure 3 As shown, in Figure 2 Based on this, when the first inference module (such as the image segmentation module) is selected, the user triggers the "Select Model" component in the module setting area 202. The visual algorithm development platform displays a window (called the Select Model window), which has a "Create Model" component. When the user triggers the "Create Model" component, the visual algorithm development platform receives a model creation instruction for the first inference module.

[0175] Understandable Figure 3 As an example only, the "New Model" component can also be set outside the model selection window. For example, it can be set in another location in the module settings area 202.

[0176] In one embodiment, the model creation command is not triggered for any inference module. When a user triggers the model creation command, the vision algorithm development platform can automatically create a first model of the default detection type.

[0177] Another way:

[0178] Step S101 above includes: in response to receiving a model import instruction, obtaining the first model indicated by the model import instruction.

[0179] In one embodiment, the model import command can be triggered for the first inference module. When the user triggers the model import command, since the visual algorithm development platform can automatically identify the detection type of the first inference module, it can automatically import the existing first model of the first detection type.

[0180] For example, such as Figure 2 and Figure 3 As shown, when the first inference module is selected, the user triggers the "Model Import" component in the module settings area 202 to trigger the model import command.

[0181] In one embodiment, the model import command is not triggered for any particular inference module. When a user triggers the model import command, the visual algorithm development platform can automatically import an existing first model of the default detection type.

[0182] In one embodiment, the visual algorithm development platform can import the first model through at least one of the following methods:

[0183] Method 2: In response to receiving a local model import command, retrieve the first model from the local machine. This supports importing the first model from the local machine, meeting different business needs of users.

[0184] In one embodiment, the visual algorithm development platform displays models local to the electronic device, and the user can select a first model from the displayed models.

[0185] Method 3: Upon receiving a cloud model import command, retrieve the first model from the cloud. This supports importing the first model from the cloud, meeting different user business needs.

[0186] In one embodiment, the visual algorithm development platform displays models in the cloud, and the user can select a first model from the displayed models.

[0187] Method 4: In response to receiving a shared model import command, retrieve the first model from the models already deployed in the vision task workflow. This supports importing the first model from other inference modules, meeting different user business needs.

[0188] In one embodiment, the visual algorithm development platform displays the models deployed in the visual task flow, and accordingly, in response to receiving a fourth selection instruction, selects a first model from the displayed models.

[0189] For example, in response to receiving a shared model import instruction, the vision algorithm platform displays all the models deployed in the vision task process included in the currently built vision task solution.

[0190] or,

[0191] In response to receiving a shared model import command, the visual algorithm platform displays the models deployed in a portion of the visual task flows included in the currently constructed visual task solution. For example, if the shared model import command is triggered for the first inference module, the visual algorithm platform displays the models deployed in the visual task flow to which the first inference module belongs. Alternatively, the visual algorithm platform displays the models deployed in other visual task flows besides those to which the first inference module belongs.

[0192] In one embodiment, the above method three includes the following steps: in response to receiving a cloud model import instruction, displaying a cloud login interface; after successful login, displaying a model in the cloud; and in response to receiving a third selection instruction, retrieving a first model from the displayed models. This verifies the request to retrieve a model from the cloud, improving the security of models stored in the cloud.

[0193] In this embodiment, when a user triggers a cloud-based model import command, the visual algorithm development platform displays the cloud-based login interface. See also... Figure 4 The visual algorithm development platform displays a login interface 203 on the cloud. For example, users can enter their username and password on this login interface 203. Then, after successful login, the visual algorithm development platform displays models stored in the cloud for the user to select.

[0194] After successful login, see Figure 5 The visual algorithm development platform displays a new window (called the cloud model display window) 204, in which the model stored in the cloud is displayed. Figure 5 In this example, we will use the cloud model import command triggered by the first inference module as an example. Since the detection type indicated by the first inference module is image segmentation, therefore... Figure 5 The information displayed shows the details of each segmentation model (including image segmentation model 1 and image segmentation model 2) stored in the cloud, including model identifier, version number, training error, precision, recall, training start time, and training duration. Figure 5 The "XXX" in the text indicates the specific information to be displayed.

[0195] In one embodiment, see Figure 6 ,exist Figure 2 Based on this, when a user triggers the "Model Import" component, the visual algorithm development platform can display import model options, including "Local Import" and "Cloud Import" options. Users can import models from local or cloud sources by triggering different options.

[0196] In one embodiment, for methods two through four described above, the first model can be obtained by: classifying and displaying models according to model type; and in response to receiving a first selection instruction, retrieving the first model from the displayed models. This facilitates user selection of the first model.

[0197] For any of methods two through four, models can be categorized and displayed according to their type. Model type includes: detection type and / or model version. For example, models of the same detection type can be displayed in the same area, and different versions of the same model can be displayed in the same area. Alternatively, multiple models of the same version can be displayed in the same area.

[0198] In one embodiment, for any of methods two through four, the model can also be displayed in other ways. For example, the models can be displayed in chronological order of their creation time, or in a random order, etc.

[0199] In one embodiment, for any of the above methods two to four, the first model can be obtained by: displaying a model that is compatible with the function of the current module; and in response to receiving a second selection instruction, obtaining the first model from the displayed models.

[0200] For any of the methods two through four, models can be filtered during display to show only models of one detection type. Functional adaptation ensures consistency across detection types. If the model import command is triggered for an inference module, then that inference module is the current module. For example, if the model import command is triggered for the first inference module, then the current module is the first inference module, and the visual algorithm development platform will display models of the first detection type.

[0201] For example, regarding method two, if the current module is the first inference module, in response to receiving the local model import instruction, the visual algorithm development platform displays models of the first detection type stored locally on the electronic device. Correspondingly, the user can select at least one model as the first model from the displayed models. For instance, the visual algorithm development platform can pop up a new window displaying models of the first detection type stored locally on the electronic device; or, the visual algorithm development platform can display models of the first detection type stored locally on the electronic device in an area of ​​the process management interface. For each model of the first detection type, model information can be displayed. Model information can include at least one of the following: model identifier, training error, precision, recall, training start time, and training duration.

[0202] In one embodiment, for a first model imported from local storage, the visual algorithm development platform can train the first model using sample images; alternatively, for a first model imported from local storage, the visual algorithm development platform can first deploy the first model to the first inference module and execute the corresponding visual task process to determine the detection performance of the first model, and then select sample images to train the first model based on the detection results. For details, please refer to the description of the following embodiments.

[0203] For example, regarding method three, if the current module is the first inference module, in response to receiving the cloud model import instruction, the visual algorithm development platform can display models of the first detection type stored in the cloud. Correspondingly, the user can select at least one model as the first model from the displayed models. For instance, the visual algorithm development platform can pop up a new window displaying models of the first detection type stored in the cloud; or, the visual algorithm development platform can display models of the first detection type stored in the cloud in an area of ​​the process management interface. For each model of the first detection type, model information can be displayed.

[0204] In one embodiment, for a first model imported from the cloud, the visual algorithm development platform can train the first model using sample images; alternatively, for a first model imported from the cloud, the visual algorithm development platform can first deploy the first model to the first inference module and execute the corresponding visual task process to determine the detection performance of the first model, and then select sample images to train the first model based on the detection results. For details, please refer to the description of the following embodiments.

[0205] For example, regarding method four, if the current module is the first inference module, in response to receiving the shared model import instruction, the visual algorithm development platform can display the models of the first detection type already deployed in the visual task flow. Correspondingly, the user can select a model from the displayed models as the first model. For instance, the visual algorithm development platform can pop up a new window displaying the models of the first detection type already deployed in the visual task flow; or, the visual algorithm development platform can display the models of the first detection type already deployed in the visual task flow in an area of ​​the flow management interface. For each model of the first detection type, model information can be displayed.

[0206] In one embodiment, before step S102, the method further includes:

[0207] Step 1: In response to receiving a sample append command, append the image specified in the command as a sample image. This allows users to append training samples so that the trained second model meets business requirements.

[0208] In one embodiment, a user can trigger a sample append command for locally imported images. Correspondingly, the visual algorithm development platform can acquire images locally from the electronic device as sample images to train a first model. Alternatively, the user can also trigger a sample append command for images acquired in real-time.

[0209] The above-described embodiments for adding samples, combined with any of the above-described embodiments for obtaining the first model, as well as subsequent embodiments for displaying the training status, batch deployment of the model, training the model, and obtaining real-time acquired images as samples, can be combined to achieve a model deployment scheme.

[0210] In one embodiment, prior to step one, the method further includes: step two: deploying the first model to the first inference module; using the first inference module to detect the image to be detected, and obtaining detection results; correspondingly, step one includes: in response to receiving a sample appending instruction, appending the detected image indicated by the sample appending instruction as a sample image; wherein, the sample appending instruction is triggered based on the detection results. Thus, it supports appending detected images as sample images, enabling the trained second model to effectively identify difficult examples and improve model performance.

[0211] In this embodiment, after obtaining the first model, the image is detected using a first inference module deployed with the first model. After obtaining the detection results, the visual algorithm development platform can display them. After viewing the detection results, the user can determine whether the detection results are accurate. If the detection results are inaccurate, it indicates that the current first model cannot effectively detect the image; therefore, the detected image can be appended as a sample image to train the first model.

[0212] In one embodiment, there are several ways to train the first model using additional sample images.

[0213] Method 1: In response to receiving a sample append instruction, after appending the image indicated by the sample append instruction as a sample image, the first model is trained using the sample image to obtain the second model.

[0214] Method 2: In response to the number of additional sample images reaching a preset number, the first model is trained using the sample images to obtain the second model.

[0215] Method 3: In response to receiving the model training instruction, the first model is trained using sample images to obtain the second model.

[0216] For example, when displaying the detection result of a detected image, the visual algorithm development platform can also display an append component. When the user triggers this append component, the visual algorithm development platform can use the detected image to train the first model; that is, after appending an image, it automatically uses the appended image to train the model (i.e., method 1 above). Alternatively, when the user triggers the append component, the visual algorithm development platform marks the detected image as a sample image. In this case, it does not directly use the detected image to train the first model. Subsequently, in response to receiving a training instruction for the first model triggered by the user, the visual algorithm development platform uses all currently labeled sample images to train the first model (i.e., method 3 above). Alternatively, when the number of appended sample images reaches a preset number (e.g., 20, 40, or 60), the visual algorithm development platform uses all currently labeled sample images to train the first model (i.e., method 2 above).

[0217] See Figure 7 The workflow management interface also includes a detection result display area 205, which comprises a thumbnail display sub-area 2051, a large image display sub-area 2052, and a result display sub-area 2053. The thumbnail display sub-area 2051 displays thumbnails of the detected images; the large image display sub-area 2052 displays the detected images that are currently selected; and the result display sub-area 2053 displays the detection results for the selected detected images. When the user triggers the "Add Sample" component in the workflow management interface based on the detection results displayed in the result display sub-area 2053, the visual algorithm development platform marks the currently selected image to be detected as a sample image. Figure 7 In the image, the diagonal rectangle represents both the thumbnail and the full-size image.

[0218] In one embodiment, the visual algorithm development platform can also display the training status of the model, allowing users to easily understand the current training status of the model.

[0219] In this embodiment, since the visual task flow may include multiple inference modules, correspondingly, the visual algorithm development platform may train multiple models in response to user operations. To facilitate user management of the models, the training status of the models can be displayed. For example, the visual algorithm development platform can display the training status in an area of ​​the flow management interface, or it can display a new interface and show the training status in that interface.

[0220] The above-described embodiment of displaying the training status, combined with any of the above-described embodiments of obtaining the first model, adding samples, and subsequent batch deployment of models, and the embodiment of training models, and the embodiment of obtaining real-time acquired images as samples, implements a model deployment scheme.

[0221] In one embodiment, the visual algorithm development platform displays the training status (including detailed training status as described below) of all models (including models that have not completed training and those that have completed training). Alternatively, the visual algorithm development platform may also display detailed training status based on user actions.

[0222] In one embodiment, the training status includes at least one of the following: the model's training progress, the training result indicating successful or failed training, and the reason for training failure. This allows users to easily understand the training progress and specific training status of each model.

[0223] In one embodiment, displaying the training status of a model includes: displaying the number of models that have not yet completed training and the number of models that have completed training; in response to receiving a request to display details of incomplete training, displaying the detailed training status of models currently being trained and the detailed training status of models waiting to be trained; and in response to receiving a request to display details of completed training, displaying the detailed training status of models that have completed training. This allows users to easily understand the specific training details of the model.

[0224] The detailed training status of an incomplete training model can include at least one of the following: model identifier, detection type, version number, and training progress. Additionally, for incomplete training models, a "Stop Training" component can be displayed; when the user triggers this component, the visual algorithm development platform cancels training for that model. Furthermore, for models that have failed to train, a "Retrain" component can be displayed; when the user triggers this component, the visual algorithm development platform retrains the model.

[0225] For example, see Figure 8 ,exist Figure 7 Based on this, the process management interface includes a number display area 206, which displays the number of models that have not yet completed training and the number of models that have completed training. Figure 8 The "Task in progress 3 / 5" message indicates that there are currently 5 model training tasks, with 3 still incomplete, meaning that 2 models have been trained.

[0226] Correspondingly, when the user triggers the display area 206, the visual algorithm development platform can display a window 207, showing both "In Progress" and "Completed" options. When the user triggers the "In Progress" option, the visual algorithm development platform displays the detailed training status of the currently training model and the detailed training status of the models waiting to be trained in window 207; when the user triggers the "Completed" option, the visual algorithm development platform displays the detailed training status of the models that have been trained in window 207. Figure 8 In the text, "3 in progress" indicates 3 models that have not yet completed training, and "2 completed" indicates 2 models that have completed training. Figure 8 When the user triggers the "In Progress" option, it displays the detailed training status of the three models that have not yet completed training (Image Segmentation Model 1, Object Detection Model 2, and Character Recognition Model 1). Image Segmentation Model 1 is the model that is currently being trained, displaying its training progress (i.e., the remaining training time). Object Detection Model 2 is the model waiting to be trained, displayed as "Task Queued". Character Recognition Model 1 is the model that failed to train, displayed as "Training Failed". When the user triggers the "Retry" component, the visual algorithm development platform retrains Character Recognition Model 1. Figure 8 In the middle, for each model, there is also an "×" component, which indicates the "End Training" component.

[0227] The detailed training status of a trained model can include at least one of the following: the model identifier, detection type, version number, training progress, and training duration.

[0228] In one embodiment, the method further includes the step of: batch deploying the second model to at least one second inference module included in the vision task scheme. This further improves the deployment efficiency of the model.

[0229] The above-described embodiment of batch deployment of models, combined with any of the above-described embodiments of obtaining the first model, adding samples, displaying the training status, training the model subsequently, and obtaining real-time acquired images as samples, can realize a model deployment scheme.

[0230] For example, after obtaining the second model, the visual algorithm development platform automatically deploys the second model to both the first inference module and the second inference module in batches.

[0231] Alternatively, after obtaining the second model, the visual algorithm development platform deploys it to the first inference module. Subsequently, based on user actions, the second model is deployed in batches to the second inference module.

[0232] In one embodiment, the detection type of the second inference module is consistent with the detection type of the first inference module.

[0233] In one embodiment, the second inference module is determined through the following steps: displaying inference modules in the visual task scheme that have the same detection type as the first inference module; and, in response to a fourth selection instruction, determining the second inference module from the displayed inference modules. This process filters out and displays inference modules with the same detection type as the first inference module, facilitating user selection of the second inference module and improving model deployment efficiency.

[0234] In one embodiment, after obtaining the second model and deploying it to the first inference module, the visual algorithm development platform can display information about the second model in the process management interface. For example... Figure 9 As shown, in Figure 2 Based on this, if the image segmentation module is selected in the task flow setup area 201, the module settings area 202 will display information about the model deployed in that image segmentation module (i.e., image segmentation model 3). The model information may include at least one of the following: model identifier (i.e., image segmentation model 3), creation time (i.e., training completion time), test results, test parameters, annotation information, label information, and remarks. Test results include at least one of the following: precision, recall, and F1 score. Test parameters include at least one of the following: number of images, confidence score, maximum overlap rate, and intersection over union (IOU). Annotation information includes the ratio of labeled to unlabeled images. Label information includes the number of images with different labels. Figure 9 In this context, "XXX" represents specific information.

[0235] Additionally, see Figure 9 The module settings area 202 also includes a "Batch Deployment" component. Correspondingly, when the user triggers this component, the visual algorithm development platform displays a batch deployment interface for the model (i.e., image segmentation model 3). See also... Figure 10 The batch deployment interface 208 includes a module selection list area 2081 and a selected list area 2082. The module selection list area 2081 is used to display the module identifier of the inference module that is consistent with the detection type of the first inference module in the visual task scheme, as well as the flow identifier of the visual task process to which each inference module belongs. The selected list area 2082 is used to display the module identifier of the second inference module selected by the user through the selection command.

[0236] Figure 10In the module selection list area 2081, the module identifiers of inference modules (including modules 1, 2, 5, and 6) that match the detection type of the first inference module in the visual task solution are displayed, along with the flow (i.e., the visual task flow, including flow 1, flow 3, and flow 5) to which each module belongs. Users can select the modules (including modules 2 and 6) that need to be deployed in batches for the second model, and the module identifiers of modules 2 and 6 are displayed accordingly in the selected list area 2082. Additionally, users can also deselect the modules using the deselection component displayed in the selected list area 2082. Figure 10 (The "×" component) can be used to deselect the corresponding module.

[0237] In one embodiment, if the model addition operation is triggered for the first inference module, regardless of how the first model is obtained, if the first inference module is currently selected after obtaining the first model, then... Figure 2 The module settings area 202 of the process management interface can display the model information of the first model. Additionally, the module settings area 202 can also include a component for triggering model training. When the user triggers this component, the visual algorithm development platform can train the first model using sample images.

[0238] After training the second model and deploying it in the first inference module, it also supports updating the second model. For example, see... Figure 9 After deploying the second model in the first inference module, the module setting area 202 displays the model information of the second model (i.e., image segmentation model 3), and also displays a component for triggering model training (the "Annotation Training" component). When the user triggers this component, the visual algorithm development platform receives a model training instruction for the second model. Accordingly, the visual algorithm development platform updates the second model, obtaining a new version of the model. That is, the visual algorithm development platform can train the first model using new sample images. For example, the visual algorithm development platform can display the training settings interface as shown in subsequent embodiments, and use this interface to determine new sample images to train the first model using these new sample images, obtaining a new version of the model. Since the first model is being retrained, the new version of the model obtained in this case belongs to the same model as the original second model, but is a different version.

[0239] Understandably, regardless of whether the visual algorithm development platform imports the first model locally, obtains the first model from the cloud, or creates a new first model, when updating the second model, the visual algorithm development platform can combine the new sample images with the sample images used when the second model was trained last time to train the first model and obtain a new version of the model.

[0240] In one embodiment, step S102 includes the following steps: displaying a training settings interface; wherein the training settings interface displays: sample images and a label setting area; obtaining labels for annotating the sample images in the label setting area; and training a first model using the sample images and the obtained labels to obtain a second model. This facilitates user annotation of sample images, improves annotation efficiency, and further enhances the efficiency of model deployment.

[0241] The above-described training model embodiments can be combined with any of the above-described embodiments for obtaining the first model, adding samples, displaying the training status, and batch deployment of the model, as well as the embodiment for subsequently obtaining real-time acquired images as samples, to realize a model deployment scheme.

[0242] In one embodiment, see Figure 11 , Figure 11 The training settings interface shown has a thumbnail display area 1101 on the left, used to display thumbnails of sample images; a large image display area 1102 in the middle, used to display the currently selected sample image; and a label setting area 1103 on the right. Accordingly, users can operate within the image displayed in the large image display area 1102 to select image content and enter labels for the selected image content in the label setting area 1103, where labels are input. Figure 11 In the image, the diagonal rectangle represents both the thumbnail and the full-size image.

[0243] In one embodiment, the user can also set training parameters in the training settings interface, such as the learning rate and batch size.

[0244] In one embodiment, the method further includes: determining images from the real-time acquired images for training the first model as sample images. This eliminates the need for the user to save the images acquired in real-time by the image acquisition device and import them into the model for training, further improving the efficiency of model deployment.

[0245] The above-described embodiment of acquiring real-time images as sample images can be combined with any of the above-described embodiments of acquiring the first model, appending samples, displaying the training status, batch deploying the model, and training the model to achieve a model deployment scheme.

[0246] In scenarios where visual task flows are used to detect images acquired in real time, the visual algorithm development platform can communicate with the image acquisition device to obtain the images acquired in real time. Accordingly, the visual algorithm development platform can train a first model based on the acquired images.

[0247] The visual algorithm development platform provides one of the following methods for determining sample images:

[0248] Method 1: In response to receiving a single image addition instruction, the currently acquired single image is determined as the sample image.

[0249] Method 2: In response to receiving a batch image addition instruction, the collected multiple images are identified as sample images.

[0250] In this embodiment, a single image can be marked as a sample image using a single image addition command, while multiple images can be marked as sample images using a batch image addition command, thus meeting different user needs. For example, in response to receiving a batch image addition command, multiple consecutively acquired images are identified as sample images.

[0251] For example, see Figure 11 , Figure 11 The training settings interface, shown in the upper left corner, displays a "Single Run" component for triggering single image addition commands and a "Continuous Run" component for triggering batch image addition commands. In other words, the sample images used are determined through the training settings interface before training the model.

[0252] Alternatively, the process management interface can also include a separate component for setting sample images. When the user triggers this component, the visual algorithm development platform can display an interface for setting sample images, where the user can then select the sample image to use based on their actions.

[0253] See Figure 12 , Figure 12 The model deployment method shown includes the following steps:

[0254] S1201: Loading / Building the Visual Task Flow.

[0255] Users can operate on the visual algorithm development platform. Based on the user's operation, the visual algorithm development platform can load existing visual task processes or create new visual task processes.

[0256] S1202: For the first inference module in the vision task flow, if no model is deployed in the first inference module, in response to receiving the model import instruction, the model is obtained from the cloud or a model deployed in another inference module is obtained and deployed to the first inference module.

[0257] S1203: For the first inference module in the vision task flow, if no model is deployed in the first inference module, create a model in response to receiving a model creation instruction.

[0258] S1204: Determine the sample image from the images acquired in real time.

[0259] S1205: Label the sample images, train the model using the sample images, and deploy it to the first inference module.

[0260] S1206: Execute the visual task flow of the first inference module to detect the image to be detected and obtain the detection result.

[0261] S1207: In response to receiving a sample append instruction for a detected image, mark the detected image indicated by the sample append instruction as a sample image, and return to execute S1205.

[0262] S1208: In response to receiving the model training instruction, return to the execution step S1204.

[0263] That is, in response to receiving a model training instruction for the model deployed in the first inference module, the model is updated using new sample images. The model training instruction is triggered by the user when they determine that the current model does not meet the requirements.

[0264] Based on the same inventive concept, this application also provides a model deployment device applied to a visual algorithm development platform. The visual algorithm development platform includes a visual task module for building visual task schemes; the visual task module includes a first inference module that uses a model for inference. (See also...) Figure 13 The device includes:

[0265] The first model acquisition unit 1301 is used to acquire the first model in response to the add model operation;

[0266] Training unit 1302 is used to train the first model using sample images to obtain the second model;

[0267] Deployment unit 1303 is used to deploy the second model to the first inference module.

[0268] In an optional embodiment, the first model acquisition unit 1301 is specifically used to create a first model in response to receiving a model creation instruction;

[0269] or,

[0270] In response to receiving a model import instruction, the first model indicated by the model import instruction is obtained.

[0271] In an optional embodiment, the first model acquisition unit 1301 is specifically configured to acquire the first model from the local source in response to receiving a local model import instruction;

[0272] or,

[0273] In response to receiving a cloud model import command, the first model is retrieved from the cloud.

[0274] or,

[0275] In response to receiving a shared model import instruction, the first model is retrieved from the models already deployed in the vision task pipeline.

[0276] In an optional embodiment, the first model acquisition unit 1301 is specifically configured to acquire the first model in the following manner:

[0277] Models are categorized and displayed according to their type.

[0278] In response to receiving the first selection instruction, the first model is retrieved from the displayed models.

[0279] In an optional embodiment, the first model acquisition unit 1301 is specifically configured to acquire the first model in the following manner:

[0280] Displays a model that is compatible with the functionality of the current module;

[0281] In response to receiving a second selection instruction, the first model is retrieved from the displayed models.

[0282] In an optional embodiment, the first model acquisition unit 1301 is specifically used to display the cloud login interface in response to receiving a cloud model import instruction;

[0283] After successful login, the model in the cloud will be displayed;

[0284] In response to receiving a third selection instruction, the first model is retrieved from the displayed models.

[0285] In an optional embodiment, the first model acquisition unit 1301 is specifically configured to display the models deployed in the visual task process in response to receiving a shared model import instruction;

[0286] In response to receiving the fourth selection instruction, the first model is retrieved from the displayed models.

[0287] In an optional embodiment, the apparatus further includes:

[0288] An appending unit is configured to, in response to receiving a sample appending instruction, append the image indicated by the sample appending instruction as a sample image before training the first model using the sample images to obtain the second model.

[0289] In an optional embodiment, the training unit 1302 is specifically configured to, in response to receiving the sample append instruction, append the image indicated by the sample append instruction as a sample image, and then train the first model using the sample image to obtain the second model.

[0290] or,

[0291] When the number of additional sample images reaches a preset number, the first model is trained using the sample images to obtain the second model.

[0292] or,

[0293] In response to receiving a model training instruction, the first model is trained using sample images to obtain a second model.

[0294] In an optional embodiment, the apparatus further includes:

[0295] The detection unit is configured to deploy the first model to the first inference module before appending the image indicated by the sample appending instruction to the sample image in response to receiving the sample appending instruction;

[0296] The first inference module is used to detect the image to be detected, and the detection result is obtained.

[0297] The appending unit is specifically configured to append the detected image indicated by the sample appending instruction to the sample image in response to receiving the sample appending instruction; wherein the sample appending instruction is triggered based on the detection result.

[0298] In an optional embodiment, the apparatus further includes:

[0299] The training status display unit is used to display the training status of the model.

[0300] In an optional embodiment, the training state includes at least one of the following:

[0301] The training progress of the model;

[0302] This indicates the training result, whether the training was successful or unsuccessful.

[0303] Reasons for training failure.

[0304] In an optional embodiment, the apparatus further includes:

[0305] A batch deployment unit is used to batch deploy the second model to at least one second inference module included in the vision task scheme.

[0306] In an optional embodiment, the detection type of the second inference module is consistent with the detection type of the first inference module.

[0307] In an optional embodiment, the apparatus further includes:

[0308] An inference module determination unit is used to display, before deploying the second model in batches to at least one second inference module included in the vision task scheme, an inference module whose detection type is consistent with that of the first inference module.

[0309] In response to the fourth selection instruction, a second inference module is determined from the displayed inference modules.

[0310] In an optional embodiment, the apparatus further includes:

[0311] The sample image determination unit is used to determine, as sample images, images from real-time acquired images before training the first model using sample images to obtain the second model.

[0312] In an optional embodiment, the sample image determination unit is specifically configured to determine the currently acquired single image as a sample image in response to receiving a single image addition instruction;

[0313] or,

[0314] In response to receiving a batch image addition instruction, the acquired multiple images are identified as sample images.

[0315] In an optional embodiment, the training unit 1302 is specifically used to display a training settings interface; wherein the training settings interface displays: sample images and a label setting area;

[0316] Obtain the labels that are used to annotate the sample image in the label setting area;

[0317] The first model is trained using sample images and acquired labels to obtain the second model.

[0318] In an optional embodiment, the first model and the second model are models with adjustable parameters.

[0319] This application also provides an electronic device, such as... Figure 14 As shown, it includes:

[0320] Memory 1401 is used to store computer programs;

[0321] When processor 1402 executes a program stored in memory 1401, it performs the following steps:

[0322] In response to the add model operation, retrieve the first model;

[0323] The first model is trained using sample images to obtain the second model;

[0324] The second model is deployed to the first inference module.

[0325] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1402, the communication interface, and the memory 1401 communicating with each other via the communication bus.

[0326] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0327] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0328] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0329] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0330] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described model deployment methods.

[0331] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the model deployment methods described above.

[0332] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0333] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0334] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0335] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A model deployment method, characterized in that, It is applied to a visual algorithm development platform, which includes a visual task module for building visual task solutions; The visual task module includes a first reasoning module that uses a model for reasoning; the method includes: In response to the add model operation, retrieve the first model; The first model is trained using sample images to obtain the second model; The second model is deployed to the first inference module.

2. The method according to claim 1, characterized in that, The step of obtaining the first model in response to the add model operation includes: In response to receiving a model creation instruction, create the first model; or, In response to receiving a model import instruction, the first model indicated by the model import instruction is obtained.

3. The method according to claim 2, characterized in that, The step of responding to receiving a model import instruction and obtaining the first model indicated by the model import instruction includes: In response to receiving a local model import command, retrieve the first model from the local machine; or, In response to receiving a cloud model import command, the first model is retrieved from the cloud. or, In response to receiving a shared model import instruction, the first model is retrieved from the models already deployed in the vision task pipeline.

4. The method according to claim 3, characterized in that, The first model is obtained through the following methods: Models are categorized and displayed according to their type. In response to receiving the first selection instruction, the first model is retrieved from the displayed models.

5. The method according to claim 3, characterized in that, The first model is obtained through the following methods: Displays a model that is compatible with the functionality of the current module; In response to receiving a second selection instruction, the first model is retrieved from the displayed models.

6. The method according to claim 3, characterized in that, The response to receiving a cloud model import instruction and obtaining the first model from the cloud includes: Upon receiving a cloud model import command, the cloud login interface is displayed; After successful login, the model in the cloud will be displayed; In response to receiving a third selection instruction, the first model is retrieved from the displayed models.

7. The method according to claim 3, characterized in that, The step of receiving a shared model import instruction and obtaining a first model from the models already deployed in the vision task workflow includes: In response to receiving a shared model import instruction, the model already deployed in the vision task workflow is displayed; In response to receiving the fourth selection instruction, the first model is retrieved from the displayed models.

8. The method according to any one of claims 1 to 7, characterized in that, Before training the first model using sample images to obtain the second model, the method further includes: In response to receiving a sample append instruction, the image indicated by the sample append instruction is appended as a sample image.

9. The method according to claim 8, characterized in that, The step of training the first model using sample images to obtain the second model includes: In response to receiving the sample append instruction, after appending the image indicated by the sample append instruction as a sample image, the first model is trained using the sample image to obtain the second model; or, When the number of additional sample images reaches a preset number, the first model is trained using the sample images to obtain the second model. or, In response to receiving a model training instruction, the first model is trained using sample images to obtain a second model.

10. The method according to claim 8, characterized in that, Before appending the image indicated by the sample append instruction to the sample image in response to receiving the sample append instruction, the method further includes: Deploy the first model to the first inference module; The first inference module is used to detect the image to be detected, and the detection result is obtained. The step of responding to receiving a sample append instruction by appending the image indicated by the sample append instruction as a sample image includes: In response to receiving a sample append instruction, the detected image indicated by the sample append instruction is appended as a sample image; wherein the sample append instruction is triggered based on the detection result.

11. The method according to any one of claims 1 to 7, characterized in that, The method further includes: This displays the training status of the model.

12. The method according to claim 11, characterized in that, The training state includes at least one of the following: Model training progress; This indicates the training result, whether the training was successful or unsuccessful. Reasons for training failure.

13. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The second model is deployed in batches to at least one second inference module included in the vision task scheme.

14. The method according to claim 13, characterized in that, The detection type of the second inference module is the same as that of the first inference module.

15. The method according to claim 14, characterized in that, Before deploying the second model in batches to at least one second inference module included in the vision task scheme, the method further includes: Display the inference module in the visual task scheme that has the same detection type as the first inference module; In response to the fourth selection instruction, a second inference module is determined from the displayed inference modules.

16. The method according to any one of claims 1 to 7, characterized in that, Before training the first model using sample images to obtain the second model, the method further includes: Images used to train the first model are determined from the real-time acquired images and used as sample images.

17. The method according to claim 16, characterized in that, The step of determining the images used to train the first model from the real-time acquired images, as sample images, includes: In response to receiving a single image addition instruction, the currently acquired single image is determined as a sample image; or, In response to receiving a batch image addition instruction, the acquired multiple images are identified as sample images.

18. The method according to any one of claims 1 to 7, characterized in that, The step of training the first model using sample images to obtain the second model includes: The training settings interface is displayed; wherein, the training settings interface displays: sample images and label setting areas; Obtain the labels that are used to annotate the sample image in the label setting area; The first model is trained using sample images and acquired labels to obtain the second model.

19. The method according to any one of claims 1 to 7, characterized in that, The first model and the second model are models with adjustable parameters.

20. A model deployment device, characterized in that, It is applied to a visual algorithm development platform, which includes a visual task module for building visual task solutions; The visual task module includes a first reasoning module that uses a model for reasoning; the device includes: The first model acquisition unit is used to acquire the first model in response to the add model operation; The training unit is used to train the first model using sample images to obtain the second model; The deployment unit is used to deploy the second model to the first inference module.

21. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-19.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-19.

Images