Processing method, device and system of image modeling platform

Abstract

The present disclosure provides a processing method, device and system of an image modeling platform, the method comprising: providing a workflow creation interface and at least one image processing unit in the image modeling platform; in response to an operation of creating a workflow, creating a workflow composed of a target image processing unit in the workflow creation interface; wherein the at least one image processing unit at least includes the target image processing unit; in response to a request of running the workflow, controlling the target image processing unit to process target image data according to corresponding configuration information.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning model technology, and more specifically, to a processing method for an image modeling platform, a processing apparatus for an image modeling platform, a system including at least one computing device and at least one storage device, and a computer-readable storage medium. Background Technology

[0002] With the development of artificial intelligence, more and more industry scenarios require the establishment of machine learning models, including but not limited to the application of statistical analysis methods, machine learning methods, deep learning methods, transfer learning methods, etc.

[0003] In actual modeling processes, there is often a need for secondary encapsulation of the algorithm results. The processing of image data often relies on senior modelers or developers to perform some logical encapsulation and custom processing, which leads to problems such as low modeling efficiency and high threshold.

[0004] Therefore, this disclosure proposes a technical solution for processing image data based on an image modeling platform. Summary of the Invention

[0005] One objective of this disclosure is to provide a new technical solution for processing image data based on an image modeling platform.

[0006] According to a first aspect of this disclosure, a processing method for an image modeling platform is provided, comprising:

[0007] The image modeling platform provides a workflow creation interface and at least one image processing unit;

[0008] In response to the workflow creation operation, a workflow consisting of target image processing units is created in the workflow creation interface; wherein the target image processing units are selected from the at least one image processing unit;

[0009] In response to a request to run the workflow, the target image processing unit is controlled to process the target image data according to the corresponding configuration information.

[0010] Optionally, in response to the workflow creation operation, creating a workflow consisting of target image processing units in the workflow creation interface includes:

[0011] In response to the operation of selecting a target image processing unit from the at least one image processing unit, the selected target image processing unit is displayed on the workflow creation interface;

[0012] In response to the operation of connecting the target image processing unit in the workflow creation interface, the target image processing unit is connected to obtain the workflow.

[0013] Optionally, the method further includes:

[0014] In response to the operation of configuring the target image processing unit, configuration items for the target image processing unit are provided;

[0015] Obtain the configuration information of the target image processing unit set by the user through the configuration items.

[0016] Optionally, the target image data includes the original image, and the target image processing unit includes a processing unit for preprocessing the image;

[0017] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0018] The original image is enhanced according to the configuration information to obtain the enhanced image.

[0019] Optionally, the configuration information includes at least one image enhancement method, which includes at least one or more of sharpening, Gaussian blur, rotation, and cropping.

[0020] Optionally, the target image data includes an original sample set composed of images; the target image processing unit includes a processing unit for splitting the sample set.

[0021] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0022] Based on the configuration information, the original sample set is split into a training sample set, a validation sample set, and a test sample set.

[0023] Optionally, the configuration information includes at least one or more of the following splitting methods: random splitting, sequential splitting, random splitting by tag hierarchy, sequential splitting by tag hierarchy, and splitting by custom rules.

[0024] Optionally, the target image data includes image samples for training the model, and the target image processing unit includes a processing unit for training the model based on the image samples;

[0025] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0026] The image samples are trained based on the configuration information to obtain the corresponding image processing model.

[0027] Optionally, the configuration information includes at least a model training algorithm, which includes at least one or more of image classification algorithms, object detection algorithms, instance segmentation algorithms, and text recognition algorithms.

[0028] Optionally, the target image data includes an image processing model, and the target image processing unit includes a processing unit for evaluating the model;

[0029] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0030] Obtain a test sample set that matches the image processing model;

[0031] The image processing model is evaluated based on the test sample set and the configuration information to obtain the evaluation result of the image processing model.

[0032] Optionally, the target image data includes a standard metafile describing the calling relationships between multiple image processing models; the target image processing unit includes a processing unit for orchestrating the image processing models.

[0033] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0034] The standard metafile is parsed to obtain the calling relationships between the multiple image processing models;

[0035] The multiple image processing models are arranged and processed according to the calling relationship.

[0036] Optionally, the target image data includes an image processing model, and the target image processing unit includes a processing unit for performing online processing on the model;

[0037] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0038] The image processing model is packaged according to the configuration information;

[0039] The packaged image processing model was released online.

[0040] According to a second aspect of this disclosure, a processing apparatus for an image modeling platform is provided, comprising:

[0041] The platform provides modules for providing a workflow creation interface and at least one image processing unit within the image modeling platform;

[0042] A workflow creation module is used to create a workflow consisting of target image processing units in the workflow creation interface in response to a workflow creation operation; wherein the at least one image processing unit includes at least the target image processing unit;

[0043] The workflow execution module is used to respond to the request to run the workflow and control the target image processing unit to process the target image data according to the corresponding configuration information.

[0044] Optionally, the workflow creation module is also used for:

[0045] In response to the operation of selecting a target image processing unit from the at least one image processing unit, the selected target image processing unit is displayed on the workflow creation interface;

[0046] In response to the operation of connecting the target image processing unit in the workflow creation interface, the target image processing unit is connected to obtain the workflow.

[0047] Optionally, the device further includes:

[0048] A configuration providing module is configured to provide configuration items for the target image processing unit in response to an operation of configuring the target image processing unit.

[0049] The configuration acquisition module is used to acquire the configuration information of the target image processing unit set by the user through the configuration items.

[0050] Optionally, the target image data includes the original image, and the target image processing unit includes a processing unit for preprocessing the image;

[0051] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0052] The original image is enhanced according to the configuration information to obtain the enhanced image.

[0053] Optionally, the configuration information includes at least one image enhancement method, which includes at least one or more of sharpening, Gaussian blur, rotation, and cropping.

[0054] Optionally, the target image data includes an original sample set composed of images; the target image processing unit includes a processing unit for splitting the sample set.

[0055] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0056] Based on the configuration information, the original sample set is split into a training sample set, a validation sample set, and a test sample set.

[0057] Optionally, the configuration information includes at least one or more of the following splitting methods: random splitting, sequential splitting, random splitting by tag hierarchy, sequential splitting by tag hierarchy, and splitting by custom rules.

[0058] Optionally, the target image data includes image samples for training the model, and the target image processing unit includes a processing unit for training the model based on the image samples;

[0059] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0060] The image samples are trained based on the configuration information to obtain the corresponding image processing model.

[0061] Optionally, the configuration information includes at least a model training algorithm, which includes at least one or more of image classification algorithms, object detection algorithms, instance segmentation algorithms, and text recognition algorithms.

[0062] Optionally, the target image data includes an image processing model, and the target image processing unit includes a processing unit for evaluating the model;

[0063] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0064] Obtain a test sample set that matches the image processing model;

[0065] The image processing model is evaluated based on the test sample set and the configuration information to obtain the evaluation result of the image processing model.

[0066] Optionally, the target image data includes a standard metafile describing the calling relationships between multiple image processing models; the target image processing unit includes a processing unit for orchestrating the image processing models.

[0067] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0068] The standard metafile is parsed to obtain the calling relationships between the multiple image processing models;

[0069] The multiple image processing models are arranged and processed according to the calling relationship.

[0070] Optionally, the target image data includes an image processing model, and the target image processing unit includes a processing unit for performing online processing on the model;

[0071] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0072] The image processing model is packaged according to the configuration information;

[0073] The packaged image processing model was released online.

[0074] According to a third aspect of this disclosure, a system is provided comprising at least one computing device and at least one storage device, wherein the at least one storage device is configured to store instructions for controlling the at least one computing device to perform the method according to a first aspect of this disclosure.

[0075] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect of this disclosure.

[0076] One beneficial effect of this disclosure is that by providing a workflow creation interface and at least one image processing unit in the image modeling platform, in response to the operation of creating a workflow, a workflow consisting of the target image processing unit is created in the workflow creation interface, and in response to the request to run the workflow, the target image processing unit is controlled to process the target image data according to the corresponding configuration information. In this way, modeling efficiency can be improved and the development threshold for modeling can be reduced.

[0077] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0078] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the present disclosure.

[0079] Figure 1 This is a block diagram illustrating an example of a hardware configuration of an electronic device that can be used to implement embodiments of the present disclosure.

[0080] Figure 2 This is a flowchart illustrating the processing method of an image modeling platform according to an embodiment of the present disclosure;

[0081] Figure 3This is a block diagram of a processing apparatus for an image modeling platform according to an embodiment of the present disclosure;

[0082] Figure 4 This is a block diagram of a system according to an embodiment of the present disclosure. Detailed Implementation

[0083] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0084] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0085] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0086] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0087] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0088] Hereinafter, various embodiments and examples according to the present disclosure will be described with reference to the accompanying drawings.

[0089] <Hardware Configuration>

[0090] Figure 1 This is a block diagram illustrating the hardware configuration of an electronic device 1000 that can implement embodiments of the present disclosure.

[0091] Electronic device 1000 can be a portable computer, desktop computer, mobile phone, tablet computer, etc. For example... Figure 1As shown, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, etc. The processor 1100 may be a central processing unit (CPU), a microprocessor (MCU), etc. The memory 1200 may include, for example, ROM (Read-Only Memory), RAM (Random Access Memory), or non-volatile memory such as a hard disk. The interface device 1300 may include, for example, a USB interface, a headphone jack, etc. The communication device 1400 may be capable of wired or wireless communication, specifically including Wi-Fi communication, Bluetooth communication, 2G / 3G / 4G / 5G communication, etc. The display device 1500 may be, for example, an LCD screen, a touch screen, etc. The input device 1600 may include, for example, a touch screen, a keyboard, motion input, etc. Users can input / output voice information through the speaker 1700 and the microphone 1800.

[0092] Figure 1 The electronic devices shown are merely illustrative and in no way intended to limit this disclosure, its application, or use. In embodiments applied to this disclosure, the memory 1200 of the electronic device 1000 is used to store instructions for controlling the processor 1100 to operate and execute the processing method of any of the image modeling platforms provided in embodiments of this disclosure. Those skilled in the art will understand that, although... Figure 1 The electronic device 1000 is shown with multiple devices shown; however, this disclosure may relate only to some of these devices. For example, electronic device 1000 may only relate to processor 1100 and storage device 1200. Those skilled in the art can design instructions based on the schemes disclosed herein. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0093] <Method Implementation>

[0094] In this embodiment, a processing method for an image modeling platform is provided. This image modeling platform processing method can be implemented by an electronic device. The electronic device can be, for example,... Figure 1 The electronic device 1000 shown. At least one image processing unit may be provided in advance in this image modeling platform.

[0095] according to Figure 2 As shown, the image modeling platform processing method of this embodiment may include the following steps S2100 to S2300:

[0096] Step S2100: Provide a workflow creation interface and at least one image processing unit in the image modeling platform.

[0097] In an image modeling platform, at least one image processing unit can be provided in advance. The image processing unit can be a standardized encapsulation and definition of common algorithms and data processing logic in image scenes.

[0098] In one example, the image modeling platform may pre-provide at least one of the following processing units: a processing unit for preprocessing images, a processing unit for splitting the sample set, a processing unit for training the model based on the image samples, a processing unit for evaluating the model, a processing unit for orchestrating the image processing model, and a processing unit for deploying the model.

[0099] The workflow creation interface in this embodiment can be an interface used to create workflows based on at least one image processing unit provided by the image modeling platform.

[0100] In one embodiment of this disclosure, the image modeling platform may also provide a list of processing units, which displays the icons, names and / or functions of all image processing units that the image modeling platform can provide, for the user to select.

[0101] In step S2200, in response to the workflow creation operation, a workflow consisting of the target image processing unit is created in the workflow creation interface.

[0102] The image modeling platform provides at least one image processing unit, which includes at least one target image processing unit, and the target image processing unit is selected from the at least one image processing unit.

[0103] In one embodiment of this disclosure, the number of target image processing units may be one. Then, in response to a workflow creation operation, a workflow consisting of the target image processing units is created in the workflow creation interface; in response to an operation of selecting a target processing unit from at least one image processing unit, the selected target processing unit is displayed in the workflow creation interface, thus obtaining the workflow.

[0104] In this embodiment, the user can select one of at least one image processing unit provided by the image modeling platform as the target image processing unit according to their actual needs. The target image processing unit selected by the user can be displayed in the workflow creation interface.

[0105] The workflow creation interface may also include a create button. After selecting the target image processing unit, the user can click the create button to trigger the image modeling platform to display the target image processing unit in the workflow creation interface as a workflow.

[0106] In another embodiment of this disclosure, the number of target image processing units can be multiple. Therefore, in response to the workflow creation operation, creating a workflow composed of target image processing units in the workflow creation interface can include the following steps S2210 to S2220:

[0107] In step S2210, in response to the operation of selecting a target processing unit from at least one image processing unit, the selected target processing unit is displayed in the workflow creation interface.

[0108] In this embodiment, the user can select multiple image processing units from at least one image processing unit provided by the image modeling platform as target image processing units according to their actual needs. The multiple target image processing units selected by the user can be displayed in the workflow creation interface. Specifically, the user can select one or more image processing units at a time.

[0109] Step S2220: In response to the operation of connecting the target image processing unit in the workflow creation interface, the target image processing unit is connected to obtain the workflow.

[0110] In this embodiment, the user can pre-connect multiple target image processing units sequentially according to their processing order, so that the connection order of the multiple target image units is the same as their processing order.

[0111] In one embodiment of this disclosure, each target image processing unit may have an upstream connection point and a downstream connection point, used to represent data input and output, respectively. A user can connect the first image processing unit and the second image processing unit by clicking the downstream connection point of the first target image processing unit and the upstream connection point of the second image processing unit, so that the first image processing unit processes data before the second image processing unit, i.e., the second image processing unit processes the image data output by the first image processing unit.

[0112] In another embodiment of this disclosure, it may also involve identifying the image processing unit associated with the latest image processing unit among the other image processing units displayed in the workflow creation interface, based on the latest image processing unit recently selected by the user and displayed in the workflow creation interface, and automatically connecting the two.

[0113] Furthermore, it can be an image processing unit whose output data type is the same as the input data type of the latest image processing unit, which can be determined from other image processing units displayed in the workflow creation interface, or an image processing unit whose input data type is the same as the output data type of the latest image processing unit, which can be used as the image processing unit associated with the latest image processing unit.

[0114] The workflow creation interface may also include a create button. After selecting the target image processing unit, the user can click the create button to trigger the image modeling platform to display multiple connected target image processing units in the workflow creation interface, thus obtaining the workflow.

[0115] In other words, a workflow can include multiple connected target image processing units.

[0116] In step S2300, in response to the request to run the workflow, the target image processing unit is controlled to process the target image data according to the corresponding configuration information.

[0117] In one embodiment of this disclosure, the image modeling platform may provide a run button, which the user clicks to trigger a request to run the workflow created in the workflow creation interface.

[0118] The configuration information in this embodiment can be set by the developer when creating the target image processing unit, or it can be set by the user in advance according to their actual needs for each target image processing unit.

[0119] Specifically, the configuration information may include at least one of the following: GPU resources used, CPU resources used, memory resources used, the address of the image called at runtime, the output field of the target image processing unit, and the target value field corresponding to the output field.

[0120] In one embodiment of this disclosure, the method may further include: providing configuration items for the target image processing unit in response to an operation of configuring the target image processing unit; and obtaining configuration information of the target image processing unit set by the user through the configuration items.

[0121] In this embodiment, the user can trigger the image modeling platform to provide a configuration button for any target image processing unit displayed in the workflow creation interface by selecting that unit. The user then clicks the configuration button to configure the target image processing unit.

[0122] Furthermore, the image modeling platform can also provide a confirmation button. After the user has set the configuration information of the target image processing unit through the configuration items, they can click the confirmation button to trigger the image modeling platform to obtain the configuration information set by the user.

[0123] When there is only one target image processing unit, it can be controlled to process the target image data according to the corresponding configuration information.

[0124] In a first embodiment of this disclosure, the target image data may include an original image, and the target image processing unit may include a processing unit for preprocessing the image.

[0125] The original image may be data in a graphic file format that is pre-stored in the electronic device executing this embodiment. The graphic file format may include any one of the following: PNG, CDR, AI, DXF, EPS, BMP, TIFF, JPEG, GIF, PSD, and PDF.

[0126] Therefore, controlling the target image processing unit to process the target image data according to the corresponding configuration information may include: enhancing the original image according to the configuration information to obtain the enhanced image.

[0127] In this embodiment, the target image processing unit used for image preprocessing can perform various preprocessing logics on the original image through data augmentation operators or custom Python scripts using the OpenCV open-source library (image data processing library) to obtain the enhanced image.

[0128] In scenarios where the amount of data in the original image is small, the number of images used to construct the data sample set is limited. The target image processing unit used in this embodiment for image preprocessing can generate some enhanced images based on the original image. A new data sample set can be constructed together based on the original image and the enhanced image, thereby expanding the data sample set.

[0129] Furthermore, the configuration information of the target image processing unit used for image preprocessing may at least include image enhancement methods. These image enhancement methods include at least one or more of sharpening, Gaussian blur, rotation, and cropping. Users can select the image enhancement method for the target image processing unit from the image enhancement methods provided by the image modeling platform, based on their own needs.

[0130] In this embodiment, the target image processing unit used to preprocess the image can enhance the original image according to the image enhancement method set by the user to obtain the enhanced image.

[0131] In a second embodiment of this disclosure, the target image data may include an original sample set composed of images, and the target image processing unit includes a processing unit for splitting the sample set.

[0132] In one example, the images in the original sample set may include the original images and / or the enhanced images obtained from the target image processing unit used to preprocess the images.

[0133] Therefore, controlling the target image processing unit to process the target image data according to the corresponding configuration information may include: splitting the original sample set into a training sample set, a validation sample set, and a test sample set according to the configuration information.

[0134] Typically, before training a model, the original sample set needs to be split into a training sample set, a validation sample set, and a test sample set. The training sample set is used to train the model, the validation sample set is used to verify the model's performance, and the test sample set is used to test the model's generalization ability.

[0135] In this embodiment, the target image processing unit used to split the sample set can achieve the effect of splitting the sample set by splitting the original sample set into a training sample set, a validation sample set, and a test sample set.

[0136] Furthermore, the configuration information of the target image processing unit used to split the sample set includes at least the splitting method. The splitting method includes at least one or more of the following: random splitting, sequential splitting, random splitting by label hierarchy, sequential splitting by label hierarchy, and splitting by custom rules. Users can select the splitting method for the target image processing unit used to split the sample set from the splitting methods provided by the image modeling platform according to their own needs.

[0137] In this embodiment, the target image processing unit used to split the sample set can split the original sample set into a training sample set, a validation sample set, and a test sample set according to the splitting method set by the user.

[0138] In a third embodiment of this disclosure, the target image data may include image samples for training the model, and the target image processing unit includes a processing unit for training the model based on the image samples.

[0139] In one example, the image samples used to train the model can be image samples from the training sample set and / or image samples from the validation sample set obtained by the target image processing unit used to split the sample set.

[0140] Therefore, controlling the target image processing unit to process the target image data according to the corresponding configuration information may include: training the image samples according to the configuration information to obtain the corresponding image processing model.

[0141] Furthermore, the configuration information of the target image processing unit used for model training based on image samples includes at least a model training algorithm. This model training algorithm includes at least one or more of image classification algorithms, object detection algorithms, instance segmentation algorithms, and character recognition algorithms. Users can select the model training algorithm used by the target image processing unit for model training based on image samples from the model training algorithms provided by the image modeling platform, according to their own needs.

[0142] In this embodiment, the target image processing unit used to split the sample set can be trained on the image samples according to the model training algorithm set by the user to obtain the corresponding image processing model.

[0143] The model training algorithm can accept image samples from the training sample set to train the model, and it can also accept image samples from the validation sample set to perform incremental learning and debugging optimization on the already trained model.

[0144] Furthermore, the image modeling platform can train the image processing model based on the parameter values ​​of the set parameters to obtain the model results learned under those set parameters. These set parameters may include hyperparameters and / or required hardware resources (including at least one of the following: GPU resources, CPU resources, and memory resources).

[0145] In a fourth embodiment of this disclosure, the target image data may include an image processing model, and the target image processing unit includes a processing unit for evaluating the model.

[0146] In one example, the image processing model can be a model obtained by using a target image processing unit for training the model based on image samples.

[0147] Therefore, controlling the target image processing unit to process the target image data according to the corresponding configuration information may include: obtaining a test sample set that matches the image processing model; evaluating the image processing model based on the test sample set and the configuration information, and obtaining the evaluation result of the image processing model.

[0148] In one example, the test sample set that matches the image processing model can be the test sample set obtained by the target image processing unit used to split the sample set.

[0149] In this embodiment, the target image processing unit used to evaluate the model can realize the prediction function of the image processing model through a test sample set, and evaluate and score the prediction results.

[0150] Furthermore, the configuration information of the target image processing unit used to evaluate the model may also include whether the evaluation results are visualized.

[0151] Therefore, if the configuration information is set to visualize the evaluation results, the target image processing unit used to evaluate the model can display the prediction results of the image processing model on the image samples in the test sample set, including the correct prediction results and the incorrect prediction results, and can also display the difference between the prediction results and the actual results.

[0152] In a fifth embodiment of this disclosure, the target image data may include a standard metafile for describing the calling relationships between multiple image processing models, and the target image processing unit includes a processing unit for orchestrating the image processing models.

[0153] Therefore, controlling the target image processing unit to process the target image data according to the corresponding configuration information may include: parsing the standard metafile to obtain the calling relationship between multiple image processing models; and arranging and processing the multiple image processing models according to the calling relationship.

[0154] The standard metafile in this embodiment can be used to describe service scheduling information between multiple image processing models.

[0155] The target image processing unit, used for orchestrating image processing models, can process the calling relationships of multiple image processing models in the entire application by parsing standard metafiles, and orchestrate multiple image processing models according to the calling relationships to obtain a complete application component.

[0156] In one example, the image processing model can be a model obtained by using a target image processing unit for training the model based on image samples.

[0157] In a sixth embodiment of this disclosure, the target image data may include an image processing model, and the target image processing unit includes a processing unit for performing online processing on the model.

[0158] Therefore, controlling the target image processing unit to process the target image data according to the corresponding configuration information may include: packaging the image processing model according to the configuration information; and publishing the packaged image processing model online.

[0159] In one example, the image processing model can be a model obtained by using a target image processing unit for training the model based on image samples.

[0160] Furthermore, the configuration information of the target image processing unit used to process the model online may include the release time and / or release address, etc.

[0161] In this embodiment, the target image processing unit used for online processing of the model can be the image processing model trained by the target image processing unit used for model training based on image samples, which is then launched and published in the application center so that users can directly use the image processing in the future.

[0162] When there are multiple target image processing units, multiple target image processing units can be controlled to process the target image data according to the connection order and the corresponding configuration information.

[0163] In this embodiment, the first target image processing unit in the workflow can process target image data, and subsequent target image processing units can process the image data output by the previous target image processing unit in the workflow. The way each target image processing unit processes its own image data can be referred to the foregoing embodiments, and will not be repeated here.

[0164] The method in this embodiment provides a workflow creation interface and at least one image processing unit in the image modeling platform. In response to the workflow creation operation, a workflow consisting of the target image processing unit is created in the workflow creation interface. Then, in response to the request to run the workflow, the target image processing unit is controlled to process the target image data according to the corresponding configuration information. In this way, the modeling efficiency can be improved and the development threshold of modeling can be reduced.

[0165] The image modeling platform of this embodiment can support the processing logic of each image processing unit. By standardizing and encapsulating the algorithm and data processing logic to a certain extent, the image processing unit is obtained and adjustable configuration information is exposed. This enables the image modeling platform to meet the modeling needs in large-scale modeling scenarios. In addition, it can also support the development and evaluation process in the image modeling process.

[0166] <Device Embodiment>

[0167] In this embodiment, a processing device 3000 for an image modeling platform is provided, such as... Figure 3As shown, the system includes a platform providing module 3100, a workflow creation module 3200, and a workflow execution module 3300. The platform providing module 3100 provides a workflow creation interface and at least one image processing unit within the image modeling platform. The workflow creation module 3200, in response to a workflow creation operation, creates a workflow composed of target image processing units within the workflow creation interface; wherein the target image processing units are selected from the at least one image processing unit. The workflow execution module 3300, in response to a request to run the workflow, controls the target image processing units to process target image data according to corresponding configuration information.

[0168] In one embodiment of this disclosure, the workflow creation module 3200 can also be used for:

[0169] In response to the operation of selecting a target image processing unit from the at least one image processing unit, the selected target image processing unit is displayed on the workflow creation interface;

[0170] In response to the operation of connecting the target image processing unit in the workflow creation interface, the target image processing unit is connected to obtain the workflow.

[0171] In one embodiment of this disclosure, the apparatus may further include:

[0172] A configuration providing module is configured to provide configuration items for the target image processing unit in response to an operation of configuring the target image processing unit.

[0173] The configuration acquisition module is used to acquire the configuration information of the target image processing unit set by the user through the configuration items.

[0174] In one embodiment of this disclosure, the target image data includes an original image, and the target image processing unit includes a processing unit for preprocessing the image;

[0175] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0176] The original image is enhanced according to the configuration information to obtain the enhanced image.

[0177] In one embodiment of this disclosure, the configuration information includes at least one or more image enhancement methods, which include sharpening, Gaussian blur, rotation, and cropping.

[0178] In one embodiment of this disclosure, the target image data includes an original sample set composed of images; the target image processing unit includes a processing unit for splitting the sample set.

[0179] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0180] Based on the configuration information, the original sample set is split into a training sample set, a validation sample set, and a test sample set.

[0181] In one embodiment of this disclosure, the configuration information includes at least a splitting method, which includes at least one or more of random splitting, sequential splitting, random splitting by tag hierarchy, sequential splitting by tag hierarchy, and splitting by custom rules.

[0182] In one embodiment of this disclosure, the target image data includes image samples for training the model, and the target image processing unit includes a processing unit for training the model based on the image samples;

[0183] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0184] The image samples are trained based on the configuration information to obtain the corresponding image processing model.

[0185] In one embodiment of this disclosure, the configuration information includes at least a model training algorithm, which includes at least one or more of image classification algorithms, object detection algorithms, instance segmentation algorithms, and text recognition algorithms.

[0186] In one embodiment of this disclosure, the target image data includes an image processing model, and the target image processing unit includes a processing unit for evaluating the model;

[0187] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0188] Obtain a test sample set that matches the image processing model;

[0189] The image processing model is evaluated based on the test sample set and the configuration information to obtain the evaluation result of the image processing model.

[0190] In one embodiment of this disclosure, the target image data includes a standard metafile for describing the calling relationships between multiple image processing models; the target image processing unit includes a processing unit for orchestrating the image processing models.

[0191] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0192] The standard metafile is parsed to obtain the calling relationships between the multiple image processing models;

[0193] The multiple image processing models are arranged and processed according to the calling relationship.

[0194] In one embodiment of this disclosure, the target image data includes an image processing model, and the target image processing unit includes a processing unit for performing online processing on the model;

[0195] The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes:

[0196] The image processing model is packaged according to the configuration information;

[0197] The packaged image processing model was released online.

[0198] Those skilled in the art will understand that the image modeling platform processing device 3000 can be implemented in various ways. For example, the image modeling platform processing device 3000 can be implemented by configuring the processor with instructions. For example, the instructions can be stored in ROM, and when the device is started, the instructions can be read from ROM into a programmable device to implement the image modeling platform processing device 3000. For example, the image modeling platform processing device 3000 can be embedded in a dedicated device (e.g., an ASIC). The image modeling platform processing device 3000 can be divided into independent units, or they can be combined together. The image modeling platform processing device 3000 can be implemented by one of the various implementation methods described above, or by a combination of two or more of the various implementation methods described above.

[0199] In this embodiment, the image modeling platform processing device 3000 can have various implementation forms. For example, the image modeling platform processing device 3000 can be any functional module running in a software product or application that provides image modeling platform processing services, or peripheral embeddings, plug-ins, patches, etc. of these software products or applications, or it can be the software products or applications themselves.

[0200] <System Implementation Example>

[0201] In this embodiment, as Figure 4As shown, a system 4000 is also provided, comprising at least one computing device 4100 and at least one storage device 4200. The at least one storage device 4200 is used to store executable instructions for controlling the at least one computing device 4100 to execute a processing method of an image modeling platform according to any embodiment of the present disclosure.

[0202] In this embodiment, the system 4000 can be a mobile phone, tablet computer, handheld computer, desktop computer, laptop computer, workstation, game console, or a distributed system composed of multiple devices.

[0203] Computer-readable storage media

[0204] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the processing method of the image modeling platform as described in any embodiment of this disclosure.

[0205] This disclosure may be an apparatus, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0206] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0207] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0208] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0209] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0210] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0211] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0212] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0213] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A processing method for an image modeling platform, comprising: The image modeling platform provides a workflow creation interface and at least one image processing unit; The image processing unit is a standardized encapsulation and definition of common algorithms and data processing logic in image scenes; In response to the workflow creation operation, a workflow consisting of target image processing units is created in the workflow creation interface; wherein, the at least one image processing unit includes at least the target image processing unit; In response to a request to run the workflow, the target image processing unit is controlled to process the target image data according to the corresponding configuration information; The target image data includes an original sample set composed of images; the target image processing unit includes a processing unit for splitting the sample set, a processing unit for training the model based on the image samples, and a processing unit for evaluating the model. The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: According to the configuration information, the original sample set is split into a training sample set, a validation sample set, and a test sample set; the configuration information includes at least a splitting method and a model training algorithm, the splitting method includes at least one or more of random splitting, sequential splitting, random splitting by label hierarchy, sequential splitting by label hierarchy, and splitting by custom rules; the model training algorithm includes at least one or more of image classification algorithms, object detection algorithms, instance segmentation algorithms, and text recognition algorithms. The image processing model is trained on the image samples in the training sample set and / or validation sample set according to the configuration information to obtain the corresponding image processing model; The image processing model is evaluated based on the test sample set and the configuration information to obtain the evaluation result of the image processing model.

2. The method according to claim 1, wherein creating a workflow consisting of target image processing units in the workflow creation interface in response to the workflow creation operation comprises: In response to the operation of selecting a target image processing unit from the at least one image processing unit, the selected target image processing unit is displayed on the workflow creation interface; In response to the operation of connecting the target image processing unit in the workflow creation interface, the target image processing unit is connected to obtain the workflow.

3. The method according to claim 1, further comprising: In response to the operation of configuring the target image processing unit, configuration items for the target image processing unit are provided; Obtain the configuration information of the target image processing unit set by the user through the configuration items.

4. The method according to claim 1, wherein the target image data includes an original image, and the target image processing unit includes a processing unit for preprocessing the image; The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: The original image is enhanced according to the configuration information to obtain the enhanced image.

5. The method according to claim 4, wherein the configuration information includes at least one or more image enhancement methods, including at least sharpening, Gaussian blur, rotation, and cropping.

6. The method according to claim 1, wherein the target image data includes a standard metafile for describing the calling relationships between multiple image processing models; and the target image processing unit includes a processing unit for arranging and processing the image processing models. The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: The standard metafile is parsed to obtain the calling relationships between the multiple image processing models; The multiple image processing models are arranged and processed according to the calling relationship.

7. The method according to claim 1, wherein the target image data includes an image processing model, and the target image processing unit includes a processing unit for performing online processing on the model; The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: The image processing model is packaged according to the configuration information; The packaged image processing model was released online.

8. A processing apparatus for an image modeling platform, comprising: The platform provides modules for providing a workflow creation interface and at least one image processing unit within the image modeling platform; The image processing unit is a standardized encapsulation and definition of common algorithms and data processing logic in image scenes; A workflow creation module is used to create a workflow consisting of target image processing units in the workflow creation interface in response to a workflow creation operation; wherein the at least one image processing unit includes at least the target image processing unit; The workflow execution module is used to respond to the request to run the workflow and control the target image processing unit to process the target image data according to the corresponding configuration information; The target image data includes an original sample set composed of images; the target image processing unit includes a processing unit for splitting the sample set, a processing unit for training the model based on the image samples, and a processing unit for evaluating the model. The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: According to the configuration information, the original sample set is split into a training sample set, a validation sample set, and a test sample set; the configuration information includes at least a splitting method and a model training algorithm, the splitting method includes at least one or more of random splitting, sequential splitting, random splitting by label hierarchy, sequential splitting by label hierarchy, and splitting by custom rules; the model training algorithm includes at least one or more of image classification algorithms, object detection algorithms, instance segmentation algorithms, and text recognition algorithms. The image processing model is trained on the image samples in the training sample set and / or validation sample set according to the configuration information to obtain the corresponding image processing model; The image processing model is evaluated based on the test sample set and the configuration information to obtain the evaluation result of the image processing model.

9. The apparatus according to claim 8, wherein the workflow creation module is further configured to: In response to the operation of selecting a target image processing unit from the at least one image processing unit, the selected target image processing unit is displayed on the workflow creation interface; In response to the operation of connecting the target image processing unit in the workflow creation interface, the target image processing unit is connected to obtain the workflow.

10. The apparatus of claim 8, further comprising: A configuration providing module is configured to provide configuration items for the target image processing unit in response to an operation of configuring the target image processing unit. The configuration acquisition module is used to acquire the configuration information of the target image processing unit set by the user through the configuration items.

11. The apparatus according to claim 8, wherein the target image data includes an original image, and the target image processing unit includes a processing unit for preprocessing the image; The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: The original image is enhanced according to the configuration information to obtain the enhanced image.

12. The apparatus of claim 11, wherein the configuration information includes at least an image enhancement method, the image enhancement method including at least one or more of sharpening, Gaussian blur, rotation, and cropping.

13. The apparatus according to claim 8, wherein the target image data includes a standard metafile for describing the calling relationships between multiple image processing models; and the target image processing unit includes a processing unit for arranging and processing the image processing models. The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: The standard metafile is parsed to obtain the calling relationships between the multiple image processing models; The multiple image processing models are arranged and processed according to the calling relationship.

14. The apparatus according to claim 8, wherein the target image data includes an image processing model, and the target image processing unit includes a processing unit for performing online processing on the model; The process of controlling the target image processing unit to process the target image data according to the corresponding configuration information includes: The image processing model is packaged according to the configuration information; The packaged image processing model was released online.

15. An integrated management system for artificial intelligence applications, comprising at least one computing device and at least one storage device, wherein, The at least one storage device is used to store instructions for controlling the at least one computing device to perform the method according to any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method as described in any one of claims 1 to 7 when executed by a processor.

Images