Algorithm operation method, device, equipment, and storage medium

Abstract

The embodiments of the present disclosure provide an algorithm execution method, apparatus, device, and storage medium. The algorithm execution method includes: obtaining grouping information for multiple target algorithms; executing the multiple target algorithms on multiple data processing devices based on the grouping information; wherein target algorithms corresponding to the same grouping information are executed on the same data processing device. The technical solution provided by the present disclosure overcomes the problem of high resource usage during algorithm execution and deployment.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, the field of artificial intelligence technology, and in particular to an algorithm operation method, apparatus, device, and storage medium. Background Art

[0002] In recent years, with the development of the field of artificial intelligence, more and more AI (Artificial Intelligence) algorithms have been developed and applied to various industries. For example, computer vision based on deep learning has been widely used in various fields. Summary of the Invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] In a first aspect, an embodiment of the present disclosure provides an algorithm operation method, comprising:

[0005] Obtain grouping information of multiple target algorithms;

[0006] Multiple groups of target algorithms are run on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein the target algorithms corresponding to the same group of grouping information are run on the same data processing device.

[0007] In an exemplary embodiment, before obtaining the grouping information of the multiple groups of target algorithms, the method further includes:

[0008] Obtaining available resources of the plurality of data processing devices and resource consumption required to deploy any target algorithm;

[0009] The multiple target algorithms are grouped in units of the data processing devices according to the available resources and the resource consumption; the same target algorithm and the algorithm model corresponding to the algorithm are divided into the same group of data processing devices, and the same group of data processing devices corresponds to at least one target algorithm and the algorithm model corresponding to the at least one target algorithm.

[0010] In an exemplary embodiment, any set of grouping information includes at least one target algorithm information and algorithm model information corresponding to the at least one target algorithm information.

[0011] In an exemplary embodiment, grouping the plurality of target algorithms in units of the data processing device according to the available resources and the resource consumption includes:

[0012] The most commonly used set of algorithm models among the multiple target algorithms is used as the current algorithm model group, and a set of data processing devices is selected as the current data processing device;

[0013] Adding the current algorithm model group to the current data processing device;

[0014] Determine, based on the available resources of the current data processing device and the resource consumption of deploying all target algorithms corresponding to the current algorithm model group, whether the available resources of the current data processing device can accommodate the deployment of all target algorithms corresponding to the current algorithm model group;

[0015] When it is determined that the current data processing device can accommodate and deploy all target algorithms corresponding to the current algorithm model group, all target algorithms corresponding to the current algorithm model group are added to the current data processing device; a group of algorithm models that are most commonly used among the multiple ungrouped target algorithms is used as the current algorithm model, and the current algorithm model group continues to be added to the current data processing device.

[0016] In an exemplary embodiment, when it is determined that the current data processing device cannot accommodate and deploy all target algorithms corresponding to the current algorithm model group, the method further includes:

[0017] Add the target algorithms corresponding to the current algorithm model group that the current data processing device can accommodate to the current data processing device, add a new group of data processing devices as the current data processing device, add the algorithm models in the previous group of data processing devices to the current data processing device, and add the ungrouped target algorithms corresponding to the current algorithm model group to the current data processing device; take the most commonly used group of algorithm models among the multiple ungrouped target algorithms as the current algorithm model group, and continue to add the current algorithm model group to the current data processing device.

[0018] In an exemplary embodiment, the obtaining of grouping information of the plurality of target algorithms includes: obtaining an algorithm deployment table, the algorithm deployment table including grouping information of the plurality of target algorithms and resource configuration information of the target algorithms, the grouping information including a plurality of algorithm group identifiers;

[0019] The step of running the multiple sets of target algorithms on multiple data processing devices according to the grouping information of the multiple sets of target algorithms comprises:

[0020] Generate multiple first configuration files according to the multiple algorithm group identifiers, and write startup commands of all target algorithms corresponding to the same algorithm group identifier into the first configuration files corresponding to the group of algorithm identifiers;

[0021] configuring a data processing device for each of the plurality of first configuration files according to resource configuration information of the plurality of groups of target algorithms corresponding to the first configuration files;

[0022] Starting a microservice container in a corresponding data processing device according to the first configuration file, and starting a model manager in the microservice container;

[0023] Controlling the model manager to load the algorithm model corresponding to the group of target algorithms;

[0024] The corresponding target algorithm is run in the corresponding microservice container; the algorithms and model managers corresponding to the same set of algorithm group identifiers are started in the same data processing device.

[0025] In an exemplary embodiment, after running the corresponding target algorithm in the corresponding microservice container, the method further includes: outputting and saving the algorithm running result.

[0026] In an exemplary embodiment, running the corresponding target algorithm in the corresponding microservice container includes: running the corresponding target algorithm in the corresponding microservice container and calling an algorithm model required by the target algorithm.

[0027] In an exemplary embodiment, the algorithm deployment table further includes an algorithm code address and an algorithm execution path;

[0028] Before starting the corresponding target algorithm in the corresponding microservice container, the method further includes: obtaining the code of the target algorithm according to the algorithm code address;

[0029] The starting of the corresponding target algorithm in the corresponding microservice container includes: running the corresponding target algorithm code in the corresponding microserver according to the algorithm running path.

[0030] In an exemplary embodiment, the algorithm deployment table further includes a test video stream address, an algorithm name, and a feedback test output address;

[0031] After obtaining the algorithm deployment table, the method further includes: obtaining a video source file according to the test video stream address, pushing the video source file for the target algorithm test into a video stream through a preset push stream mirror, generating a pull stream address, and using the pull stream address to update the first configuration file of the corresponding target algorithm; the video stream address and the pull stream address include a video name, and the video name has a corresponding relationship with the corresponding algorithm name;

[0032] After running the corresponding target algorithm in the corresponding microservice container, the method further includes: traversing the target algorithm that needs to test the video stream according to the algorithm deployment table, starting the test platform, starting the target algorithm that needs to test the video stream to perform a playback test according to the corresponding video stream address, waiting for a preset time, collecting test reports fed back by multiple target algorithms, and sending information that failed the test to the abnormal information feedback platform through the feedback test output address.

[0033] In an exemplary embodiment, the algorithm deployment table also includes algorithm model information;

[0034] Before obtaining the grouping information of the multiple groups of target algorithms, the method further includes: converting the original algorithm model in the model warehouse into an open neural network exchange format, converting the open neural network exchange format to obtain a TensorRT model, and saving the TensorRT model to the model warehouse; and merging a portion of the network layers in the original algorithm model during the conversion to the TensorRT model;

[0035] The controlling the model manager to load the algorithm model corresponding to the group of target algorithms includes: obtaining algorithm model information corresponding to the target algorithm, and controlling the model manager to load the TensorRT model corresponding to the algorithm model information from the model warehouse.

[0036] In an exemplary embodiment, after running the corresponding target algorithm in the corresponding microservice container, the method further includes:

[0037] Test all the target algorithms according to the service deployment table, and output and save the test results.

[0038] In an exemplary embodiment, before obtaining the grouping information of the plurality of target algorithms, the method further includes: triggering periodic deployment;

[0039] After running the multiple groups of target algorithms on multiple data processing devices according to the grouping information of the multiple groups of target algorithms, the method further includes: triggering periodic detection.

[0040] In a second aspect, the present disclosure also provides an algorithm execution device.

[0041] Including acquisition module and operation module;

[0042] The acquisition module is configured to acquire grouping information of multiple groups of target algorithms;

[0043] The running module is configured to run multiple groups of target algorithms on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein the target algorithms corresponding to the same grouping information are run on the same data processing device.

[0044] In a third aspect, an embodiment of the present disclosure further provides an algorithm execution device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, to execute:

[0045] Obtain grouping information of multiple target algorithms;

[0046] Multiple groups of target algorithms are run on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein the target algorithms corresponding to the same group of grouping information are run on the same data processing device.

[0047] In a fourth aspect, an embodiment of the present disclosure further provides a non-volatile computer-readable storage medium, which is used to store computer program instructions, wherein the computer program instructions, when executed, can implement the algorithm operation method described in any of the above embodiments.

[0048] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide an understanding of the technical solutions of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions of the present disclosure.

[0050] Figure 1 Shown is a flow chart of an algorithm operation method provided by an embodiment of the present disclosure;

[0051] Figure 2a FIG2 is a schematic diagram of a logical structure of an automated deployment module provided by an exemplary embodiment of the present disclosure;

[0052] Figure 2b 1. It is a schematic diagram of the logical architecture of the AI ​​algorithm automatic detection provided by the exemplary embodiment of the present disclosure;

[0053] Figure 2c Shown is a schematic diagram of a Jenkins framework structure provided by an exemplary embodiment of the present disclosure;

[0054] Figure 3 FIG2 is a flowchart of an AI platform operation status check provided by an exemplary embodiment of the present disclosure;

[0055] Figure 4 FIG2 is a flowchart of an AI algorithm operation status check provided by an exemplary embodiment of the present disclosure;

[0056] Figure 5 Shown is a logical structure diagram of an algorithm indicator test provided by an exemplary embodiment of the present disclosure;

[0057] Figure 6a FIG2 is a logical framework diagram of a video source processing provided by an exemplary embodiment of the present disclosure;

[0058] Figure 6b FIG2 is a logical framework diagram of a video source processing provided by an exemplary embodiment of the present disclosure;

[0059] Figure 7FIG2 is a schematic diagram of a module of an algorithm running device provided by an embodiment of the present disclosure;

[0060] Figure 8 Shown is a schematic diagram of an algorithm running device module provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other in any manner.

[0062] Unless otherwise defined, the technical or scientific terms used in the embodiments of the present disclosure should have the ordinary meaning understood by a person of ordinary skill in the field to which the invention belongs. The words "first", "second" and similar terms used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. The words "include" or "comprising" and similar words always include the elements or objects listed after the word and their equivalents, without excluding other elements or errors.

[0063] In this specification, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they can refer to fixed, removable, or integral connections; mechanical or electrical connections; direct or indirect connections through intermediaries; and internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0064] After completing algorithm research and development, many problems will be encountered during the implementation of algorithm engineering, such as algorithm adaptation issues with the server environment, slow model inference speed, high resource usage, and cumbersome testing processes after algorithm deployment. These problems lead to low efficiency and high cost of AI algorithm engineering.

[0065] The present disclosure provides an algorithm operation method, such as Figure 1 As shown, the algorithm operation method may include:

[0066] Step M1: Obtain grouping information of multiple groups of target algorithms;

[0067] Step M2: running multiple groups of target algorithms on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein, target algorithms corresponding to the same grouping information are run on the same data processing device.

[0068] The algorithm execution method provided by the embodiments of the present disclosure runs multiple target algorithms on multiple data processing devices based on their grouping information. Target algorithms corresponding to the same grouping information are executed on the same data processing device. The method provided by the embodiments of the present disclosure can overcome the problem of high resource usage during algorithm deployment and operation, thereby achieving high efficiency and low cost in AI algorithm engineering.

[0069] In an exemplary embodiment, any group of grouping information includes at least one target algorithm information and at least one algorithm model information corresponding to the target algorithm information. In the embodiment of the present disclosure, multiple target algorithms running in the same data processing device are the same group of target algorithms, and the algorithm models corresponding to the multiple target algorithms and the corresponding multiple target algorithms are all running in the same data processing device, wherein the multiple target algorithms and the corresponding multiple model algorithms divided into the same group can usually be divided according to actual business needs. For example, under the restricted area monitoring and warning business process, there may be a human body recognition algorithm and a vehicle recognition algorithm, then the algorithm model corresponding to the human body recognition algorithm includes a human body detection algorithm model that has been deeply learned / trained, and the algorithm model corresponding to the vehicle recognition algorithm includes a vehicle detection algorithm model that has been deeply learned / trained; in the actual process of running the algorithm, the human body recognition algorithm is used as an example for explanation: in the process of running the algorithm, it is necessary to detect whether the human body enters the restricted area, then the human body recognition algorithm calls the human body detection algorithm model to detect whether the human body enters the restricted area, and outputs a warning message when the human body enters the restricted area.

[0070] In the embodiments of the present disclosure, among the multiple target algorithms and the multiple algorithm models running on the same data processing device, the number of target algorithms and the number of algorithm models may not be in a one-to-one relationship, but the multiple model algorithms running on the same data processing device are called when at least one of the multiple target algorithms is running. For example, a group of target algorithms may include ten target algorithms and five algorithm models, of which three target algorithms use five algorithm models during operation, and the other two target algorithms only call one or two of the target algorithms during operation.

[0071] In the embodiments of the present disclosure, multiple target algorithms in the same group of target algorithms, and the algorithm models corresponding to the multiple target algorithms, are run on the same data processing device, which can save resources. When a problem occurs with the data processing device or a failure occurs on the platform carrying the data processing device, the multiple groups of target algorithms can be run on other platforms that have not failed and data processing devices that have not had problems, thereby improving the disaster tolerance of the algorithm operation.

[0072] In an exemplary embodiment, before step M1, steps S1 and S2 may be included:

[0073] Step S1: Obtain available resources of multiple data processing devices and resource consumption required to deploy any target algorithm;

[0074] Step S2: Group multiple target algorithms into data processing devices based on available resources and resource consumption; the same target algorithm and the algorithm model corresponding to the algorithm are divided into the same group of data processing devices, and the same group of data processing devices corresponds to at least one target algorithm and at least one algorithm model corresponding to the target algorithm.

[0075] In the embodiment of the present disclosure, steps S1 to S2 may be performed manually or automatically through a script program or algorithm.

[0076] In the embodiment of the present disclosure, the operation method of the above-mentioned steps M1 and M2 can be applied to the algorithm deployment process.

[0077] In the embodiments of the present disclosure, the data processing device may be a GPU card (or GPU processor), but is not limited to a GPU card (or GPU processor). For example, the data processing device may be a CPU processor. The data processing device may be arranged in a target device, and the target device may be a cloud server, but is not limited to a cloud server. For example, the target device may be any server in a server cluster.

[0078] In the embodiments of the present disclosure, the resource consumption of the target algorithm may be the resources of the data processing device consumed when running the target algorithm. For example, the resource consumption of the target algorithm may be the space occupied by the GPU and / or CPU during the operation of the target algorithm. For example, if a target algorithm requires 50MB of GPU space during operation, the resource consumption of the target algorithm includes 50MB of GPU space.

[0079] In an exemplary embodiment, in step S2, grouping multiple target algorithms into data processing devices based on available resources and resource consumption may include steps S21 to S23:

[0080] Step S21: taking a group of algorithm models that are most commonly used among the multiple target algorithms as the current algorithm model group, and selecting a group of data processing devices as the current data processing devices;

[0081] Step S22: adding the current algorithm model group to the current data processing device;

[0082] Step S23: Based on the available resources of the current data processing device and the resource consumption of deploying all target algorithms corresponding to the current algorithm model group, determine whether the available resources of the current data processing device can accommodate the deployment of all target algorithms corresponding to the current algorithm model group; when it is determined that the current data processing device can accommodate the deployment of all target algorithms corresponding to the current algorithm model group, add all target algorithms corresponding to the current algorithm model group to the current data processing device; use the most commonly used group of algorithm models among the multiple ungrouped target algorithms as the current algorithm model, and continue to add the current algorithm model group to the current data processing device.

[0083] In an exemplary embodiment, in step S23, when it is determined that the current data processing device cannot accommodate and deploy all target algorithms corresponding to the current algorithm model group, the following steps may also be included:

[0084] Add the target algorithms corresponding to the current algorithm model group that the current data processing device can accommodate to the current data processing device, add a new group of data processing devices as the current data processing device, add the algorithm models in the previous group of data processing devices to the current data processing device, and add the ungrouped target algorithms corresponding to the current algorithm model group to the current data processing device; take the most commonly used group of algorithm models among the multiple ungrouped target algorithms as the current algorithm model group, and continue to add the current algorithm model group to the current data processing device.

[0085] In an exemplary embodiment, step M1 may include: obtaining an algorithm deployment table, the algorithm deployment table including grouping information of the multiple groups of target algorithms and resource configuration information of the target algorithms, the grouping information including multiple algorithm identification information;

[0086] In an exemplary embodiment, the algorithm deployment table may be a CSV file, and the algorithm deployment table may be filled in by an algorithm developer, or a CSV file may be generated based on information filled in by a user.

[0087] Step M2 may include:

[0088] Step M21: generating multiple first configuration files according to multiple algorithm group identifiers, and writing startup commands of all target algorithms corresponding to the same algorithm group identifier into the first configuration files corresponding to the group of algorithm identifiers;

[0089] Step M22: configuring a data processing device for each of the plurality of first configuration files according to resource configuration information of the plurality of target algorithms corresponding to the first configuration files; Step M23: starting a microservice container in the corresponding data processing device according to the first configuration files, and starting a model manager in the microservice container;

[0090] In an embodiment of the present disclosure, the first configuration file may be a Kubernetes configuration file, the microservice container may be a Kubernetes container, and the model manager may be a Triton server.

[0091] Step M2: Control the model manager to load the algorithm model corresponding to the target algorithm;

[0092] Step M25: Run the corresponding target algorithm in the corresponding microservice container; the algorithms and model managers corresponding to the same set of algorithm group identifiers are started in the same data processing device.

[0093] In an exemplary embodiment, after running the corresponding target algorithm in the corresponding microservice container, the method further includes: outputting and saving the algorithm running result.

[0094] In an exemplary embodiment, running a corresponding target algorithm in a corresponding microservice container includes: running the corresponding target algorithm in the corresponding microservice container, and calling an algorithm model required by the target algorithm.

[0095] In an exemplary embodiment, the output algorithm operation results can be fed back to the JIRA platform through the JIRA interface. The relevant person in charge can obtain the corresponding operation results by logging into the JIRA platform, thereby realizing closed-loop management of algorithm deployment and improving algorithm deployment efficiency.

[0096] In the embodiment of the present disclosure, the algorithms and model managers corresponding to the same set of algorithm grouping identifiers are started in the same data processing device, so that the algorithms and model managers of the same group only need one GPU resource to run, without any other environmental requirements, and can be run on any GPU in the same cluster, thereby having a relatively high disaster tolerance.

[0097] In an exemplary embodiment, the algorithm deployment table may further include an algorithm code address and an algorithm execution path;

[0098] Before executing step M25 and starting the corresponding target algorithm in the corresponding microservice container, the method may further include: obtaining the code of the target algorithm according to the algorithm code address;

[0099] In step M25, starting the corresponding target algorithm in the corresponding microservice container may include: running the corresponding target algorithm code in the corresponding microservice according to the algorithm running path.

[0100] In an exemplary embodiment, the algorithm deployment table further includes a test video stream address, an algorithm name, and a feedback test output address;

[0101] After executing step M1 to obtain the algorithm deployment table, the following steps may also be included: obtaining a video source file according to the test video stream address, pushing the video source file for the target algorithm test into a video stream through a preset push stream mirror, generating a pull stream address, and using the pull stream address to update the first configuration file corresponding to the target algorithm; the video stream address and the pull stream address include a video name, and the video name has a corresponding relationship with the corresponding algorithm name;

[0102] In step M21, after running the corresponding target algorithm in the corresponding microservice container, it can also include: traversing the target algorithm that needs to test the video stream according to the algorithm deployment table, starting the test platform, starting the target algorithm that needs to test the video stream to perform a playback test according to the corresponding video stream address, waiting for a preset time, collecting test reports fed back by multiple target algorithms, and sending the information that failed the test to the abnormal information feedback platform through the feedback test output address.

[0103] In an exemplary embodiment, the video name and the corresponding algorithm name have a corresponding relationship, which may include: the video name is the same as the corresponding algorithm name, or other corresponding relationships.

[0104] In an exemplary embodiment, the abnormal information feedback platform may be a JIRA platform, and the feedback test output address may be a jira ID corresponding to the target algorithm.

[0105] In an exemplary embodiment, the algorithm deployment table may further include algorithm model information;

[0106] Before executing step M1 to obtain the grouping information of multiple groups of target algorithms, the method may further include: converting the original algorithm model in the model warehouse into an open neural network exchange format, converting the open neural network exchange format to obtain a TensorRT model, and saving the TensorRT model to the model warehouse; in the process of converting to the TensorRT model, merging a portion of the network layers in the original algorithm model;

[0107] In step M24, controlling the model manager to load the algorithm model corresponding to the target algorithm may include: obtaining the algorithm model information corresponding to the target algorithm, and controlling the model manager to load the TensorRT model corresponding to the algorithm model information from the model repository.

[0108] In the disclosed embodiment, the original algorithm model may be a PyTorch model. In the disclosed embodiment, converting the original PyTorch algorithm model in the model repository to an open neural network exchange format, and converting the open neural network exchange format to obtain a TensorRT model can improve the inference speed of the model.

[0109] In an exemplary embodiment, after running the corresponding target algorithm in the corresponding microservice container, step M25 may further include: testing all target algorithms according to the business deployment table, and outputting and saving the test results. In an exemplary embodiment, the output test results can be fed back to the JIRA platform via the JIRA interface. Relevant personnel can access the corresponding test results by logging into the JIRA platform, thereby achieving closed-loop management of algorithm deployment and improving algorithm deployment efficiency.

[0110] In an exemplary embodiment, before executing step M1, the following may also be included: triggering periodic deployment. In the embodiment of the present disclosure, when deploying large-scale algorithms, the algorithm deployment process can be controlled by an automated deployment script. There are two types of automated deployment methods: one is manual execution by a user logging into the deployment server, and the other is periodically triggering the automated deployment platform through Jenkins to automatically deploy the target algorithm to the target device.

[0111] In an exemplary embodiment, after executing step M2, the process may further include: triggering periodic detection.

[0112] In an exemplary embodiment, the automated detection platform (hereinafter referred to as the detection platform) can be periodically triggered by Jenkins to automatically detect the target algorithm deployed on the target device, which can improve the real-time performance and detection efficiency of the algorithm detection.

[0113] In the disclosed embodiment, Jenkins is used to periodically trigger automated detection and periodically trigger automated deployment, and the test results and the running results after the algorithm is deployed are fed back to the JIRA platform through the JIRA interface. Users can obtain the corresponding test results or deployment and running results by logging into the JIRA platform, thereby forming a closed-loop development and improving development efficiency, thereby improving the efficiency of algorithm engineering implementation and reducing the cost of algorithm engineering implementation.

[0114] In the embodiment of the present disclosure, the above-mentioned periodic triggering of automatic detection can be process-controlled through test scripts.

[0115] In the embodiment of the present disclosure, the above-mentioned target algorithm can be an AI algorithm. The following is a detailed description of the algorithm deployment: In the embodiment of the present disclosure, after the target algorithm has been trained with a model and the algorithm code has been written, the next step facing the AI ​​algorithm is deployment. Model deployment is different from model training. When various AI algorithms are put into productization, they must maintain various performance indicators of the algorithm and be fast enough (the minimum requirement must be able to process in real time). Various algorithms are mostly deployed to cloud servers according to different business scenarios. The main challenge is concurrent service capabilities, and the main indicators are throughput and latency.

[0116] TensorRT is a software stack from NVIDIA for accelerating deep learning models. It provides numerous model optimization methods, including the fusion of deep neural network layers, automatic selection of the optimal kernel implementation based on the target GPU, memory reuse, and INT8 type quantization.

[0117] Triton Inference Server is an open-source software stack that transforms AI inference into a service. It unifies the management of models from various deep learning frameworks, such as TensorFlow, PyTorch, TensorRT, and ONNX. Triton Inference Server also supports concurrent model inference.

[0118] like Figure 2a The figure below is a schematic diagram of the logical structure of the automatic deployment module for deploying the target algorithm. The automatic deployment module can include the following modules:

[0119] 1. Resource warehouse: including model warehouse, algorithm code warehouse, and image warehouse.

[0120] Model warehouse: used to store the weight files of each functional model after training. When the AI ​​algorithm is deployed, the model weights are uniformly pulled from the model warehouse according to the models required by each business.

[0121] Algorithm code warehouse: The algorithm code warehouse is used to store the policy code and corresponding algorithm code of each business. When the AI ​​algorithm is deployed, the algorithm code is pulled from the algorithm code warehouse according to the algorithm business.

[0122] Image repository: This can be a Docker image repository, used to store the Docker images used in the AI ​​algorithm deployment process. During deployment, a fixed version of the image is directly used to start the Kubernetes container.

[0123] 2. Model Acceleration Module

[0124] The disclosed embodiments utilize a model acceleration technology stack from PyTorch to Onnx to TensorRT. The original model is first converted to Onnx (Open Neural Network Exchange) and then to a TensorRT model. This conversion merges some of the original model's network layers and optimizes them for NVIDIA GPUs, thereby increasing the model's inference speed. The converted model is also stored in the model repository for deployment.

[0125] 3. Algorithm deployment table

[0126] The algorithm deployment table, also known as the business deployment table, is the core file for automated deployment and testing of this system. The business deployment table used during algorithm deployment and testing is a CSV file with columns such as: algorithm name, author, algorithm path, model used, test video address, model group, and jira ID. This business deployment table contains all the information required to deploy and test the AI ​​algorithm. During subsequent automated deployment, the automated deployment script will start all AI algorithms that need to be deployed based on the information in the business deployment table. The business deployment table is filled out by business developers. Among them:

[0127] Algorithm name field: can uniquely identify an algorithm, and the test video name is the same as the algorithm name;

[0128] Algorithm Path field: This field identifies the algorithm group and contains the path to the target algorithm's entry file. The automatic deployment program will directly run this file to start the AI ​​service.

[0129] Model group field: This field can be used to identify the algorithm group mentioned above. It is used to identify the target algorithm group. Algorithm businesses in the same group and the algorithm models they require will run on the same GPU, corresponding to a Kubernetes pod.

[0130] The jiraID field is the jira bug report address for this business. If the automated test for this business fails, the log file and failure information of this business will be automatically reported to this address.

[0131] IV. Model and Business Grouping

[0132] The disclosed embodiment deploys large-scale AI algorithms in a grouping manner. Since large-scale deployment requires numerous business models and a single graphics card is insufficient to support all businesses, the present invention innovatively groups the deployed businesses into units of GPU graphics cards.

[0133] Grouping methods can include:

[0134] Step 001: The most commonly used algorithm model group among multiple target algorithms is used as the current algorithm model group, and a group of data processing devices is selected as the current data processing device;

[0135] Step 002: Add the current algorithm model group to the current data processing device;

[0136] Step 003: judging whether the available resources of the current data processing device can accommodate the deployment of all target algorithms corresponding to the current algorithm model group based on the available resources of the current data processing device and the resource consumption of deploying all target algorithms corresponding to the current algorithm model group;

[0137] When it is determined that the current data processing device can accommodate and deploy all target algorithms corresponding to the current algorithm model group, all target algorithms corresponding to the current algorithm model group are added to the current data processing device; the most commonly used algorithm model among the multiple ungrouped target algorithms is used as the current algorithm model, and step 002 is continued;

[0138] When it is determined that the current data processing device cannot accommodate all target algorithms corresponding to the current algorithm model group, the target algorithms corresponding to the current algorithm model group that can be accommodated by the current data processing device are added to the current data processing device, a new group of data processing devices is added as the current data processing device, and the algorithm models in the previous group of data processing devices are added to the current data processing device. It is determined whether there are any ungrouped target algorithms corresponding to the current algorithm model. If so, the ungrouped target algorithms corresponding to the current algorithm model group are added to the current data processing device. Otherwise, step 004 is executed.

[0139] Step 004: Take the most commonly used set of algorithm models among the multiple ungrouped target algorithms as the current algorithm model group, and continue to execute step 002.

[0140] 5. Automated Deployment Module

[0141] In the disclosed embodiments, the automated deployment module is triggered by an entry script file that sequentially calls all deployment program modules. There are two ways to trigger the automated deployment script: manual execution by the deployer logging into the server, and automatic scheduled execution using Jenkins. Jenkins is a Java-based continuous integration tool. In the disclosed embodiments, Jenkins' scheduled deployment and testing approach provides timely bug feedback and accelerates development iterations.

[0142] like Figure 2a As shown, the automated deployment module may include:

[0143] Automatically pull the latest code: Pull the latest AI algorithm code from the code repository to ensure that the deployed AI algorithm is consistent with the remote code repository.

[0144] Automatic streaming: According to the test video address field in the algorithm deployment table, the test video required by each algorithm is pushed into a real-time video stream for the algorithm to pull.

[0145] The method for automatic streaming can include: the test video name corresponding to the target algorithm is the same as the algorithm name, finding the target algorithm and the corresponding test video address belonging to the same group, using the streaming image in the image repository to push this video into a video stream, and updating the streaming address in the target algorithm configuration file to this video stream address.

[0146] Automatically generate a Kubernetes configuration file and start the container: The automatic deployment module writes the startup commands for all AI algorithms in the same group into the Kubernetes configuration file based on the algorithm deployment table. It then configures the container's mount directory, image name, and other settings based on a Kubernetes configuration template. The Kubernetes configuration file can be understood as the first configuration file mentioned above, and the Kubernetes configuration template can be understood as a second configuration file different from the first configuration file. The second configuration file can be set independently or can be set within the first configuration file.

[0147] Starting the Triton server: In this disclosed embodiment, the Triton server and the AI ​​algorithm are started in the same Kubernetes container. The model managed by the Triton server and the AI ​​algorithm are grouped together. The advantage of this approach is that the AI ​​algorithm and model group only require a single GPU resource to run, without any other environmental requirements. This method is highly disaster-tolerant and can run on any GPU in the Kubernetes cluster. After the Kubernetes container starts, the Triton server is first started to load the deep learning model required by the AI ​​algorithm group.

[0148] Start the target algorithm:

[0149] After the automatic deployment module loads the model, it will start all AI algorithms in the same group and store the output logs of the AI ​​algorithms in a fixed directory for debuggers to view.

[0150] Start the automatic testing program: The last step of the automated deployment module is to start the automatic testing program. The automatic testing program will test all AI algorithms in the same group according to the algorithm deployment table and automatically report the algorithm operation results.

[0151] 6. Automated testing module

[0152] The automatic testing program will test all AI algorithms in the same group according to the algorithm deployment table and automatically report the results.

[0153] The automated testing method may include: the automatic testing program traverses all programs in the same group that need to be automatically tested according to the algorithm deployment table. If one of the algorithm businesses in the algorithm deployment table belongs to the group that needs to be checked, a process is started to check all the files that should be output by this AI algorithm during the test video test. After waiting for 15 minutes, each process will feed back the test results of the business it is responsible for to the parent process. The parent process collects these test information, summarizes them and sends them to the jira test report. If there is a business that fails the test, the failure result and algorithm log will be sent separately to the jiraID corresponding to this algorithm business in the algorithm deployment table.

[0154] like Figure 2b The figure shows the logical framework of the automated testing platform in the automated testing module. After completing model development and strategy development, R&D personnel submit the algorithm to the code storage server (which can be understood as the above-mentioned code library). The operation and maintenance platform or the testing platform periodically triggers the automated detection of the algorithm in the code storage server through Jenkins. The algorithm detection content may include configuration detection, compilation detection, model detection, AI platform startup status detection and algorithm operation status detection. In the embodiments of the present disclosure, the AI ​​platform can be understood as the above-mentioned target device or the data processing equipment in the above-mentioned target device.

[0155] like Figure 2c As shown, the Jenkins framework can include six configuration modules:

[0156] General module: Some basic configurations of the build task, discarding old builds, setting the build history preservation strategy; choose to set a parameterized build process, you can configure different parameters, which are convenient for referencing these parameters during the build.

[0157] Source code management module: Select GIT and set corresponding GIT parameters. In an exemplary embodiment, setting the GIT parameters may be setting a GIT address, which may be an SVN address for accessing a code storage server.

[0158] Build the trigger module: Select Scheduled Build and set the corresponding time parameters. After building the trigger module, you can trigger the test periodically.

[0159] Build environment module: Select Delete workspace before build starts build tool.

[0160] Building a module: Generally, building a module environment is to write an executable file. This embodiment of the disclosure does not make any settings for this module.

[0161] Post-construction operation module: implemented by designing calling commands and writing scripts.

[0162] According to the cycle set by the build trigger module and the GIT parameters set by the source code management module, the algorithm code is periodically pulled from the GIT address for testing.

[0163] The algorithm detection method is described in detail below:

[0164] (1) Jenkins automatically pulls Git code: When the test cycle arrives, periodic automated testing is triggered. The testing platform automatically pulls the algorithm code corresponding to the Git address from the code storage server through Jenkins. Each algorithm code in the code storage server corresponds to a Git address, and the testing platform can access the corresponding algorithm code in the code storage server through the Git address. In an exemplary embodiment, the code storage server can be referred to as the code storage platform.

[0165] In an exemplary embodiment, the operation and maintenance platform obtains the corresponding algorithm code through a git address. When batching algorithms online or testing them, Jenkins can be used to obtain the corresponding git address from the code storage server. Based on the git address, the corresponding multiple algorithm codes are obtained, thereby implementing batch algorithm online or batch algorithm testing. In an exemplary embodiment, the same git address can correspond to multiple algorithms in the batch, or each algorithm can correspond to a git address.

[0166] (2) Generate configuration benchmark file: Generate configuration benchmark file based on algorithm code.

[0167] In an exemplary embodiment, the configuration benchmark file may include the algorithm names, algorithm model parameters, database path parameters required for algorithm operation, resource configuration parameters, and video stream information used as algorithm input for multiple algorithms in the batch algorithm. The video stream information includes information such as the corresponding algorithm name, algorithm strategy information, and frame rate threshold. In an exemplary embodiment, the configuration benchmark file may also include information about the person in charge of algorithm development and the person in charge of the platform.

[0168] In an exemplary embodiment, resource configuration parameters may include occupying resources such as a CPU and a GPU. For example, an algorithm may occupy 100M of space on a CPU or a GPU and 50M of space on a GPU.

[0169] (3) Generate a configuration file in CSV format based on the configuration benchmark file.

[0170] In an exemplary embodiment, in order to adapt to the format requirements of the detection platform for the configuration file, a configuration file in CSV format is generated according to the configuration reference file, and the configuration file in CSV format is used as the standard in the subsequent detection process.

[0171] In an exemplary embodiment, a CSV-formatted configuration file may include two sequentially arranged sections. The first section may include basic information for multiple algorithms, including the algorithm name, algorithm model parameters, and database path parameters required for algorithm operation. The second section may include algorithm input information, including the aforementioned video stream information. The basic information for multiple algorithms in the first section may be sequentially arranged, and the algorithm input information for multiple algorithms in the second section may be sequentially arranged.

[0172] (4) CSV generation check: Check whether the configuration file in CSV format is in the standard format specified by the detection platform. If it is not in the standard format specified by the detection platform, call the JIRA interface to report the configuration bug of the corresponding algorithm.

[0173] In an exemplary embodiment, Comma-Separated Values ​​(CSV), sometimes also called character-separated values ​​because the delimiting character can be anything other than a comma, stores tabular data (numbers and text) in plain text format. Plain text means the file is a sequence of characters and does not contain data that must be interpreted like binary numbers. A CSV file consists of any number of records, separated by some type of line break character. Each record consists of fields, separated by other characters or strings, most commonly a comma or a tab.

[0174] In an exemplary embodiment, the CSV check may include checking whether a configuration file in CSV format complies with the format requirements of a standard configuration file. For example, the format of a standard configuration file is that records are separated by commas. If a semicolon is found between records in a CSV format configuration file, an abnormal CSV generation occurs.

[0175] (5) Configuration checking may include: checking whether the algorithm name in the basic information of the configuration file is consistent with the algorithm name in the algorithm input information; if not, calling the JIRA interface to report configuration bugs of the corresponding algorithm.

[0176] In an exemplary embodiment, some algorithms do not require input video streams, and the absence of input video stream information can be marked in the algorithm name of the basic information. When it is detected that the algorithm name is marked with no input video stream information, it can be determined based on the marking that no abnormality has occurred, and the JIRA interface does not need to be called to report the bug of the corresponding algorithm.

[0177] In an exemplary embodiment, even if some algorithms do not use video stream information as input during actual operation, video stream information is configured during model development and strategy development, but the corresponding video stream resources are not used during operation. In this case, if the video stream information corresponding to the algorithm name is not detected during the configuration check, the JIRA interface can be called to report the bug of the corresponding algorithm.

[0178] In an exemplary embodiment, bug is a general term for loopholes, defects, and errors in software, programs, codes, algorithms, and computer systems.

[0179] (6) Compile code: Jenkins calls the compilation interface according to the compilation instructions to compile the algorithm code.

[0180] In an exemplary embodiment, Jenkins obtains the compilation instructions of the corresponding algorithm from the git address and automatically calls the compilation interface to compile the algorithm code, which can reduce the manual deployment of the compilation environment and the manual compilation process, thereby reducing labor costs and improving efficiency.

[0181] (7) Compilation checking may include checking whether the algorithm compilation process reports an error and checking whether the algorithm compilation result is successful. If the compilation process reports an error or the compilation result is unsuccessful, calling the JIRA interface to report the compilation bug of the corresponding algorithm. In an exemplary embodiment, checking whether the algorithm compilation process reports an error and checking whether the algorithm compilation result is successful may include obtaining the Jenkins compilation log and checking whether there are any errors in the compilation log. For example, checking whether there is information such as "error" in the compilation log.

[0182] (8) Model checking may include: checking whether the model files required by the algorithm to be launched are prepared correctly according to the configuration file; if it is detected that the model files are not prepared, calling the JIRA interface to report the model bug of the corresponding algorithm.

[0183] In an exemplary embodiment, checking whether the model file required by the algorithm currently being launched is prepared correctly may include searching whether the model file of the corresponding algorithm exists according to the model parameters in the configuration file.

[0184] In an exemplary embodiment, during the above-mentioned CSV generation check, configuration check, and model check, if an exception occurs, the JIRA interface service is called to automatically submit the corresponding bug to the JIRA server. The JIRA server displays the corresponding bug to the user through the browser, and the corresponding developer can view the corresponding bug through the corresponding browser.

[0185] During the manual rollout process, when problems arise in operations or testing, operations or testers typically communicate with developers. Furthermore, operations or testers don't fully understand where the development bugs originated, leading to high communication costs. In the disclosed embodiment, bug information is uploaded to a JIRA server via the JIRA interface. Developers, platform managers, testers, or operations personnel can view the corresponding bug information by logging into their JIRA accounts, significantly reducing communication costs. In an exemplary embodiment, Jenkins is an open-source continuous integration (CI) tool with a user-friendly interface. It is primarily used for continuously and automatically building / testing software projects and monitoring the execution of external tasks.

[0186] (8) Check the operating status of the AI ​​platform.

[0187] In the embodiments of the present disclosure, the AI ​​platform can be understood as a cloud platform, or other platforms deployed with AI algorithms.

[0188] like Figure 3 As shown, the AI ​​platform operation status check may include the following steps:

[0189] Step 11: Start the AI ​​platform and wait for the first preset time to execute step 12.

[0190] In an exemplary embodiment, the first preset time may be 1 minute to 5 minutes. For example, the first preset time may be 3 minutes.

[0191] In an exemplary embodiment, the AI ​​platform may be started after the code is compiled, and the AI ​​platform operating status check may be performed after the compilation check and the model check are performed. After the execution of the AI ​​platform operating status check, step 12 may be performed after waiting for a first preset time.

[0192] Step 12: Check whether the AI ​​platform service exists. If yes, complete the check; otherwise, proceed to step 13.

[0193] In an exemplary embodiment, to check whether the AI ​​platform service exists, it is possible to check whether the process of the AI ​​platform is started. If the process is not started, step 13 is executed; if the process is already started, the check is completed.

[0194] Step 13: Link to JIRA interface and submit the bug.

[0195] In step 13, the AI ​​platform startup exception is submitted to the JIRA server through the JIRA interface. The user (AI platform manager or developer) logs in to the JIRA server to view the corresponding bug and solve the corresponding problem. In the embodiment of the present disclosure, the JIRA server can be the above-mentioned exception information feedback platform.

[0196] (9) Check the algorithm running status.

[0197] like Figure 4 As shown, the AI ​​algorithm running status check may include the following steps:

[0198] Step 21: Start the thread group corresponding to the algorithm.

[0199] In an exemplary embodiment, after the process in the AI ​​platform is started, the operation of starting the algorithm can be executed.

[0200] In an exemplary embodiment, the startup algorithm can start threads corresponding to the number of algorithms after the AI ​​platform process is started. When multiple algorithms are started, each algorithm corresponds to a thread, and a thread group of multiple threads is started in the process.

[0201] Step 22: Read the configuration file and add the algorithms marked in the configuration file that need to be detected to the thread group of the AI ​​platform.

[0202] In an exemplary embodiment, during batch algorithm testing or rollout, due to limited thread group resources, only a portion of the algorithms may be added to the current thread group. The remaining algorithms can be added to other thread groups or tested during the next test. In an exemplary embodiment, all algorithms recorded in the configuration file can be tested by default, without setting a flag indicating whether testing is required.

[0203] In an exemplary embodiment, each algorithm is loaded into one of the threads in the thread group.

[0204] Step 23: Run multiple thread groups. When any AI algorithm in the multiple thread groups runs abnormally, information about the corresponding AI algorithm running abnormality is sent to the abnormal information feedback platform.

[0205] In an exemplary embodiment, when an abnormality occurs in the detection algorithm, the JIRA interface will be linked to submit a bug and feedback to the JIRA service platform (i.e., the abnormality feedback platform). The algorithm manager can log in to the JIRA server, view the JIRA bug, and handle the corresponding algorithm abnormality.

[0206] In an exemplary embodiment, after the algorithm detection is performed, if there is no abnormality in the detection result, the output result of the algorithm can be obtained.

[0207] Step 24: Start the summary thread, summarize the test results and feed them back to the JIRA platform through the JIRA interface.

[0208] In an exemplary embodiment, the configuration file may include the email addresses of the R&D manager and the AI ​​platform manager. After the JIRA platform receives the corresponding bug, it may send the corresponding bug information to the corresponding R&D manager or AI platform manager via the email address.

[0209] In an exemplary embodiment, the summary thread feeds the overall test results to the JIRA server via the JIRA interface. The AI ​​platform manager then logs in to the JIRA server to retrieve the test results and, based on the test results, confirms whether the algorithm's launch results meet expectations. In an exemplary embodiment, the overall test results may include: the total number of algorithm codes tested, the number of successful and failed algorithm tests, a list of successful and failed tests, and a list of failed tests.

[0210] In an exemplary embodiment, if a bug exists during the thread execution algorithm detection process, the test is considered unsuccessful, and the corresponding bug information is uploaded to the JIRA platform through JIRA.

[0211] In an exemplary embodiment, the success list includes a list of algorithms that have successfully been tested, and the failure list includes a list of algorithms that have failed the test.

[0212] In an exemplary embodiment, the person in charge of the AI ​​platform confirms whether the results of the algorithm launch meet expectations based on the test results. The judgment can be made based on the type of the algorithm that failed or succeeded in the test. For example, there are 21 algorithms in total for batch testing. If an algorithm test has an exception, and the platform person in charge evaluates that the algorithm with the exception is not required to be launched this time, then only the 20 algorithms that have successfully been tested can be launched, and the algorithm test meets expectations this time. If there are 21 algorithms in total for testing, and 10 algorithms that must be launched have an exception in the test, then expectations cannot be met and the algorithm cannot be launched. The corresponding R&D person in charge needs to resolve the corresponding bug and then retest, that is, repeat the above test process (1) to (9) until the test expectations are met before the algorithm can be launched.

[0213] In an exemplary embodiment, the detection results are automatically submitted to the JIRA platform server, which can achieve a pipeline effect, eliminate the need for manual operation, and save labor costs.

[0214] In an exemplary embodiment, the detection results may include a detection log and an exception record. The detection log may include the detection time and the above-mentioned success list and failure list. For example, the detection log is as follows:

[0215] 2021-10-18 16:10:25[model_repository2]auto test end! total:16failed:7

[0216] FAILED LIST:['highway_lowspeed','drive_without_license',drive_license_without_permission','drive_inout','driver_car_match','station_leave','wandering_alarm']

[0217] NEW JIRA LIST: []

[0218] YF2021430-131

[0219] The above test log records that the test ended at 16:10:25 on October 18, 2021. A total of 16 tests were conducted, with 7 failures. The algorithms that failed the tests in the failure list include:

[0220] 'highwayjowspeed','drive_withoutjicense','drive_license_without_permission','drivejncut','driver_car_match','stationjeave','wandering_alarm'

[0221] The exception record summary includes:

[0222] [AI300OnlineCheck:C-Video][check.CorfigCheckLog]ERRORBUG exists invehiclebreakin

[0223] [AI300OnlineCheck:C-Video][CHECK_CompleCheckLog]ERRORBUG exists inNonVehiclelllegalParkingDetect

[0224] [Al300OnlineCheck:C-Video][check_CorfigCheckLog]ERRORBUG exists invehiclebreakin

[0225] In an exemplary embodiment, the Jenkins integration can be set up to run automatic online detection at a scheduled time to improve detection efficiency. For example, the online detection service can be set up on Jenkins to run automatically at 11:30 am and 4:30 pm every weekday to facilitate the algorithm to go online in the morning or afternoon.

[0226] In the embodiment of the present disclosure, the algorithm runs on the AI ​​platform to provide a message interface for the business, and the actual business scenario may require the access of one or more cameras. If the platform resources are insufficient, problems such as video stream processing failure and service shutdown may occur. In order to avoid problems such as video stream processing failure and service shutdown due to insufficient platform resources after access, after the algorithm detection has no abnormalities and is successfully launched, and before multiple cameras are connected, the algorithm indicators of multiple cameras when connected to the AI ​​platform can be tested. In an exemplary embodiment, the algorithm indicators of N cameras connected to the AI ​​platform under a single card / single machine configuration can be tested, and a curve relationship diagram of the algorithm indicator value and the number of cameras under the existing service configuration of the platform can be obtained, which has data significance for the advance planning and design of product implementation and resource allocation. In the embodiment of the present disclosure, a single card can refer to a graphics processing unit (GPU), also known as a display core, a visual processor, or a display chip. A single machine can be a physical machine configured with multiple GPU cards.

[0227] In the embodiment of the present disclosure, the logic of the algorithm indicator test is as follows: Figure 5 As shown, the following describes the video stream, AI service platform, and indicator data:

[0228] Video stream: The input source for AI platform services. Multiple video streams can be simulated using video files, or by converting one video stream into multiple streams.

[0229] In an exemplary embodiment, a video file may be copied into N copies, and the N copies of the video files may be respectively transferred to form N video streams; or a video file may be transferred to form a video stream, and the video stream may be copied into N copies to form N video streams.

[0230] AI platform services: Algorithm services based on the AI ​​platform framework. Input is one or more video streams; output includes frame rate, number of processed messages, message files, and system resource usage (such as CPU / GPU usage). AI platform services include video stream decoding, algorithm processing, and the recording and output of indicator data.

[0231] Metrics: These are the metrics outputs required by the AI ​​platform service when processing N streams. For example, for the perimeter intrusion algorithm, the required outputs include the number of alarm messages, average processing frame rate (fps), pixel location of the alarm image detection frame, and system resource utilization (CPU / GPU).

[0232] In an exemplary embodiment, Figure 6a and Figure 6b As shown in the figure, it is a logical framework diagram for processing two video sources. Figure 6a As shown, the video file is used as the video source:

[0233] Streaming service: Provides video file conversion services. A single video file can be converted into N specified video streams. The converted video streams serve as the video stream input for the AI ​​platform service.

[0234] AI platform services: Figure 5 The services provided by the AI ​​service platform shown are described in detail in the description of the AI ​​service platform above and will not be repeated here.

[0235] Result data processing: Perform data processing on the output of AI platform services to obtain the corresponding indicator relationship diagram.

[0236] like Figure 6b As shown, a real camera is used as the video source input:

[0237] Streaming service: Provides streaming services. It can convert a camera's video stream into N specified video streams. The output video streams serve as the video stream input for the AI ​​platform service.

[0238] AI platform services: Figure 5 The services provided by the AI ​​service platform shown are described in detail in the description of the AI ​​service platform above and will not be repeated here.

[0239] Result data processing: Perform data processing on the output of AI platform services to obtain the corresponding indicator relationship diagram.

[0240] In an exemplary embodiment, the obtained indicator relationship graph may include an accuracy-camera number curve graph.

[0241] In the embodiments of the present disclosure, Figure 6a and Figure 6b The result data processing can be implemented in the form described in step S2 above. The result data processing ultimately obtains the test results of the algorithm indicators.

[0242] In the embodiments of the present disclosure, a video stream can be generated in a simulated manner, which has the following advantages compared to a video stream from a real camera:

[0243] (1) It can ensure the consistency of input sources and the comparison of indicator conclusions.

[0244] (2) It can ensure that the density of a single frame meets specific requirements. For example, the number of people in a single frame must reach 30, and the index value of the capacity test can be obtained; however, it is difficult for a real camera to ensure the density of a single frame.

[0245] (3) Easy to expand and build. The indicator values ​​of N channels (such as 8 channels, 16 channels, 32 channels, and 100 channels) can be compared according to actual needs.

[0246] Based on the above three points, when there are many routes to compare, it is difficult to quickly achieve the number of cameras, procurement, installation, and crowd density simulation using real cameras.

[0247] In an exemplary embodiment, Figure 6a and Figure 6b In the two video stream simulation methods, the above Figure 6a The video stream obtained by using the video file is the same as the above Figure 6b Compared with the video stream simulated by a real camera, it is easy to formulate scene videos that meet the single-frame image density.

[0248] The present disclosure also provides an algorithm running device, such as Figure 7 As shown, it may include an acquisition module 01 and an operation module 02;

[0249] Acquisition module 01 can be configured to acquire grouping information of multiple target algorithms;

[0250] The running module 02 may be configured to run multiple sets of target algorithms on multiple data processing devices according to the grouping information of the multiple sets of target algorithms; wherein the target algorithms corresponding to the same set of grouping information are run on the same data processing device.

[0251] The present disclosure also provides an algorithm execution device, such as Figure 8 As shown, the system may include a memory, a processor, and a computer program stored in the memory and executable on the processor to perform:

[0252] Obtain grouping information of multiple target algorithms;

[0253] Multiple sets of target algorithms are executed on multiple data processing devices based on the grouping information of the multiple sets of target algorithms; wherein the target algorithms corresponding to the same set of grouping information are executed on the same data processing device. The disclosed embodiments also provide a non-transitory computer-readable storage medium for storing computer program instructions, wherein the computer program instructions, when executed, implement the algorithm execution method described in any of the above embodiments.

[0254] The disclosed embodiments provide an algorithm execution method, apparatus, device, and storage medium. In the algorithm execution method, multiple target algorithms are executed on multiple data processing devices based on their grouping information. Target algorithms corresponding to the same grouping information are executed on the same data processing device. The methods provided by the disclosed embodiments can overcome the high resource usage during algorithm deployment and execution, thereby enabling efficient and cost-effective AI algorithm engineering.

[0255] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0256] The drawings of the embodiments of the present disclosure only involve the structures involved in the embodiments of the present disclosure, and other structures may refer to general designs.

[0257] In the absence of conflict, the embodiments of the present invention, that is, the features in the embodiments, can be combined with each other to form new embodiments.

[0258] Although the embodiments disclosed herein are as described above, the contents are merely embodiments for facilitating understanding of the present invention and are not intended to limit the present invention. Any person skilled in the art may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope of the present invention. However, the scope of patent protection of the present invention shall still be subject to the scope defined by the appended claims.

Claims

1. An algorithm operation method, characterized in that: include: Periodically triggering the automated deployment platform to automatically deploy the target algorithm to the target device, and setting the data processing equipment in the target device; Obtain grouping information of multiple target algorithms; The obtaining of grouping information of the plurality of target algorithms comprises: obtaining an algorithm deployment table, wherein the algorithm deployment table comprises grouping information of the plurality of target algorithms and resource configuration information of the target algorithms, wherein the grouping information comprises a plurality of algorithm group identifiers; Running multiple groups of target algorithms on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein the multiple target algorithms corresponding to the same grouping information and the multiple algorithm models corresponding to the multiple target algorithms are run on the same data processing device; Periodically trigger the automated detection platform to automatically detect the target algorithm deployed on the target device; The step of running the multiple sets of target algorithms on multiple data processing devices according to the grouping information of the multiple sets of target algorithms comprises: Generate multiple first configuration files according to the multiple algorithm group identifiers, and write startup commands of all target algorithms corresponding to the same algorithm group identifier into the first configuration files corresponding to the group of algorithm identifiers; configuring a data processing device for each of the plurality of first configuration files according to resource configuration information of the plurality of groups of target algorithms corresponding to the first configuration files; Starting a microservice container in a corresponding data processing device according to the first configuration file, and starting a model manager in the microservice container; Controlling the model manager to load the algorithm model corresponding to the group of target algorithms; The corresponding target algorithm is run in the corresponding microservice container; the algorithms and algorithm models corresponding to the same set of algorithm group identifiers are started in the same data processing device.

2. The algorithm operation method according to claim 1, characterized in that: Before obtaining the grouping information of the multiple target algorithms, the method further includes: Obtaining available resources of the plurality of data processing devices and resource consumption required to deploy any target algorithm; The multiple target algorithms are grouped in units of the data processing devices according to the available resources and the resource consumption; the same target algorithm and the algorithm model corresponding to the algorithm are divided into the same group of data processing devices, and the same group of data processing devices corresponds to at least one target algorithm and the algorithm model corresponding to the at least one target algorithm.

3. The algorithm operation method according to claim 2, characterized in that: The grouping of the plurality of target algorithms based on the data processing device according to the available resources and the resource consumption includes: The most commonly used set of algorithm models among the multiple target algorithms is used as the current algorithm model group, and a set of data processing devices is selected as the current data processing device; Adding the current algorithm model group to the current data processing device; Determine, based on the available resources of the current data processing device and the resource consumption of deploying all target algorithms corresponding to the current algorithm model group, whether the available resources of the current data processing device can accommodate the deployment of all target algorithms corresponding to the current algorithm model group; When it is determined that the current data processing device can accommodate and deploy all target algorithms corresponding to the current algorithm model group, all target algorithms corresponding to the current algorithm model group are added to the current data processing device; a group of algorithm models that are most commonly used among the multiple ungrouped target algorithms is used as the current algorithm model, and the current algorithm model group continues to be added to the current data processing device.

4. The algorithm operation method according to claim 3, characterized in that: When it is determined that the current data processing device cannot accommodate and deploy all target algorithms corresponding to the current algorithm model group, the method further includes: Add the target algorithms corresponding to the current algorithm model group that the current data processing device can accommodate to the current data processing device, add a new group of data processing devices as the current data processing device, add the algorithm models in the previous group of data processing devices to the current data processing device, and add the ungrouped target algorithms corresponding to the current algorithm model group to the current data processing device; take the most commonly used group of algorithm models among the multiple ungrouped target algorithms as the current algorithm model group, and continue to add the current algorithm model group to the current data processing device.

5. The algorithm operation method according to claim 4, characterized in that: After running the corresponding target algorithm in the corresponding microservice container, it also includes: outputting and saving the algorithm running results.

6. The algorithm operation method according to claim 1, characterized in that: The running of the corresponding target algorithm in the corresponding microservice container includes: running the corresponding target algorithm in the corresponding microservice container and calling an algorithm model required by the target algorithm.

7. The algorithm operation method according to claim 1, characterized in that: The algorithm deployment table also includes the algorithm code address and the algorithm running path; Before starting the corresponding target algorithm in the corresponding microservice container, the method further includes: obtaining the code of the target algorithm according to the algorithm code address; The starting of the corresponding target algorithm in the corresponding microservice container includes: running the corresponding target algorithm code in the corresponding microserver according to the algorithm running path.

8. The algorithm operation method according to claim 1, characterized in that: The algorithm deployment table also includes a test video stream address, an algorithm name, and a feedback test output address; After obtaining the algorithm deployment table, the method further includes: obtaining a video source file according to the test video stream address, pushing the video source file for the target algorithm test into a video stream through a preset push stream mirror, generating a pull stream address, and using the pull stream address to update the first configuration file of the corresponding target algorithm; the video stream address and the pull stream address include a video name, and the video name has a corresponding relationship with the corresponding algorithm name; After running the corresponding target algorithm in the corresponding microservice container, the method further includes: traversing the target algorithm that needs to test the video stream according to the algorithm deployment table, starting the test platform, starting the target algorithm that needs to test the video stream to perform a playback test according to the corresponding video stream address, waiting for a preset time, collecting test reports fed back by multiple target algorithms, and sending information that failed the test to the abnormal information feedback platform through the feedback test output address.

9. The algorithm operation method according to claim 1, characterized in that: The algorithm deployment table also includes algorithm model information; Before obtaining the grouping information of the multiple groups of target algorithms, the method further includes: converting the original algorithm model in the model warehouse into an open neural network exchange format, converting the open neural network exchange format to obtain a TensorRT model, and saving the TensorRT model to the model warehouse; and merging a portion of the network layers in the original algorithm model during the conversion to the TensorRT model; The controlling the model manager to load the algorithm model corresponding to the group of target algorithms includes: obtaining algorithm model information corresponding to the target algorithm, and controlling the model manager to load the TensorRT model corresponding to the algorithm model information from the model warehouse.

10. The algorithm operation method according to claim 1, characterized in that: After running the corresponding target algorithm in the corresponding microservice container, the method further includes: Test all target algorithms according to the service deployment table, and output and save the test results.

11. An algorithm operation device, characterized in that: Including acquisition module and operation module; The acquisition module is configured to acquire grouping information of multiple groups of target algorithms; the acquisition module is specifically configured to acquire an algorithm deployment table, the algorithm deployment table including the grouping information of the multiple groups of target algorithms and resource configuration information of the target algorithms, the grouping information including multiple algorithm group identifiers; The running module is configured to run multiple groups of target algorithms on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein, the multiple target algorithms corresponding to the same group of grouping information and the multiple algorithm models corresponding to the multiple target algorithms are run on the same data processing device; the running module is specifically configured to generate multiple first configuration files according to the multiple algorithm grouping identifiers, and write the startup commands of all target algorithms corresponding to the same algorithm grouping identifier into the first configuration file corresponding to the group of algorithm identifiers; according to the resource configuration information of the multiple groups of target algorithms corresponding to the first configuration files, configure a data processing device for each of the multiple first configuration files; according to the first configuration file, start a microservice container in the corresponding data processing device, and start a model manager in the microservice container; control the model manager to load the algorithm model corresponding to the group of target algorithms; run the corresponding target algorithm in the corresponding microservice container; the algorithm and algorithm model corresponding to the same group of algorithm grouping identifiers are started in the same data processing device; The operation module is also configured to periodically trigger the automated deployment platform to automatically deploy the target algorithm on the target device, and the data processing equipment is set in the target device; it is also configured to periodically trigger the automated detection platform to automatically detect the target algorithm deployed on the target device.

12. An algorithm running device, characterized in that: comprising a memory, a processor, and a computer program stored in the memory and executable on the processor to perform: Periodically triggering the automated deployment platform to automatically deploy the target algorithm to the target device, and setting the data processing equipment in the target device; Obtain grouping information of multiple target algorithms; The obtaining of grouping information of the plurality of target algorithms comprises: obtaining an algorithm deployment table, wherein the algorithm deployment table comprises grouping information of the plurality of target algorithms and resource configuration information of the target algorithms, wherein the grouping information comprises a plurality of algorithm group identifiers; Running multiple groups of target algorithms on multiple data processing devices according to the grouping information of the multiple groups of target algorithms; wherein the multiple target algorithms corresponding to the same grouping information and the multiple algorithm models corresponding to the multiple target algorithms are run on the same data processing device; Periodically trigger the automated detection platform to automatically detect the target algorithm deployed on the target device; The step of running the multiple sets of target algorithms on multiple data processing devices according to the grouping information of the multiple sets of target algorithms comprises: Generate multiple first configuration files according to the multiple algorithm group identifiers, and write startup commands of all target algorithms corresponding to the same algorithm group identifier into the first configuration files corresponding to the group of algorithm identifiers; configuring a data processing device for each of the plurality of first configuration files according to resource configuration information of the plurality of groups of target algorithms corresponding to the first configuration files; Starting a microservice container in a corresponding data processing device according to the first configuration file, and starting a model manager in the microservice container; Controlling the model manager to load the algorithm model corresponding to the group of target algorithms; The corresponding target algorithm is run in the corresponding microservice container; the algorithms and algorithm models corresponding to the same set of algorithm group identifiers are started in the same data processing device.

13. A non-transitory computer-readable storage medium, characterized in that The storage medium is used to store computer program instructions, wherein the computer program instructions can implement the algorithm operation method according to any one of claims 1 to 10 when executed.

Images