Integrated management system, method and application for pathology AI services

Abstract

The present invention proposes an integrated management system, method and application for pathological AI services, and the system includes communication, task scheduling, resource monitoring and container management modules. The method includes client requests, and the external communication module processes according to memory and other conditions; the task scheduling module creates and allocates tasks according to resources and request types; the internal communication module divides the transmission subgraph; the container management module controls the container; and the resource monitoring module manages resources. There are specific steps for subgraph division, and container management and resource monitoring have their own logic. The invention can monitor and manage multi-device and multi-container resources, improve resource utilization and system response speed, and solve the defects of existing pathological AI service deployment.

Description

Technical Field

[0001] The present invention relates to the fields of information systems, pathology AI and resource scheduling technology, and in particular to an integrated management system, method and application thereof for pathology AI services. Background Art

[0002] With the application of artificial intelligence technology in the medical field, AI-based auxiliary diagnosis technology has gradually emerged. Deploying AI services through containerization technology can achieve a certain degree of resource isolation and management. In the AI ​​service process, it generally includes three stages: data pre-processing, model reasoning, and result post-processing.

[0003] For pathology AI, its pre-processing is mainly to divide the entire pathology image into sub-images, and post-processing is to merge the results. However, there are differences between pathology AI and ordinary AI services. The input image size of pathology AI is large, and the size can reach hundreds of thousands of pixels by hundreds of thousands of pixels at high resolution. It will be regarded as more than 10,000 sub-images, which leads to a long pre- and post-processing time. And the same pathology image may be input into different AI services.

[0004] There are many disadvantages in the common AI service container deployment method. On the one hand, the common AI service container deployment will deploy data pre- and post-processing and model reasoning together in the container and allocate resources. This method requires the allocation of fixed resources to ensure that the three steps can be carried out continuously. Due to the long pre- and post-processing time of pathology AI, this method reduces the utilization efficiency of system resources. On the other hand, there are situations where different AI services infer a pathology image, which will lead to repeated pre-processing.

[0005] In addition, in the application scenario of pathological images, in addition to servers that can be used to deploy AI services, the host that is paired with the pathological slice scanner can also be used for AI deployment. In this case, it is difficult for ordinary AI deployment methods to efficiently assign tasks to devices with different hardware resources.

[0006] In summary, the existing technology has shortcomings in the integrated management of pathology AI services and cannot meet the needs of efficient utilization of system resources and reasonable allocation of tasks. Therefore, an integrated management system for pathology AI services is needed to solve these problems. Summary of the invention

[0007] The embodiments of the present invention provide an integrated management system, method and application for pathological AI services, which address the defects of current technologies such as low resource utilization efficiency and difficulty in task allocation, including unreasonable resource allocation, repeated pre-processing and difficulty in adapting to different hardware devices.

[0008] The core technology of the present invention is mainly an integrated management system and method for pathology AI services. Through modules such as communication, task scheduling, resource monitoring and container management, pathology AI service requests and multi-device and multi-container resources are monitored and managed. Specific sub-graph partitioning, task allocation, resource allocation and container control logic are adopted to improve resource utilization and system response speed.

[0009] In a first aspect, the present invention provides an integrated management method for pathology AI services, the method comprising the following steps:

[0010] The client makes a request to the external communication module, which accepts or rejects the request according to the situation. If accepted, it notifies the client to transmit data to the specified device;

[0011] The task scheduling module creates tasks and assigns them to different devices based on system resources and request types. Pre- and post-processing tasks are assigned to the data processing container of the specified device, and AI reasoning tasks are assigned to the corresponding device. The request types include real-time / non-real-time, single service / multi-service, partial processing / partial reasoning. The request information sent by the client is recorded at the same time, and duplicate requests are filtered. The pre-processing is to cut the pathological image into pieces, and the post-processing is to merge the results.

[0012] The internal communication module divides the subgraph and transmits it to the device selected by the task scheduling module;

[0013] The container management module controls the opening, termination and resource release of containers;

[0014] The resource monitoring module monitors hardware resources and container status, and allocates resources to containers according to policies.

[0015] Furthermore, the request information initiated by the client includes the IP, request number i, reservation number, priority, service type and image size; the external communication module responds to the client's request and responds to the client according to the priority and system memory status through the task scheduling module;

[0016] If the memory is sufficient, the client is notified to start transmitting data to the specified device;

[0017] If the memory is insufficient, for real-time requests, the client waits for the first preset seconds and then changes the reservation number to request again. If it is still insufficient, the resource monitoring module calculates and allocates the relevant time to request again. For non-real-time requests, the client makes a second request after the second preset minute after the first wait. If there is still waiting, the client makes a third request after the third preset minute. If there is no memory for the third time, the information is recorded. When the real-time task accounts for the fourth preset percentage, a non-real-time task is created according to the reservation time.

[0018] Furthermore, the specific steps of subgraph partitioning are:

[0019] The original pathological image is cut into W times H sub-images, and the number of distributed devices is N, where W>H, and W represents rows and H represents columns;

[0020] First, all sub-graphs are divided into multiple regions every N H columns according to the number of devices N. If there are less than N columns, the remaining sub-graphs are divided into one region, and sub-graphs are divided again in each region.

[0021] The task scheduling module calculates the number of subgraphs in the sets distributed to different devices, and distributes subgraphs to different sets in each area by calculating the proportion of subgraphs in each set to all subgraphs.

[0022] The first area is sorted from small to large by the proportion of sub-graphs, and is recorded as {a1,a2...a N}, continuously allocate sub-graphs to corresponding sets in proportion, and the excess ones are evenly allocated to the set with the least number of sub-graphs. For different columns, continuous is defined as the last sub-graph of the previous column is connected to the first sub-graph of the next column. When arriving at the next area, the first one in the sorting is moved to the last one for allocation.

[0023] Furthermore, the control logic of the container management module includes:

[0024] When starting a container, first consider whether it is a real-time task;

[0025] When a new container is opened for a real-time task, if the container has opened other containers for processing real-time tasks, part of the resources of the opened container will be allocated to the newly opened container; if other containers in the device process non-real-time tasks, they will stop and release dedicated resources after other containers finish processing the current tasks, and then allocate the remaining resources to the newly opened container;

[0026] When the system opens a new container for a non-real-time task, there will be no container in the device to process the real-time task, and resources will be evenly distributed to all containers.

[0027] Furthermore, the control logic of the container management module also includes:

[0028] In the same device, when a container completes all real-time tasks, the dedicated video memory is released to other containers processing real-time tasks.

[0029] When all real-time tasks are processed, the video memory resources are evenly distributed to the containers that still have non-real-time tasks;

[0030] When a container has been idle for a number of minutes, all resources for the container are released and the container is shut down.

[0031] Furthermore, the control logic of the resource monitoring module includes:

[0032] Track all AI service containers on all devices, record the number of containers on each device, the amount of resources used by each container, and the operation status of the AI ​​service containers;

[0033] For pathological AI, memory resources and video memory resources are managed separately. For memory, a device opens a fixed data processing container and will not be terminated. For video memory, it is divided into fixed video memory, dedicated video memory and shared video memory.

[0034] Furthermore, the control logic of the resource monitoring module also includes:

[0035] Predict and adjust dedicated video memory. Predict the video memory situation for the next inference based on the video memory situation of the previous batch and adjust the size of dedicated video memory.

[0036] In a second aspect, the present invention provides an integrated management system for pathology AI services, including:

[0037] The communication module includes an external communication module and an internal communication module. The external communication module is used to establish communication between the client and the system, inform the target device of data transmission, and respond to the client according to the task priority and memory status. The internal communication module is responsible for controlling the data transmission across devices, and evenly divides the sub-images of the same pathological image into different sets and transmits them.

[0038] The task scheduling module creates tasks and assigns them to different devices based on system resources and request types. Pre- and post-processing tasks are assigned to the data processing container of the specified device, and AI reasoning tasks are assigned to the corresponding device. The request types include real-time / non-real-time, single service / multi-service, partial processing / partial reasoning. The request information sent by the client is recorded at the same time, and duplicate requests are filtered. The pre-processing is to cut the pathological image into pieces, and the post-processing is to merge the results.

[0039] Resource monitoring module, which monitors hardware resources and container status, and allocates resources to containers according to policies;

[0040] Container management module, the container management module controls the opening, termination and resource release of the container.

[0041] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned integrated management method for pathology AI services.

[0042] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process, and the process includes the integrated management method for pathology AI services according to the above-mentioned method.

[0043] The main contributions and innovations of the present invention are as follows:

[0044] 1. Resource Utilization

[0045] (1) Improve system resource utilization efficiency

[0046] The prior art deploys data pre- and post-processing and model reasoning together in a container and allocates fixed resources. The long pre- and post-processing time of pathology AI leads to low resource utilization efficiency. The present invention separates the pre- and post-processing of pathology image data from model reasoning, and deploys the pre- and post-processing of data in the same container. It can flexibly allocate resources according to resource conditions and request types, thereby improving the utilization efficiency of system resources.

[0047] (2) Reduce hardware resource usage

[0048] The present invention can monitor all containers, hardware resources and task requests, select real-time allocation strategies according to resource conditions and request types, control the application and release of container resources and the allocation of task priorities, and effectively reduce the occupation of hardware resources by idle AI services.

[0049] 2. Task processing

[0050] (1) Avoid repeated pre-processing

[0051] The prior art will repeat pre-processing when different AI services infer a pathological image. The present invention uniformly pre-processes the pathological image, then opens the container and distributes the task according to the resource situation, request type and allocation strategy, thus avoiding repeated pre-processing.

[0052] (2) Efficient task allocation

[0053] For devices with different hardware resources in pathological image application scenarios (such as servers and hosts equipped with pathological slice scanners), it is difficult for existing technologies to efficiently allocate tasks. The task scheduling module and container management module of the present invention can reasonably allocate tasks to different devices according to system resource conditions and request types, thereby improving the efficiency of task allocation.

[0054] 3. System performance

[0055] (1) Improve system response speed

[0056] Through the above-mentioned optimization of resource utilization and task processing, the present invention can improve the response speed of the system and better meet the needs of pathological AI services.

[0057] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0059] Figure 1 is a framework diagram of an integrated management system for pathology AI services according to an embodiment of the present invention;

[0060] Figure 2 is a flow chart of an integrated management method for pathology AI services according to an embodiment of the present invention;

[0061] Figure 3 is a schematic diagram of subgraph division according to an embodiment of the present invention;

[0062] Figure 4 is a flowchart of graph container management according to an embodiment of the present invention;

[0063] Figure 5 is a flowchart of a system processing a client request according to an embodiment of the present invention;

[0064] Figure 6 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0066] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0067] Embodiment 1

[0068] The present invention aims to propose an integrated management method for pathological AI services, specifically, referring to Figure 1-Figure 2 , the method comprising:

[0069] The client makes a request to the external communication module, which accepts or rejects the request according to the situation. If accepted, it notifies the client to transmit data to the specified device;

[0070] The task scheduling module creates tasks and assigns them to different devices based on system resources and request types. Pre- and post-processing tasks are assigned to the data processing container of the specified device, and AI reasoning tasks are assigned to the corresponding device. The request types include real-time / non-real-time, single service / multi-service, partial processing / partial reasoning. The request information sent by the client is recorded at the same time, and duplicate requests are filtered. The pre-processing is to cut the pathological image into pieces, and the post-processing is to merge the results.

[0071] The internal communication module divides the subgraph and transmits it to the device selected by the task scheduling module;

[0072] The container management module controls the opening, termination and resource release of containers;

[0073] The resource monitoring module monitors hardware resources and container status, and allocates resources to containers according to policies.

[0074] In this embodiment, the logic of client request processing and external communication module processing is as follows:

[0075] 1. Request initiation

[0076] The client makes a request to the external communication module. The request message contains information such as IP, request number i, reservation number (initial -1), priority (real-time request or non-real-time request), service type, image size, etc.

[0077] 2. External communication module processing

[0078] The external communication module responds to the client based on task priority and system memory status.

[0079] If the memory is sufficient, notify the client to start transferring data to the specified device.

[0080] If the memory is insufficient, for real-time requests, the client waits for 5 seconds and then requests again (the reservation number is changed to -2). If it is still insufficient, the resource monitoring module calculates the time to release enough memory after the AI ​​service is completed, allocates a reservation number and waiting time to notify the client to request again; for non-real-time requests, when the first request returns to waiting, the client waits for 10 minutes before making a second request. If it still returns to waiting, it waits for 20 minutes before making a third request. If the third request is still insufficient, the system records the request information. When real-time tasks account for 20% of all tasks in all containers, non-real-time tasks are created in order of reservation time.

[0081] In this embodiment, the logic of task scheduling is as follows:

[0082] 1. Task creation and analysis

[0083] The task scheduling module creates tasks based on system resource conditions and request types (real-time / non-real-time, single service / multi-service, processing-oriented / reasoning-oriented).

[0084] Record the request information sent by the client (IP address, request number, request service type and pathological image size) and filter duplicate requests.

[0085] 2. Task Allocation

[0086] Pre- and post-processing tasks are assigned to the data processing container of the specified device (pre-processing cuts the pathology image into pieces).

[0087] According to the calculated new task start time t (considering the container running status, whether it is idle or not, whether it is started, etc.), the real-time tasks are sorted in ascending order to obtain the container idle time sequence {t n}, calculate the time T required for the current real-time task, and find x so that , containers 0 to x-1 are the selected containers to which AI inference tasks are assigned.

[0088] Calculate the number of sub-images in each container. Non-real-time requests for a sub-image of a pathological image are passed to the device as a sub-image set. Real-time tasks need to be assigned to different devices. The number of sub-images is calculated as follows:

[0089]

[0090] Among them, N i represents the number of subgraphs in the subgraph set distributed by the i-th container, α represents the time required for the AI ​​service to process a subgraph, and N i The upper limit is 5. This prevents the sub-images of subsequent pathological sections from being allocated to all devices when the task completion time of each device is similar, which increases the time consumption of processing continuous real-time tasks. 5 is an empirical value. Setting too much will result in outputting a small number of sub-images to multiple devices, which will take too long to transmit. Setting too little will reduce real-time performance.

[0091] In this embodiment, the logic of internal communication is as follows:

[0092] The internal communication module is responsible for dividing the sub-graphs of the same pathological image into different sets and transmitting them to other devices. The hardware resources and time required for the AI ​​model to infer similar images are similar. For example, images with more cells require more video memory during the inference process, and the inference time is also longer. If divided into different sets in the order of the sub-graph positions, some sets will contain images with more cells, while other sets will contain images with fewer cells, resulting in a large difference in resource requirements and time consumption for subsequent AI services. In order to accurately assess resource requirements and assign tasks, sub-graphs need to be evenly divided into different sets. Although randomly dividing sub-graphs can ensure balance, sub-graph management will become complicated, and the time consumption will increase when the results are merged. The following is the sub-graph division method of the present invention:

[0093] The original pathological image is cut into W times H sub-images (W>H), and the number of distributed devices is N. First, the sub-image is divided into multiple regions every N H, and the sub-image is divided in each region. When there are less than N columns, the remaining ones are divided into one region.

[0094] The number of subgraphs in the sets distributed to different devices is calculated by the task scheduling module, which calculates the proportion of subgraphs in each set to all subgraphs, and allocates subgraphs to different sets in each area.

[0095] The first area is sorted from small to large by the proportion of sub-graphs, and is recorded as {a1,a2...a N}, continuously allocate sub-graphs to the corresponding set in proportion, that is, the number of sub-graphs in the current region multiplied by a i And round up. The excess is evenly distributed to the set with the least number of sub-graphs. For different columns, continuity is defined as the last sub-graph of the previous column and the first sub-graph of the next column are connected.

[0096] Each time we reach the next region, we move the first one in the sorting to the end, and the second region is {a2,a3...a n ,a1}, and then assign subgraphs consecutively.

[0097] This method transmits multiple continuous subgraphs, which takes less time to process than random allocation. It can ensure that subgraphs are evenly distributed to different sets. When the final results are merged, they can also be processed in continuous memory, which reduces the time required for merging. Figure 3 This is an example of partitioning into three devices.

[0098] In this embodiment, the logic of resource monitoring is as follows:

[0099] The resource monitoring module tracks all AI service containers on all devices, records the number of containers on each device, the amount of resources used by each container, and the operation status of the AI ​​service container, and allocates resources for the container based on the above information. Since the pre- and post-processing of pathological images is handled by the data processing container and does not require a GPU, while the AI ​​service container is mainly AI model inference, the system can manage memory resources and video memory resources for the two containers separately. For memory, a device will open a fixed data processing container and will not terminate. For video memory, the system will divide it according to the use of video memory. The size of the loaded model is fixed, so the model will occupy a fixed video memory. During the inference process, due to the difference in model input data, the video memory required for inference will vary within a certain range. At the same time, multiple containers can use the same video memory, so it can be divided into dedicated video memory and shared video memory. The fixed video memory of the container is the video memory for loading the model. The dedicated video memory can only be used by the container to which it belongs, and the common video memory can be used by all containers. The fixed video memory size is determined by the model, and the dedicated video memory size will be set to an initial setting and will be adjusted adaptively.

[0100] The specific method of video memory allocation is as follows:

[0101] Set the model input image size S and number BatchSize and perform multiple inferences, record the maximum value of video memory usage in each inference, sort the maximum value in descending order, and then calculate the video memory A required when the input image size is S = (memory usage size sorted in the 10% / BatchSize). Then the size of the container's dedicated video memory is set to BatchSize*A. After allocating fixed video memory and dedicated video memory, the remaining video memory is shared video memory. During runtime, if there is insufficient shared video memory, the BatchSize of the corresponding container will be reduced (insufficient video memory size / A). At the same time, the resource monitoring module will adjust the video memory allocation for other devices with the same situation, where the same situation means that the type and number of containers opened are the same, and the input image size and number are also the same. The module will also record the adjusted video memory allocation ratio and use it as the initial resource configuration of the new container.

[0102] Predict and adjust dedicated video memory: Pathological images will be cut into pieces and processed in different batches. Since the sub-image partitioning set takes into account the balance, the video memory usage of different batches of model inference is similar. Therefore, the system can predict the video memory usage of the next inference based on the video memory usage of the previous batch and adjust the dedicated video memory size. In addition, sub-images of the same pathological image may be inferred on different devices, so the predicted and adjusted dedicated video memory size D t+1 The definition is as follows:

[0103]

[0104] Among them, D t+1is the size of the dedicated video memory after adjustment, B is the BatchSize set for the current device, is the memory usage of the i-th device in the first t training batches, B i BatchSize is set for the ith device. Avg represents the average memory required for each sub-image on all devices, and σ represents the standard deviation of the memory usage of each training batch. The mean plus the standard deviation can represent most cases of memory changes, and it is multiplied by B to get the size of the dedicated memory. Therefore, D t+1 Can meet the memory requirements of AI model reasoning. When the memory requirements exceed D t+1 When , the probability of exceeding is small, and the use of shared video memory by different containers will not conflict. By predicting the video memory usage and adjusting the video memory allocation, the video memory resource conflict can be effectively reduced. This method is only applicable to the scenario where each sub-image is similar, that is, the sub-images divided by the pathological image.

[0105] In this embodiment, the logic of resource monitoring is as follows:

[0106] The container management module is responsible for opening, stopping, allocating resources, and closing AI service containers. This module configures opened and newly opened containers based on the memory resources of the current device, the resources occupied by opened containers in each device, and the number of unopened containers. The resource monitoring module divides the fixed memory, dedicated memory, and shared memory of all devices in a unified way, and the container management module allocates resources to containers on the same device on this basis.

[0107] like Figure 4 In the system, when a container is opened, the first consideration is whether it is a real-time task. For opening a new container for a real-time task, when the container has opened other containers that process real-time tasks, the module will allocate part of the resources of the opened container to the newly opened container so that the container can get equal resources. When other containers in the device process non-real-time tasks, they will stop and release dedicated resources after other containers process the current tasks, and then allocate the remaining resources to the newly opened container. When the system opens a new container for a non-real-time task, there will be no container processing real-time tasks in the device, so the resources will be evenly distributed to all containers. In the same device, when a container completes all real-time tasks, the dedicated video memory will be released to other containers that process real-time tasks. When all real-time tasks are processed, the video memory resources will be evenly distributed to the containers that still have non-real-time tasks. Because the time it takes to open a container is relatively short, when the container is idle for 10 minutes, all the resources of the container are released and the container is closed.

[0108] Secondly, consider whether it is a multi-service request. Multi-service requests require the use of multiple AI services, starting the AI ​​service containers in turn and performing reasoning, and then transmitting the results back to the initial device, which performs data post-processing and result merging. Since AI reasoning and data post-processing require different hardware resources, and data post-processing comes after AI reasoning, the completion time of multi-service requests is determined by data post-processing. Prioritizing AI services with longer post-processing times can complete the task fastest, where the processing time is determined by the time complexity of the corresponding algorithm. Therefore, the AI ​​service container corresponding to the task that takes a long time to post-process is started first, and the next container is started before the previous AI service ends.

[0109] Embodiment 2

[0110] Based on the same concept, Figure 1 As shown, the present invention also proposes an integrated management system for pathological AI services, including:

[0111] The communication module includes an external communication module and an internal communication module. The external communication module is used to establish communication between the client and the system, inform the target device of data transmission, and respond to the client according to the task priority and memory status. The internal communication module is responsible for controlling the data transmission across devices, and uses a specific method to evenly divide the sub-images of the same pathological image into different sets and transmit them.

[0112] The task scheduling module records and analyzes client requests, filters duplicate requests, creates tasks and allocates containers based on request priority. It determines the task allocation method based on the request type (real-time / non-real-time, single service / multiple services, partial processing / partial reasoning), calculates the start time of the new task and selects the corresponding container, and calculates the number of subgraphs in each container.

[0113] The resource monitoring module tracks the AI ​​service containers on all devices and manages memory resources and video memory resources separately. According to the different needs of data processing containers and AI service containers, the memory and video memory are reasonably allocated, including the division and adjustment of fixed video memory, dedicated video memory and shared video memory.

[0114] The container management module is responsible for starting, stopping, allocating resources, and shutting down AI service containers. It allocates resources reasonably based on whether it is a real-time task, whether there are other containers processing real-time tasks, and multiple service requests, including the allocation and adjustment of video memory, as well as the start and stop strategies of containers.

[0115] In this embodiment, if Figure 5As shown in the figure, the process of client request processing in this system is as follows: First, after the pathology AI integrated management system is started, it will continue to listen to messages from the doctor client. When receiving an analysis request, the system will determine whether the resources are sufficient. If not, the client will be asked to wait. If sufficient, it will further determine whether it is a repeated request. If it is a repeated request, the request will be discarded. For non-repeated requests, the system will record the request, create a task, start a container, allocate a container, and then the allocated container will process the task corresponding to the request. Finally, the system will merge the results of the AI ​​service container and return it to the client.

[0116] Embodiment 3

[0117] This embodiment also provides an electronic device, referring to Figure 6 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0118] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0119] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In corresponding cases, the memory 404 may include a removable or non-removable (or fixed) medium. In corresponding cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). In corresponding cases, the ROM can be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, referred to as PROM), an erasable PROM (Erasable Programmable Read-Only Memory, referred to as EPROM), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), an electrically alterable ROM (Electrically Alterable Read-Only Memory, referred to as EAROM) or a flash memory (FLASH) or a combination of two or more of these. In corresponding cases, the RAM can be a static random access memory (Static Random-Access Memory, referred to as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, referred to as DRAM), wherein the DRAM can be a fast page mode dynamic random access memory 404 (Fast Page Mode Dynamic Random Access Memory, referred to as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, referred to as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, referred to as SDRAM), etc.

[0120] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0121] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the integrated management methods for pathology AI services in the above embodiments.

[0122] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0123] The transmission device 406 can be used to receive or send data via a network. The specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0124] Input / output devices 408 are used to input or output information.

[0125] Embodiment 4

[0126] This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes an integrated management method for pathology AI services according to embodiment one.

[0127] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0128] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0129] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 2 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0130] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. An integrated management method for pathology AI services, characterized in that: The following steps are involved: The client makes a request to the external communication module, which accepts or rejects the request according to the situation. If accepted, it notifies the client to transmit data to the specified device; The task scheduling module creates tasks and assigns them to different devices based on system resources and request types. Pre- and post-processing tasks are assigned to the data processing container of the specified device, and AI reasoning tasks are assigned to the corresponding device. The request types include real-time / non-real-time, single service / multi-service, partial processing / partial reasoning. The request information sent by the client is recorded at the same time, and duplicate requests are filtered. The pre-processing is to cut the pathological image into pieces, and the post-processing is to merge the results. The internal communication module divides the subgraph and transmits it to the device selected by the task scheduling module; The container management module controls the opening, termination and resource release of containers; The resource monitoring module monitors hardware resources and container conditions, and allocates resources to containers based on policies; The request information initiated by the client includes IP, request number i, reservation number, priority, service type and image size; the external communication module responds to the request of the client through the task scheduling module according to the priority and system memory status, and the request information is divided into real-time request and non-real-time request according to the priority; If the memory is sufficient, the client is notified to start transmitting data to the specified device; If the memory is insufficient, for real-time requests, the client waits for the first preset seconds and then changes the reservation number to request again. If it is still insufficient, the resource monitoring module calculates and allocates relevant time to request again. For non-real-time requests, when the first request returns to wait, the client waits for the second preset minute to make the second request. If it still returns to wait, it waits for the third preset minute to make the third request. If the memory is still insufficient for the third request, the information is recorded. When the real-time tasks account for the fourth preset percentage, the non-real-time tasks are created according to the appointment time. The specific steps of sub-graph division of the internal communication module are as follows: The original pathological image is cut into W times H sub-images, and the number of distributed devices is N, where W>H, and W represents rows and H represents columns; First, all sub-graphs are divided into multiple regions every N H columns according to the number of devices N. If there are less than N columns, the remaining sub-graphs are divided into one region, and sub-graphs are divided again in each region. The task scheduling module calculates the number of subgraphs in the sets distributed to different devices, and distributes subgraphs to different sets in each area by calculating the proportion of subgraphs in each set to all subgraphs. The first area is sorted from small to large by the proportion of sub-graphs, and is recorded as {a1,a2...a N }, continuously allocate sub-graphs to corresponding sets in proportion, and evenly allocate the excess sub-graphs to the set with the least number of sub-graphs. For different columns, continuous is defined as the last sub-graph of the previous column is connected to the first sub-graph of the next column. When arriving at the next area, move the first one in the sorting to the last one for allocation; The control logic of the container management module includes: When starting a container, first consider whether it is a real-time task; When a new container is opened for a real-time task, if the container has opened other containers for processing real-time tasks, part of the resources of the opened container will be allocated to the newly opened container; if other containers in the device process non-real-time tasks, they will stop and release dedicated resources after other containers finish processing the current tasks, and then allocate the remaining resources to the newly opened container; When the system opens a new container for a non-real-time task, there will be no container in the device to process the real-time task, and resources will be evenly distributed to all containers.

2. An integrated management method for pathology AI services as claimed in claim 1, characterized in that: The control logic of the container management module also includes: In the same device, when a container completes all real-time tasks, the dedicated video memory is released to other containers processing real-time tasks. When all real-time tasks are processed, the video memory resources are evenly distributed to the containers that still have non-real-time tasks; When a container has been idle for a number of minutes, all resources for the container are released and the container is shut down.

3. An integrated management method for pathology AI services as described in claim 1 or 2, characterized in that: The control logic of the resource monitoring module includes: Track all AI service containers on all devices, record the number of containers on each device, the amount of resources used by each container, and the operation status of the AI ​​service containers; For pathological AI, memory resources and video memory resources are managed separately. For memory, a device opens a fixed data processing container and will not be terminated. For video memory, it is divided into fixed video memory, dedicated video memory and shared video memory.

4. An integrated management method for pathology AI services as claimed in claim 3, characterized in that: The control logic of the resource monitoring module also includes: Predict and adjust dedicated video memory. Predict the video memory situation for the next inference based on the video memory situation of the previous batch and adjust the size of dedicated video memory.

5. An integrated management system for pathology AI services, characterized in that: include: The communication module includes an external communication module and an internal communication module. The external communication module is used to establish communication between the client and the system, inform the target device of data transmission, and respond to the client according to the task priority and memory status. The internal communication module is responsible for controlling the data transmission across devices, and evenly divides the sub-images of the same pathological image into different sets and transmits them. The task scheduling module creates tasks and assigns them to different devices based on system resources and request types. Pre- and post-processing tasks are assigned to the data processing container of the specified device, and AI reasoning tasks are assigned to the corresponding device. The request types include real-time / non-real-time, single service / multi-service, partial processing / partial reasoning. The request information sent by the client is recorded at the same time, and duplicate requests are filtered. The pre-processing is to cut the pathological image into pieces, and the post-processing is to merge the results. Resource monitoring module, which monitors hardware resources and container status, and allocates resources to containers according to policies; Container management module, which controls the opening, termination and resource release of containers; The request information initiated by the client includes IP, request number i, reservation number, priority, service type and image size; the external communication module responds to the request of the client through the task scheduling module according to the priority and system memory status, and the request information is divided into real-time request and non-real-time request according to the priority; If the memory is sufficient, the client is notified to start transmitting data to the specified device; If the memory is insufficient, for real-time requests, the client waits for the first preset seconds and then changes the reservation number to request again. If it is still insufficient, the resource monitoring module calculates and allocates relevant time to request again. For non-real-time requests, when the first request returns to wait, the client waits for the second preset minute to make the second request. If it still returns to wait, it waits for the third preset minute to make the third request. If the memory is still insufficient for the third request, the information is recorded. When the real-time tasks account for the fourth preset percentage, the non-real-time tasks are created according to the appointment time. The specific steps of sub-graph division of the internal communication module are as follows: The original pathological image is cut into W times H sub-images, and the number of distributed devices is N, where W>H, and W represents rows and H represents columns; First, all sub-graphs are divided into multiple regions every N H columns according to the number of devices N. If there are less than N columns, the remaining sub-graphs are divided into one region, and sub-graphs are divided again in each region. The task scheduling module calculates the number of subgraphs in the sets distributed to different devices, and distributes subgraphs to different sets in each area by calculating the proportion of subgraphs in each set to all subgraphs. The first area is sorted from small to large by the proportion of sub-graphs, and is recorded as {a1,a2...a N }, continuously allocate sub-graphs to corresponding sets in proportion, and evenly allocate the excess sub-graphs to the set with the least number of sub-graphs. For different columns, continuous is defined as the last sub-graph of the previous column is connected to the first sub-graph of the next column. When arriving at the next area, move the first one in the sorting to the last one for allocation; The control logic of the container management module includes: When starting a container, first consider whether it is a real-time task; When a new container is opened for a real-time task, if the container has opened other containers for processing real-time tasks, part of the resources of the opened container will be allocated to the newly opened container; if other containers in the device process non-real-time tasks, they will stop and release dedicated resources after other containers finish processing the current tasks, and then allocate the remaining resources to the newly opened container; When the system opens a new container for a non-real-time task, there will be no container in the device to process the real-time task, and resources will be evenly distributed to all containers.

6. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the integrated management method for pathology AI services according to any one of claims 1 to 4.

7. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes the integrated management method for pathology AI services according to any one of claims 1 to 4.

Images