Method and device for dynamic testing of pathology AI analysis system

Abstract

This application proposes a method and apparatus for dynamically testing a pathology AI analysis system, comprising the following steps: constructing a policy pool with multiple pre-set test request policies; setting a policy selection algorithm that selects the optimal test request policy from the policy pool based on task requirements and real-time system status; executing the optimal test request policy to test the pathology AI analysis system, obtaining test results, and optimizing the policy selection algorithm based on the test results. This solution uses reinforcement learning to dynamically find the most appropriate test request policy and test the pathology analysis system, thereby helping users quickly identify system performance bottlenecks.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for dynamically testing a pathology AI analysis system. Background Art

[0002] With the rapid development of artificial intelligence, AI services have been widely penetrated into many industries and fields. Pathology AI services mainly assist doctors in diagnosing diseases more accurately through intelligent analysis of pathology images, such as identifying cancer cell characteristics and determining whether tumors are benign or malignant. However, the development of pathology AI is constrained by its sensitivity and there is still much room for improvement. Pathology AI systems have extremely high requirements for accuracy and stability, because their analysis results are directly related to the accuracy of medical diagnosis, which in turn affects patients' treatment plans and health status.

[0003] In order to ensure that the pathology AI system can operate stably and reliably in complex and changeable actual application scenarios, testing of the pathology AI system is crucial. An effective testing system can not only help developers conduct comprehensive evaluations of different AI modules and optimize their performance, but also ensure the quality and reliability of AI services in actual use. However, traditional testing methods rely on a lot of manual intervention, and the testing efficiency is low, making it difficult to meet the needs of large-scale and high-efficiency testing. In addition, existing technologies lack a feedback mechanism for real-time monitoring of the testing process, and cannot dynamically adjust the testing strategy according to actual conditions to optimize the test results.

[0004] Because pathology AI services can reach hundreds of thousands by hundreds of thousands of pixels in size at high resolution and are greatly affected by multiple factors such as data, algorithms, and hardware, a single fixed request strategy is simply unable to meet the diverse needs in complex scenarios. Therefore, how to make full use of existing hardware resources, simulate dynamically changing request strategies, and thus provide higher-quality pathology AI services is a key issue that needs to be urgently addressed. Summary of the Invention

[0005] The embodiments of the present application provide a method and device for dynamically testing a pathology AI analysis system, which uses reinforcement learning to dynamically find the most appropriate test request strategy and test the pathology analysis system, thereby helping users quickly identify system performance bottlenecks.

[0006] In a first aspect, an embodiment of the present application provides a method for dynamically testing a pathology AI analysis system, the method comprising:

[0007] Build a policy pool with multiple test request policies preset;

[0008] Setting a strategy selection algorithm, wherein the strategy selection algorithm selects an optimal test request strategy from the strategy pool based on task requirements and real-time system status, wherein the expected benefit of each test request strategy in the strategy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption is calculated, and the test request strategy with the highest expected benefit is selected as the optimal test request strategy;

[0009] Execute the optimal test request strategy to test the pathology AI analysis system to obtain test results, and optimize the strategy selection algorithm based on the test results.

[0010] In a second aspect, an embodiment of the present application provides a device for dynamically testing a pathology AI analysis system, comprising:

[0011] A request simulation module is used to build a strategy pool with multiple test request strategies preset, and set a strategy selection algorithm. The strategy selection algorithm selects the optimal test request strategy from the strategy pool based on task requirements and real-time system status. The expected benefits of each test request strategy in the strategy pool under the three optimization goals of analysis accuracy, analysis time, and analysis resource consumption are calculated, and the test request strategy with the highest expected benefit is selected as the optimal test request strategy. The optimal test request strategy is executed to test the pathology AI analysis system to obtain test results, and the strategy selection algorithm is optimized based on the test results.

[0012] The monitoring module is used to monitor the system status of the pathology AI analysis system and the analysis accuracy, analysis time and analysis resource consumption during the pathology analysis process in real time;

[0013] The result recording module is used to record and store the information data monitored by the monitoring module;

[0014] The result display module is used to display the test results of the pathology AI analysis system to users;

[0015] The communication module is used to transmit information between the request simulation module, the monitoring module, the result recording module and the result display module.

[0016] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform a method for dynamically testing a pathology AI analysis system.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. When the program code is executed by a processor, a method for dynamically testing a pathology AI analysis system is implemented.

[0018] The main contributions and innovations of the present invention are as follows:

[0019] The embodiment of the present application constructs a strategy pool containing a variety of types, traversal methods and frequency distribution combinations to simulate the random, sequential, timed and other diverse request scenarios of the pathology AI system in single / multiple pathology slice analysis scenarios, covering different request frequency scenarios such as low peak in remote areas, daily outpatient cycle fluctuations, and batch uploads in hospitals; based on task requirements (high / normal / low concurrency) and real-time system status (GPU / CPU memory occupancy, task queue length), the optimal strategy is selected by weighted calculation of the expected benefits of the three objectives of analysis accuracy, time and resource consumption, and the test results are used to construct a reward value dynamic optimization strategy selection algorithm to achieve adaptive adjustment of the test strategy, and always keep the most appropriate test request strategy to test the pathology AI analysis system.

[0020] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 is a flow chart of a method for dynamically testing a pathology AI analysis system according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of a method for constructing a strategy pool according to the present application;

[0024] Figure 3 This is a structural block diagram of a device for dynamically testing a pathology AI analysis system according to an embodiment of the present application;

[0025] Figure 4 Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0027] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0028] Example 1

[0029] The embodiment of the present application provides a method for dynamically testing a pathology AI analysis system, which uses reinforcement learning to dynamically find the most appropriate test request strategy and test the pathology analysis system, thereby helping users quickly identify the performance bottleneck of the system. Specifically, refer to Figure 1 , the method comprising:

[0030] Build a policy pool with multiple test request policies preset;

[0031] Setting a strategy selection algorithm, wherein the strategy selection algorithm selects an optimal test request strategy from the strategy pool based on task requirements and real-time system status, wherein the expected benefit of each test request strategy in the strategy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption is calculated, and the test request strategy with the highest expected benefit is selected as the optimal test request strategy;

[0032] Execute the optimal test request strategy to test the pathology AI analysis system to obtain test results, and optimize the strategy selection algorithm based on the test results.

[0033] In some embodiments, the strategy pool is constructed as follows: Figure 2 As shown, the types of test request strategies preset in the strategy pool meet all task requirements of actual applications.

[0034] Further, according to Figure 2It can be seen that this solution pre-selects the types analyzed by the pathology AI system when constructing the strategy pool, including single type and multiple types. The single type is used to simulate the situation where the pathology AI system only analyzes one type of pathology section, and the multiple types are used to simulate the situation where the pathology AI system analyzes multiple types of pathology sections. For example, when the pathology AI analysis system of the test only analyzes one type of pathology section, Ki67, a single type of strategy is adopted. When the pathology AI analysis system of the test analyzes two types of pathology sections, Ki67 and Stomach, a multiple type of strategy is adopted. Different traversal methods are included under single type and multiple types, and the types of traversal methods include unordered traversal, ordered serial traversal, and ordered polling traversal. , unordered traversal is used to simulate the scenario where the pathology AI analysis system performs random analysis on pathology sections, ordered serial traversal is used to simulate the scenario where pathology sections are analyzed sequentially, and polling traversal is used to simulate the scenario where pathology sections are analyzed at preset time points; different frequency distributions are included in each traversal mode, and the types of frequency distribution are sparse, uniform, and intensive. The sparse type is used to simulate the scenario where pathology analysis requests in remote areas or during low-peak hours at night may be sparse and have long intervals, or there are analysis scenarios where some pathology AI types are used by a small group of people; the uniform type is used to simulate the scenario where pathology analysis requests in daily outpatient clinics usually show periodic fluctuations; the intensive type is used to simulate the scenario where when hospitals upload pathology images in batches for centralized analysis, requests surge in a short period of time.

[0035] In other words, this solution combines any type, any traversal method, and any frequency distribution to form a complete test request strategy. The purpose of this is to increase the complexity of the request type so that all pathology AI containers can be taken into account at the same time, and there will be no situation where one pathology AI type is idle for a long time.

[0036] In some specific embodiments, sparse frequency distributions include Weibull distribution, gamma distribution, and power-law distribution. The Weibull distribution and gamma distribution are skewed distributions with adjustable tail thickness. The Weibull distribution can describe the variation in the time interval between events, and is therefore suitable for simulating situations where the time interval between pathology analysis requests varies greatly. The probability density function of the Weibull distribution is as follows:

[0037]

[0038] in, is the scale parameter, is the shape parameter, in this scheme =1, This indicates that the average number of requests per minute is 1.33.

[0039] The probability density function of the gamma distribution is as follows:

[0040]

[0041] in, is the shape parameter, is the scale parameter, is the gamma function, in this scheme =2, , indicating that the average number of requests per minute is 2 times.

[0042] The probability density function of the power law distribution is as follows:

[0043]

[0044] in, is the power law exponent, yes The minimum value of =2.5, Indicates that an analysis request is sent to the pathology AI analysis system at least once per minute.

[0045] In some specific embodiments, the uniform frequency distribution is a normal distribution, which describes a situation where data is symmetrically distributed around a mean value, and is therefore suitable for simulating a situation where the time intervals between arrivals of pathology section analysis requests are relatively uniform. The probability density function of the normal distribution is as follows:

[0046]

[0047] in, is the mean, is the standard deviation.

[0048] In some specific embodiments, the intensive frequency distribution includes exponential distribution and Poisson distribution. The exponential distribution simulates memoryless random events and is suitable for simulating the situation where the time interval between the arrival of pathology slice analysis requests is short. The probability density function of the exponential distribution is as follows:

[0049]

[0050] in, is the rate at which events occur, is the time interval.

[0051] The Poisson distribution is suitable for describing the number of random events that occur within a unit time. Therefore, it is suitable for simulating the situation where pathology section analysis requests arrive intensively. The probability density formula of the Poisson distribution is expressed as follows:

[0052]

[0053] in, is the average number of occurrences per unit time, is the number of times the event occurs.

[0054] In some specific embodiments, the strategy selection algorithm is constructed based on a reinforcement learning model and is iteratively updated based on task requirements and real-time system status, wherein the task requirements include high concurrency requirements, normal concurrency requirements, and low concurrency requirements, and the real-time system status includes GPU memory occupancy, CPU memory occupancy, and the growth of the current task queue.

[0055] Specifically, the real-time system state is represented as a vector , where x represents the real-time system status, They are respectively the current GPU memory occupancy, CPU occupancy, and the length of the current task queue.

[0056] Specifically, the task requirement is represented as a vector , where r represents the task requirement, They are high concurrency, normal concurrency and low concurrency respectively.

[0057] Specifically, let S represent the strategy pool, then the strategy pool , Represents the test request strategy in the strategy pool and defines multiple objective functions ,in, For analytical accuracy, To analyze time, To analyze resource consumption.

[0058] In some specific embodiments, weights are assigned to the analysis accuracy, analysis time, and analysis resource consumption, and the expected multi-objective scores of each test request strategy under multiple optimization objectives are weightedly calculated. Then, based on the expected multi-objective scores of each test request strategy, the expected benefits of each test request strategy under task requirements and real-time system status are calculated.

[0059] Specifically, the formula for weighted calculation of the expected multi-objective score of each test request strategy under multiple optimization objectives is expressed as follows:

[0060]

[0061] in, Is the test request strategy Expected multi-goal score, To optimize the goal The weight of .

[0062] Specifically, the weight of each optimization objective in this solution is set manually. That is, if the pathology analysis system focuses more on accuracy, the weight of analysis accuracy is increased; if the computing resources of the pathology analysis system are tight, the weight of resource consumption is increased.

[0063] Specifically, the expected benefit of the test request strategy is expressed as , where x is the real-time system state and r is the task requirement. The formula for selecting the test request strategy with the highest expected benefit as the optimal test request strategy is as follows:

[0064]

[0065] in, is the optimal test request strategy.

[0066] In some specific embodiments, during the process of testing the pathology analysis system with the optimal test request strategy, the pathology test system is monitored to obtain test results. The test results are the actual performance of the optimal request test strategy under different optimization objectives. A reward value is constructed based on the test results, and the strategy selection algorithm is optimized based on the reward value.

[0067] That is to say, the test results include actual analysis accuracy, actual analysis time and actual analysis resource consumption. A reward value is constructed based on the actual analysis accuracy, actual analysis time and actual analysis resource consumption, and the strategy selection algorithm is optimized based on the reward value, thereby ensuring that the pathology AI analysis system is always tested with the most appropriate test request strategy during the test process.

[0068] Specifically, the formula for calculating the reward value is as follows:

[0069]

[0070] Among them, R is the reward value, Indicates the optimal test request strategy in optimizing the target The above expression, To optimize the goal The weight of .

[0071] In some specific embodiments, the formula for optimizing the strategy selection algorithm based on the reward value is expressed as follows:

[0072]

[0073] in, is the learning rate, is the reward value, is the discount factor, and They represent the new system status and task requirements respectively.

[0074] In other words, the strategy selection algorithm in this solution can dynamically find the most appropriate test request strategy to test the pathology analysis system according to task requirements and real-time system status to obtain the most appropriate test results.

[0075] Example 2

[0076] Based on the same concept, refer to Figure 3 , this application also proposes a device for dynamically testing a pathology AI analysis system, comprising:

[0077] A request simulation module is used to build a strategy pool with multiple test request strategies preset, and set a strategy selection algorithm. The strategy selection algorithm selects the optimal test request strategy from the strategy pool based on task requirements and real-time system status. The expected benefits of each test request strategy in the strategy pool under the three optimization goals of analysis accuracy, analysis time, and analysis resource consumption are calculated, and the test request strategy with the highest expected benefit is selected as the optimal test request strategy. The optimal test request strategy is executed to test the pathology AI analysis system to obtain test results, and the strategy selection algorithm is optimized based on the test results.

[0078] The monitoring module is used to monitor the system status of the pathology AI analysis system and the analysis accuracy, analysis time and analysis resource consumption during the pathology analysis process in real time;

[0079] The result recording module is used to record and store the information data monitored by the monitoring module;

[0080] The result display module is used to display the test results of the pathology AI analysis system to users;

[0081] The communication module is used to transmit information between the request simulation module, the monitoring module, the result recording module and the result display module.

[0082] In some specific embodiments, after the monitoring module receives the message parsed by the communication module, the monitoring module will monitor the analysis process of the pathology AI analysis system and the resource usage of the pathology AI analysis system. The analysis process of the pathology AI analysis system is monitored to obtain the pathology AI analysis results, analysis time and waiting time. The pathology AI analysis result refers to whether the pathology AI analysis of the request sent is successful. The analysis time refers to the time it takes for the pathology AI to complete the analysis of an analysis request. The waiting time refers to the waiting time for the request that is not loaded into the container when the requests are dense. These three indicators analyze and measure the success rate of the pathology AI container's processing requests, the processing rate of a single request and the degree of congestion of the pathology AI window under high concurrency.

[0083] In other words, the monitoring module mainly monitors the memory consumption, CPU occupancy and GPU memory changes during the operation of pathology AI. These indicators are of great significance for the optimization of pathology AI models.

[0084] In some specific embodiments, the main function of the result recording module is to systematically record and store the data and results generated during the test. It can capture and save the key indicator data provided by the monitoring module in real time, including pathological AI analysis results, analysis time, waiting time, memory consumption, CPU occupancy, and video memory changes, etc. These data are stored in the database as structured data by the result recording module to facilitate subsequent query and analysis.

[0085] The result logging module also supports recording and marking abnormalities during testing. For example, when a pathology AI analysis fails or resource usage exceeds a preset threshold, the module automatically generates a corresponding log with detailed contextual information such as timestamp, request content, and error type. This log information is invaluable for troubleshooting and optimizing system performance.

[0086] In some specific embodiments, the result display module is responsible for presenting test data to users in an intuitive format. It uses charts, tables, and other methods to display key indicators such as pathology AI analysis results, analysis time, wait time, and resource consumption in real time, helping users quickly understand system performance. This not only improves the readability and usability of test data but also provides strong support for performance optimization, troubleshooting, and strategy evaluation. Through intuitive data display, it is possible to more efficiently understand system performance and make informed decisions.

[0087] In some specific embodiments, the main functions of the communication module include sending and receiving. The sending part is responsible for sending the simulated request to the pathology AI management system to be tested according to a predefined strategy. Its core functions include:

[0088] Request sending: Based on the request data generated by the request simulation module, it is sent according to predefined strategies (such as request type, frequency distribution, etc.).

[0089] Protocol support: supports multiple communication protocols (such as HTTP, gRPC, WebSocket, etc.) to adapt to different pathology AI management systems.

[0090] The receiving part is responsible for promptly receiving messages returned by the pathology AI management system and passing the results to the monitoring module and result recording module. Its core functions include:

[0091] Real-time reception: Monitor the response of the pathology AI management system in real time to ensure timely receipt of messages.

[0092] Multi-threaded processing: Supports multi-threaded or asynchronous processing mechanisms to improve the efficiency of message receiving.

[0093] Protocol parsing: Parse the received message according to the communication protocol and extract valid data.

[0094] Example 3

[0095] This embodiment also provides an electronic device, referring to Figure 4 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0096] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0097] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0098] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0099] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the methods for dynamically testing the pathology AI analysis system in the above embodiments.

[0100] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0101] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0102] The input / output device 408 is used to input or output information. In this embodiment, the input information may be various test request strategies, and the output information may be test results.

[0103] Optionally, in this embodiment, the processor 402 may be configured to execute the following steps through a computer program:

[0104] Build a policy pool with multiple test request policies preset;

[0105] Setting a strategy selection algorithm, wherein the strategy selection algorithm selects an optimal test request strategy from the strategy pool based on task requirements and real-time system status, wherein the expected benefit of each test request strategy in the strategy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption is calculated, and the test request strategy with the highest expected benefit is selected as the optimal test request strategy;

[0106] Execute the optimal test request strategy to test the pathology AI analysis system to obtain test results, and optimize the strategy selection algorithm based on the test results.

[0107] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0108] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0109] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 4 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0110] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above embodiments merely illustrate several embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for dynamic testing of a pathology AI analysis system, characterized in that: The following steps are involved: Build a policy pool with multiple test request policies preset; Setting a strategy selection algorithm, wherein the strategy selection algorithm selects the optimal test request strategy from the strategy pool based on task requirements and real-time system status, wherein the task requirements include high concurrency requirements, normal concurrency requirements, and low concurrency requirements, and the real-time system status includes GPU memory occupancy, CPU occupancy, and the length of the current task queue, and by calculating the expected benefits of each test request strategy in the strategy pool under the three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy; Execute the optimal test request strategy to test the pathology AI analysis system to obtain test results, and optimize the strategy selection algorithm based on the test results.

2. A method for dynamically testing a pathology AI analysis system according to claim 1, characterized in that: The types of test request strategies preset in the strategy pool include single type and multiple types. The single type is used to simulate the situation where the pathology AI system only analyzes one type of pathology slice, and the multiple types are used to simulate the situation where the pathology AI system analyzes multiple types of pathology slices.

3. A method for dynamically testing a pathology AI analysis system according to claim 2, characterized in that: Different traversal methods are included under single type and multiple types. The traversal methods include out-of-order traversal, ordered serial traversal and ordered polling traversal. Out-of-order traversal is used to simulate the scenario where the pathology AI analysis system performs random analysis on pathology sections. Ordered serial traversal is used to simulate the scenario where pathology sections are analyzed sequentially. Polling traversal is used to simulate the scenario where pathology sections are analyzed at preset time points.

4. A method for dynamically testing a pathology AI analysis system according to claim 3, characterized in that: Each traversal mode includes different frequency distributions, and the types of frequency distribution are sparse, uniform, and dense. The sparse type is used to simulate the analysis scenarios in remote areas or low-peak hours at night where pathology analysis requests are sparse and have long intervals, or where there are some pathology AI types with a small user group. The uniform type is used to simulate the scenario in which pathology analysis requests in daily outpatient clinics show periodic fluctuations. The dense type is used to simulate the scenario in which requests surge in a short period of time when hospitals upload pathology images in batches for centralized analysis.

5. A method for dynamically testing a pathology AI analysis system according to claim 1, characterized in that: Assign weights to the analysis accuracy, analysis time, and analysis resource consumption, calculate the expected multi-objective scores of each test request strategy under multiple optimization objectives, and then calculate the expected benefits of each test request strategy under task requirements and real-time system status based on the expected multi-objective scores of each test request strategy.

6. A method for dynamically testing a pathology AI analysis system according to claim 1, characterized in that: The test results are actual performances of the optimal request test strategy under different optimization objectives. A reward value is constructed based on the test results, and the strategy selection algorithm is optimized based on the reward value.

7. A device for dynamically testing a pathology AI analysis system, characterized in that: include: A request simulation module is used to construct a strategy pool with multiple test request strategies preset, and set a strategy selection algorithm. The strategy selection algorithm selects the optimal test request strategy in the strategy pool based on task requirements and real-time system status, wherein the task requirements include high concurrency requirements, normal concurrency requirements, and low concurrency requirements, and the real-time system status includes GPU memory occupancy, CPU occupancy, and the length of the current task queue. By calculating the expected benefits of each test request strategy in the strategy pool under the three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy, the optimal test request strategy is executed to test the pathology AI analysis system to obtain test results, and the strategy selection algorithm is optimized based on the test results; The monitoring module is used to monitor the system status of the pathology AI analysis system and the analysis accuracy, analysis time and analysis resource consumption during the pathology analysis process in real time; The result recording module is used to record and store the information data monitored by the monitoring module; The result display module is used to display the test results of the pathology AI analysis system to users; The communication module is used to transmit information between the request simulation module, the monitoring module, the result recording module and the result display module.

8. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for dynamically testing a pathology AI analysis system according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes program code for controlling a process to execute a process. When the program code is executed by a processor, a method for dynamically testing a pathology AI analysis system as described in any one of claims 1 to 6 is implemented.

Images