Method and device for dynamically testing pathological AI analysis system

Abstract

The invention provides a method and a device for dynamically testing a pathology AI analysis system. The method comprises the following steps: constructing a strategy pool in which a plurality of test request strategies are preset; setting a strategy selection algorithm, wherein the strategy selection algorithm selects an optimal test request strategy in the strategy pool based on a task demand and a real-time system state; and executing the optimal test request strategy to test the pathology AI analysis system to obtain a test result, and optimizing the strategy selection algorithm based on the test result. According to the scheme, the most suitable test request strategy is dynamically searched through reinforcement learning, and the pathological analysis system is tested, so that a user is helped to quickly identify the performance bottleneck of the system.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for dynamically testing a pathology AI analysis system. Background Art

[0002] With the rapid development of artificial intelligence, AI services have been widely penetrated into many industries. Pathology AI services mainly assist doctors in diagnosing diseases more accurately through intelligent analysis of pathology images, such as identifying the characteristics of cancer cells and judging the benign and malignant nature of tumors. However, the development of pathology AI is constrained by its sensitivity and there is still great room for improvement. Pathology AI systems have extremely high requirements for accuracy and stability, because their analysis results are directly related to the accuracy of medical diagnosis, which in turn affects the patient's treatment plan and health status.

[0003] In order to ensure that the pathology AI system can run stably and reliably in complex and changeable actual application scenarios, testing of the pathology AI system is crucial. An effective testing system can not only help developers to comprehensively evaluate different AI modules and optimize their performance, but also ensure the quality and reliability of AI services in actual use. However, traditional testing methods rely on a lot of manual intervention, and the testing efficiency is low. It is difficult to meet the needs of large-scale and high-efficiency testing. In addition, the existing technology lacks a feedback mechanism for real-time monitoring of the testing process, and cannot dynamically adjust the testing strategy according to actual conditions to optimize the test results.

[0004] Since the size of pathology AI services can reach hundreds of thousands by hundreds of thousands of pixels at high resolution and is greatly affected by multiple factors such as data, algorithms, and hardware, a single fixed request strategy is simply unable to meet the diverse needs in complex scenarios. Therefore, how to make full use of existing hardware resources, simulate dynamically changing request strategies, and provide better pathology AI services is a key issue that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a method and device for dynamically testing a pathology AI analysis system, which dynamically finds the most appropriate test request strategy through reinforcement learning and tests the pathology analysis system, thereby helping users quickly identify the performance bottlenecks of the system.

[0006] In a first aspect, an embodiment of the present application provides a method for dynamically testing a pathology AI analysis system, the method comprising: Build a strategy pool with multiple test request strategies preset; Set a policy selection algorithm that selects the optimal test request policy from the policy pool based on task requirements and the real-time system state. Among them, by calculating the expected benefits of each test request policy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request policy with the highest expected benefit is selected as the optimal test request policy; Execute the optimal test request policy to test the pathological AI analysis system to obtain test results, and optimize the policy selection algorithm based on the test results.

[0007] In a second aspect, an embodiment of the present application provides a device for dynamically testing a pathological AI analysis system, including: A request simulation module for constructing a policy pool with multiple test request policies preset and setting a policy selection algorithm that selects the optimal test request policy from the policy pool based on task requirements and the real-time system state. Among them, by calculating the expected benefits of each test request policy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request policy with the highest expected benefit is selected as the optimal test request policy, execute the optimal test request policy to test the pathological AI analysis system to obtain test results, and optimize the policy selection algorithm based on the test results; A monitoring module for real-time monitoring of the system state of the pathological AI analysis system and the analysis accuracy, analysis time, and analysis resource consumption during the pathological analysis process; A result recording module for recording and storing the information data monitored by the monitoring module; A result display module for displaying the test results of the pathological AI analysis system to the user; A communication module for information transfer between the request simulation module, the monitoring module, the result recording module, and the result display module.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute a method for dynamically testing a pathological AI analysis system.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium. A computer program is stored in the readable storage medium. The computer program includes program codes for controlling a process to execute the process. When the program codes are executed by a processor, a method for dynamically testing a pathological AI analysis system is implemented.

[0010] The main contributions and innovations of the present invention are as follows: By constructing a strategy pool that includes various types, traversal methods, and frequency distribution combinations, the embodiments of the present application can simulate diverse request scenarios such as random, sequential, and timed scenarios in single / multiple pathological slide analysis scenarios of a pathological AI system, covering different request frequency scenarios such as low peaks in remote areas, fluctuations in daily outpatient cycles, and batch uploads in hospitals; based on task requirements (high / ordinary / low concurrency) and real-time system status (GPU / CPU video memory occupancy rate, task queue length), the optimal strategy is selected by weighted calculation of the expected benefits of three objectives: analysis accuracy, time, and resource consumption, and a reward value dynamic optimization strategy selection algorithm is constructed using the test results to achieve adaptive adjustment of the test strategy, and always maintain the most appropriate test request strategy to test the pathological AI analysis system.

[0011] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings: Figure 1 is a flowchart of a method for dynamically testing a pathological AI analysis system according to an embodiment of the present application; Figure 2 is a flowchart of a method for constructing a strategy pool according to the present application; Figure 3 is a structural block diagram of a device for dynamically testing a pathological AI analysis system according to an embodiment of the present application; Figure 4 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments as detailed in the appended claims.

[0014] It should be noted that: In other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0015] Embodiment 1 An embodiment of the present application provides a method for dynamically testing a pathological AI analysis system. By reinforcement learning, the most suitable test request strategy is dynamically found to test the pathological analysis system, so as to help users quickly identify the performance bottleneck of the system. Specifically, referring to Figure 1 , the method includes: Construct a policy pool preset with multiple test request strategies; Set a policy selection algorithm, which selects the optimal test request strategy in the policy pool based on the task requirements and the real-time system state. Among them, by calculating the expected benefits of each test request strategy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy; Execute the optimal test request strategy to test the pathological AI analysis system to obtain test results, and optimize the policy selection algorithm based on the test results.

[0016] In some embodiments, the construction method of the policy pool is as Figure 2 shown, and the types of test request strategies preset in the policy pool meet all task requirements of practical applications.

[0017] Furthermore, according to Figure 2It can be seen that when constructing the policy pool, the types analyzed by the pathological AI system are pre-selected in this solution, including single type and multiple types. The single type is used to simulate the situation where the pathological AI system only analyzes one type of pathological section, and the multiple types are used to simulate the situation where the pathological AI system analyzes multiple types of pathological sections. For example, when the tested pathological AI analysis system only analyzes one type of pathological section, namely Ki67, a single-type policy is adopted; when the tested pathological AI analysis system analyzes two types of pathological sections, namely Ki67 and Stomach, a multiple-type policy is adopted. Under the single type and multiple types, different traversal methods are included respectively. The types of traversal methods include unordered traversal, ordered serial traversal, and ordered polling traversal. Unordered traversal is used to simulate the scenario where the pathological AI analysis system randomly analyzes pathological sections. Ordered serial traversal is used to simulate the scenario where pathological sections are analyzed sequentially. Polling traversal is used to simulate the scenario where pathological sections are analyzed at preset time points. Under each traversal method, different frequency distributions are included respectively. The types of frequency distributions are sparse type, uniform type, and dense type. The sparse type is used to simulate the scenario where pathological analysis requests in remote areas or during low-peak hours at night may be sparse and have long intervals, or there are fewer users of some pathological AI types. The uniform type is used to simulate the scenario where pathological analysis requests in daily outpatient clinics usually show periodic fluctuations. The dense type is used to simulate the scenario where requests surge within a short period when hospitals upload pathological images in batches for centralized analysis.

[0018] That is to say, in this solution, an arbitrary type plus an arbitrary traversal method plus an arbitrary frequency distribution are selected to form a complete test request policy. The purpose of doing this is to increase the complexity of request types, be able to take into account all pathological AI containers at the same time, and avoid the situation where a certain pathological AI type is idle for a long time.

[0019] In some specific embodiments, the sparse-type frequency distribution includes Weibull distribution, gamma distribution, and power-law distribution. The Weibull distribution and gamma distribution are skewed distributions with adjustable tail thickness. The Weibull distribution can describe the variation of the time interval between events, so it is suitable for simulating the situation where the time interval of pathological analysis requests varies greatly. The probability density function of the Weibull distribution is as follows:

[0020] Among them, is the scale parameter, is the shape parameter. In this solution = 1, indicating that the average number of requests per minute is 1.33 times.

[0021] The probability density function of the gamma distribution is as follows:

[0022] Among them, is the shape parameter, is the scale parameter, is the gamma function. In this solution = 2, , indicating that the average number of requests per minute is 2 times.

[0023] The probability density function of the power-law distribution is as follows:

[0024] Among them, is the power-law exponent, is the minimum value of = 2.5, indicating that at least one analysis request is sent to the pathological AI analysis system per minute.

[0025] In some specific embodiments, the uniform frequency distribution is a normal distribution, and the normal distribution describes the situation where data is symmetrically distributed around the mean, so it is suitable for simulating the relatively uniform time interval of the arrival of pathological slice analysis requests. The probability density function of the normal distribution is as follows:

[0026] Among them, is the mean, is the standard deviation.

[0027] In some specific embodiments, the intensive frequency distribution includes an exponential distribution and a Poisson distribution. The exponential distribution simulates memoryless random events and is suitable for simulating the relatively short time interval of the arrival of pathological slice analysis requests. The probability density function of the exponential distribution is as follows:

[0028] Among them, is the rate of event occurrence, is the time interval.

[0029] The Poisson distribution is applicable to describing the number of random events occurring within a unit time, so it is suitable for simulating the intensive arrival of pathological slice analysis requests. The probability density formula of the Poisson distribution is expressed as follows:

[0030] Among them, is the average number of times the event occurs within a unit time, is the number of times the event occurs.

[0031] In some specific embodiments, the policy selection algorithm is constructed based on a reinforcement learning model and iteratively updated based on task requirements and the real-time system state. Among them, the task requirements include high concurrency requirements, normal concurrency requirements, and low concurrency requirements, and the real-time system state includes the GPU video memory occupancy rate, the CPU video memory occupancy rate, and the growth of the current task queue.

[0032] Specifically, represent the real-time system state as a vector , where x represents the real-time system state, are the current GPU video memory occupancy rate, the CPU occupancy rate, and the length of the current task queue respectively.

[0033] Specifically, represent the task requirements as a vector , where r represents the task requirements, are high concurrency, normal concurrency, and low concurrency respectively.

[0034] Specifically, use S to represent the policy pool. Then the policy pool , represents the test request policy in the policy pool. Define the multi-objective function , where is the analysis accuracy, is the analysis time, is the analysis resource consumption.

[0035] In some specific embodiments, assign weights to the analysis accuracy, analysis time, and analysis resource consumption, calculate the expected multi-objective score of each test request policy under multiple optimization objectives through weighted calculation, and then calculate the expected benefit of each test request policy under the task requirements and the real-time system state based on the expected multi-objective score of each test request policy.

[0036] Specifically, the formula for calculating the expected multi-objective score of each test request policy under multiple optimization objectives through weighted calculation is as follows:

[0037] Among them, is the expected multi-objective score of the test request policy , is the weight of the optimization objective .

[0038] Specifically, the weight of each optimization objective in this solution is set manually. That is to say, if the pathological analysis system pays more attention to accuracy, then increase the weight of the analysis accuracy. If the computing resources of the pathological analysis system are tight, then increase the resource consumption weight.

[0039] Specifically, the calculation of the expected benefit of the test request policy is expressed as , where \(x\) is the real-time system state and \(r\) is the task requirement. The formula for selecting the test request strategy with the highest expected return as the optimal test request strategy is as follows:

[0040] where is the optimal test request strategy.

[0041] In some specific embodiments, during the process of testing the pathological analysis system with the optimal test request strategy, the pathological test system is monitored to obtain test results. The test results are the actual performances of the optimal request test strategy under different optimization objectives. Based on the test results, a reward value is constructed, and the strategy selection algorithm is optimized based on the reward value.

[0042] That is to say, the test results include the actual analysis accuracy, actual analysis time, and actual analysis resource consumption. Based on the actual analysis accuracy, actual analysis time, and actual analysis resource consumption, a reward value is constructed, and the strategy selection algorithm is optimized based on the reward value, so as to ensure that the pathological AI analysis system is always tested with the most appropriate test request strategy during the test process.

[0043] Specifically, the formula for calculating the reward value is as follows:

[0044] where \(R\) is the reward value, represents the representation of the optimal test request strategy on the optimization objective , is the optimization objective weight.

[0045] In some specific embodiments, the formula for optimizing the strategy selection algorithm based on the reward value is as follows:

[0046] where is the learning rate, is the reward value, is the discount factor, and represent the new system state and task requirement respectively.

[0047] That is to say, the strategy selection algorithm in this solution can dynamically find the most appropriate test request strategy according to the task requirement and the real-time system state to test the pathological analysis system to obtain the most appropriate test results.

[0048] Embodiment 2 Based on the same concept, refer to Figure 3, this application also proposes a device for dynamically testing a pathological AI analysis system, including: A request simulation module, which is used to construct a policy pool with multiple test request strategies preset and set a policy selection algorithm. The policy selection algorithm selects the optimal test request strategy from the policy pool based on task requirements and the real-time system state. Among them, by calculating the expected benefits of each test request strategy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy, and the optimal test request strategy is executed to test the pathological AI analysis system to obtain test results, and the policy selection algorithm is optimized based on the test results; A monitoring module, which is used to monitor the system state of the pathological AI analysis system and the analysis accuracy, analysis time, and analysis resource consumption during the pathological analysis process in real time; A result recording module, which is used to record and store the information data monitored by the monitoring module; A result display module, which is used to display the test results of the pathological AI analysis system to the user; A communication module, which is used to transfer information between the request simulation module, the monitoring module, the result recording module, and the result display module.

[0049] In some specific embodiments, after the monitoring module receives the message parsed by the communication module, the monitoring module will monitor the analysis process of the pathological AI analysis system and the resource usage of the pathological AI analysis system. What is obtained by monitoring the analysis process of the pathological AI analysis system is the pathological AI analysis result, analysis time, and waiting time. The pathological AI analysis result refers to whether the sent request for pathological AI analysis is successful. The analysis time refers to the time taken for a pathological AI to complete the analysis of an analysis request. The waiting time refers to the time that requests not loaded into the container wait when the requests are intensive. These three indicators analyze and measure the success rate of the pathological AI container in processing requests, the processing rate of a single request, and the degree of congestion of the pathological AI window during high concurrency.

[0050] That is to say, the monitoring module mainly monitors the memory consumption, CPU occupancy rate, and GPU video memory changes during the operation of the pathological AI. These indicators are of great significance for the optimization of the pathological AI model.

[0051] In some specific embodiments, the main function of the result recording module is to systematically record and store the data and results generated during the test process. It can capture and save various key indicator data provided by the monitoring module in real time, including pathological AI analysis results, analysis time, waiting time, memory consumption, CPU occupancy rate, and video memory changes, etc. These data are stored in the database by the result recording module as structured data for subsequent query and analysis.

[0052] In addition, the result recording module also supports recording and marking abnormal situations during the test. For example, when the pathological AI analysis fails or the resource usage exceeds the preset threshold, the module will automatically generate corresponding logs and record detailed context information, such as timestamps, request content, error types, etc. These log information are of great reference value for troubleshooting problems and optimizing system performance.

[0053] In some specific embodiments, the result display module is responsible for presenting the test data to the user in an intuitive form. It uses various methods such as charts and tables to display key metrics such as pathological AI analysis results, analysis time consumption, waiting time consumption, and resource consumption in real time, helping users quickly grasp the system performance. It not only improves the readability and usability of the test data, but also provides strong support for performance optimization, problem troubleshooting, and strategy evaluation. Through intuitive data display, the system performance can be understood more efficiently and scientific decisions can be made.

[0054] In some specific embodiments, the main functions of the communication module include two parts: sending and receiving. The sending part is responsible for sending the simulated requests to the pathological AI management system to be tested according to predefined strategies. Its core functions include: Request sending: Send the request data generated by the request simulation module according to predefined strategies (such as request type, frequency distribution, etc.).

[0055] Protocol support: Support multiple communication protocols (such as HTTP, gRPC, WebSocket, etc.) to adapt to different pathological AI management systems.

[0056] The receiving part is responsible for timely receiving the messages returned by the pathological AI management system and passing the results to the monitoring module and the result recording module. Its core functions include: Real-time reception: Real-time monitor the responses of the pathological AI management system to ensure timely message reception.

[0057] Multi-threaded processing: Support multi-threaded or asynchronous processing mechanisms to improve the efficiency of message reception.

[0058] Protocol parsing: Parse the received messages according to the communication protocol and extract the valid data.

[0059] Embodiment 3 This embodiment also provides an electronic device, refer to Figure 4 , including a memory 404 and a processor 402. The memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0060] Specifically, the above-mentioned processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present application.

[0061] Among them, the memory 404 may include a mass storage 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In suitable cases, the memory 404 may include removable or non-removable (or fixed) media. In suitable cases, the memory 404 may be internal or external to the data processing device. In a particular embodiment, the memory 404 is non-volatile memory. In a particular embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). In suitable cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. In suitable cases, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended date out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0062] The memory 404 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402.

[0063] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for dynamically testing the pathological AI analysis system in the above embodiments.

[0064] Optionally, the above electronic device may further include a transmission device 406 and an input / output device 408. Among them, the transmission device 406 is connected to the above processor 402, and the input / output device 408 is connected to the above processor 402.

[0065] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above network may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0066] The input / output device 408 is used to input or output information. In this embodiment, the input information can be various test request strategies, etc., and the output information can be test results, etc.

[0067] Optionally, in this embodiment, the above processor 402 can be set to execute the following steps through a computer program: Construct a policy pool with a variety of test request strategies preset; Set a policy selection algorithm, which selects the optimal test request strategy in the policy pool based on task requirements and the real-time system state. Among them, by calculating the expected benefits of each test request strategy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy; Execute the optimal test request strategy to test the pathological AI analysis system to obtain test results, and optimize the policy selection algorithm based on the test results.

[0068] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and alternative embodiments, and will not be elaborated here.

[0069] In general, various embodiments can be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. Some aspects of the present invention can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, a microprocessor, or other computing devices, but the present invention is not limited thereto. Although various aspects of the present invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, the blocks, devices, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or controllers, or other computing devices, or some combination thereof.

[0070] Embodiments of the present invention can be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. A computer software or program (also referred to as a program product), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer-executable components that are configured to perform the embodiments when the program runs. One or more computer-executable components can be at least one software code or a part thereof. Additionally, in this regard, it should be noted that any box in the logical flow, as Figure 4 described, can represent a program step, or interconnected logic circuits, boxes, and functions, or a combination of program steps and logic circuits, boxes, and functions. The software can be stored on physical media such as memory chips or storage blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. The physical media are non-transitory media.

[0071] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification.

[0072] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but should not be construed as a limitation on the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for dynamically testing a pathological AI analysis system, characterized in that, It includes the following steps: Construct a policy pool with multiple test request strategies preset; Set a policy selection algorithm, which selects the optimal test request strategy from the policy pool based on the task requirements and the real-time system state. Among them, by calculating the expected benefits of each test request strategy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy; Execute the optimal test request strategy to test the pathological AI analysis system to obtain test results, and optimize the policy selection algorithm based on the test results.

2. The method for dynamically testing a pathological AI analysis system according to claim 1, wherein The types of test request strategies preset in the policy pool include single type and multiple types. The single type is used to simulate the situation where the pathological AI system only analyzes one type of pathological section, and the multiple types are used to simulate the situation where the pathological AI system analyzes multiple types of pathological sections.

3. The method for dynamically testing a pathological AI analysis system according to claim 2, characterized in that Under the single type and multiple types, different traversal methods are included respectively. The types of traversal methods include unordered traversal, ordered serial traversal, and ordered polling traversal. Unordered traversal is used to simulate the scenario where the pathological AI analysis system randomly analyzes pathological sections, ordered serial traversal is used to simulate the scenario of sequentially analyzing pathological sections, and polling traversal is used to simulate the scenario of analyzing pathological sections at a preset time point.

4. A method for dynamically testing a pathological AI analysis system according to claim 3, characterized in that, Under each traversal method, different frequency distributions are included respectively. The types of frequency distributions are sparse type, uniform type, and dense type. The sparse type is used to simulate the scenario where the pathological analysis requests in remote areas or during the low peak period at night are sparse and have a long interval, or there is a small number of users of some pathological AI types. The uniform type is used to simulate the scenario where the pathological analysis requests in daily outpatient clinics show periodic fluctuations. The dense type is used to simulate the scenario where when the hospital uploads pathological images in batches for centralized analysis, the requests will surge within a short period of time.

5. A method for dynamically testing a pathological AI analysis system according to claim 1, characterized in that, The task requirements include high concurrency requirements, normal concurrency requirements, and low concurrency requirements. The real-time system state includes GPU video memory occupancy rate, CPU video memory occupancy rate, and the length of the current task queue.

6. A method for dynamically testing a pathological AI analysis system according to claim 1, characterized in that, Assign weights to the analysis accuracy, analysis time, and analysis resource consumption, calculate the expected multi-objective scores of each test request strategy under multiple optimization objectives through weighted calculation, and then calculate the expected benefits of each test request strategy under the task requirements and the real-time system state based on the expected multi-objective scores of each test request strategy.

7. A method for dynamically testing a pathological AI analysis system according to claim 1, characterized in that, The test result is the actual performance of the optimal request test strategy under different optimization objectives. Based on the test result, a reward value is constructed, and the policy selection algorithm is optimized based on the reward value.

8. A device for dynamically testing a pathological AI analysis system, characterized in that, It includes: A request simulation module, configured to build a policy pool with multiple preset test request strategies and set a policy selection algorithm. The policy selection algorithm selects an optimal test request strategy from the policy pool based on task requirements and the real-time system state. Specifically, by calculating the expected benefits of each test request strategy in the policy pool under three optimization objectives of analysis accuracy, analysis time, and analysis resource consumption, the test request strategy with the highest expected benefit is selected as the optimal test request strategy. The optimal test request strategy is executed to test the pathological AI analysis system to obtain test results, and the policy selection algorithm is optimized based on the test results; A monitoring module, configured to monitor in real time the system state of the pathological AI analysis system, as well as the analysis accuracy, analysis time, and analysis resource consumption during the pathological analysis process; A result recording module, configured to record and store the information data monitored by the monitoring module; A result display module, configured to display the test results of the pathological AI analysis system to the user; A communication module, configured to transfer information between the request simulation module, the monitoring module, the result recording module, and the result display module.

9. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for dynamically testing a pathological AI analysis system according to any one of claims 1-7.

10. A readable storage medium, characterized in that, A computer program is stored in the readable storage medium. The computer program includes program codes for controlling a process to execute the process. When the program codes are executed by the processor, the method for dynamically testing a pathological AI analysis system according to any one of claims 1-7 is implemented.

Images