Slice analysis method and device and storage medium

Abstract

The invention discloses a slice analysis method and device and a storage medium, and relates to the field of pathological sections.The method comprises the steps that when a browsing operation on a to-be-analyzed slice is detected, a region of interest of the to-be-analyzed slice is determined; obtaining a rapid analysis result of the region of interest, and outputting the rapid analysis result at a target position of the region of interest; when it is detected that the browsing duration of the region of interest is greater than or equal to a preset duration, obtaining an accurate analysis result of the region of interest from a preset cache region; and outputting the accurate analysis result in the current browsing area. According to the method, the rapid analysis result is directly provided for the user, the accurate analysis result is directly output after the user browses the rapid analysis result, and the problem that the analysis content is not output in time is solved by sequentially outputting the analysis results with different accuracies.

Description

Technical Field

[0001] This application relates to the field of pathological slide technology, and in particular to slide analysis methods, equipment and storage media. Background Technology

[0002] With the development of digital pathology technology, pathologists are now able to view high-resolution web scan images (WSI) online. Current technologies primarily employ a "pyramid segmentation" technique to achieve smooth WSI previews and utilize asynchronous task queues for AI-assisted analysis (such as lesion detection and tissue segmentation). A typical workflow involves the user first viewing the entire slide, then manually triggering a background analysis task when determining the content to be analyzed. After analysis using a deep learning model, the results (such as bounding boxes and heatmaps) are overlaid and displayed on the image. However, this approach, which uses complex deep learning models, takes a long time (often exceeding one second) to perform a single inference operation on a high-resolution viewport, failing to meet the requirements for real-time interaction (sub-second response). Users must wait a considerable amount of time to view the analysis content, resulting in excessively long system response times.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a slice analysis method, device and storage medium, which aims to solve the technical problem that users cannot obtain the analysis content in real time.

[0005] To achieve the above objectives, this application proposes a slice analysis method applied to a front-end, the steps of which include: When a browsing operation on a slide to be analyzed is detected, the region of interest for the slide to be analyzed is determined, and the region of interest is determined according to the pathological type corresponding to the slide to be analyzed. The rapid analysis results of the region of interest are obtained and output at the target location corresponding to the region of interest. The rapid analysis results include candidate lesions and the fuzzy boundaries of the candidate lesions. If the browsing time of the area of ​​interest is greater than or equal to the preset time, the precise analysis result of the area of ​​interest is obtained from the preset cache. The fast analysis result is generated by the front end, and the precise analysis result is generated by the cloud. The precise analysis result includes lesions and clear boundaries of lesions. The precise analysis results are output at the target location.

[0006] For example, the step of determining the region of interest in the slice to be analyzed includes: Monitor the user's first interactive operation while browsing the slice to be analyzed, and determine the current activity level of the currently browsed area based on the first interactive operation; When the current activity level is less than or equal to a preset activity level threshold, the currently browsed area is designated as the area of ​​interest.

[0007] For example, the step of obtaining the rapid analysis results of the region of interest includes: A quick analysis request is generated based on the region of interest, and the quick analysis request is sent to the quick analysis model, wherein the quick analysis model is a lightweight model deployed on the front end; Receive the rapid analysis results in response to the rapid analysis model.

[0008] For example, the step of obtaining the accurate analysis results of the region of interest from the preset cache includes: A precise analysis request is generated based on the region of interest, and the precise analysis request is sent to a precise analysis model deployed in the cloud. The precise analysis model responds to the precise analysis request by obtaining the slice data corresponding to the region of interest, and generates the precise analysis result based on the slice data. Receive and cache the precise analysis results fed back by the precise analysis model.

[0009] For example, the step of generating a precise analysis request based on the region of interest includes: Obtain rapid analysis results for the region of interest, including the target pathological region and / or target pathological features; The precise analysis request is generated based on the rapid analysis results.

[0010] For example, the step of generating the precise analysis request based on the rapid analysis results includes: Obtain the second interactive operation of the user when browsing the quick analysis results, and determine the target area of ​​interest based on the second interactive operation; Generate a precise analysis request that includes the target region of interest.

[0011] For example, the step of outputting the precise analysis result at the target location includes: Obtain alignment rules, which are used to define the alignment rules between the fast analysis results of the fast analysis model and the accurate analysis results of the accurate analysis model in terms of spatial location and semantic category; Based on the alignment rules, an alignment operation is performed on the precise analysis results and the rapid analysis results. The alignment operation includes a smooth replacement operation and / or an enhancement operation.

[0012] For example, the slice analysis method further includes: Obtain the runtime load parameters of the precise analysis model; Adjust the preset activity threshold based on the aforementioned operating load parameters.

[0013] Furthermore, to achieve the above objectives, this application also proposes a slice analysis apparatus, the slice analysis apparatus comprising: A fast analysis model, which is a lightweight model deployed on the front end, is used to generate fast analysis results based on fast analysis requests; A precise analysis model, which is a high-precision analysis model deployed on a cloud server, is used to generate precise analysis results based on precise analysis requests.

[0014] In addition, to achieve the above objectives, this application also proposes a slice analysis device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the slice analysis method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the slice analysis method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the slice analysis method described above.

[0017] The one or more technical solutions proposed in this application have at least the following technical effects: When it is determined that a user is browsing a slice to be analyzed, the region of interest for which the user needs to view the analysis content is first obtained, and then a quick analysis result is directly output. The user can gain a preliminary understanding of the region of interest based on the quick analysis result. When it is determined that the user still needs to analyze the region of interest in detail based on the user's browsing time, the precise analysis result is directly obtained from the pre-set cache and output for the user to view analyses of different precision levels. Since the quick analysis result is output directly first, the user does not need to manually trigger the query task for the region of interest to view the corresponding analysis content. At the same time, when the user views the quick analysis result, a more precise analysis result is cached in advance for the user, and when it is determined that the user needs to view a more precise analysis result, the precise analysis result can also be directly output. This solves the problem that the user needs to manually trigger the query task for the region of interest and has to wait a long time to see the analysis result, thus improving the efficiency of viewing the analysis result. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the module structure of the slice analysis device in the first embodiment of the slice analysis method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the slice analysis method of this application; Figure 3 This is a detailed flowchart of step S10 in the second embodiment of the slice analysis method of this application; Figure 4 This is a detailed flowchart of step S20 in the second embodiment of the slice analysis method of this application; Figure 5 This is a detailed flowchart of step S30 in the second embodiment of the slice analysis method of this application; Figure 6 This is a detailed flowchart of step S40 in the second embodiment of the slice analysis method of this application; Figure 7 This is a technical architecture diagram of the slice analysis method according to the second embodiment of the present application; Figure 8 This is a flowchart of the slice analysis method involved in the second embodiment of the slice analysis method of this application; Figure 9 This is a detailed flowchart of step S31 in the third embodiment of the slice analysis method of this application; Figure 10 This is a detailed flowchart of step S312 in the third embodiment of the slice analysis method of this application; Figure 11 This is a schematic diagram of the device structure of the hardware operating environment involved in the slice analysis method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] In related technologies, if a doctor wants to obtain analysis results for pathological slides, they need to manually trigger the analysis process. In response, the front end sends the pathological slides to a large model deployed in the cloud. After receiving the pathological slides, the cloud generates analysis results based on them and returns them to the front end for display. Due to the limited analysis time and communication latency of the large cloud model, there is a significant delay between the front end receiving the doctor's request and outputting the analysis results. This can easily disrupt the doctor's diagnostic process, leading to low diagnostic efficiency.

[0024] This application provides a solution that first identifies the area of ​​interest through automatic detection, then generates and outputs rapid analysis results at a sub-second speed, allowing doctors to make a preliminary diagnosis based on these results. During the user's browsing process, precise analysis results for the area of ​​interest are generated synchronously and pre-cached. When the browsing time is greater than or equal to a preset time, if it is determined that the doctor needs more precise analysis results for diagnosis, the pre-cached precise analysis results are directly retrieved from the preset cache and output, achieving a "seamless switching" between precise analysis results.

[0025] This application delivers rapid analysis results to users in the shortest possible time, and generates and caches precise analysis results in advance while users are browsing these results. This reduces the time spent waiting for precise analysis results, solves the problem of untimely output of analysis results, and improves diagnostic efficiency.

[0026] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0027] First Embodiment In this application, the slice analysis method can be performed by a slice analysis device. This embodiment provides a slice analysis device, which can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or slice analysis device capable of performing the above functions.

[0028] Please refer to Figure 1 The slice analysis device 10 provided in this embodiment includes: A fast analysis model 101 is a lightweight model deployed on the front end, used to generate fast analysis results based on fast analysis requests. The precise analysis model 102 is a high-precision analysis model deployed on a cloud server, used to generate precise analysis results based on precise analysis requests.

[0029] In one feasible implementation, the slice analysis device 10 includes a fast analysis model 101. This fast analysis model 101 is a lightweight model. When deployed on front-end and / or edge devices, it runs locally in the user's browser or device, eliminating the need to send data to a server for processing, thus avoiding network transmission delays. Users can receive feedback almost immediately after operation, significantly improving interaction smoothness and obtaining real-time fast analysis results. Simultaneously, deploying the fast analysis model 101 on front-end and / or edge devices distributes the analysis tasks to the client, greatly reducing backend computing pressure and bandwidth consumption. It eliminates the need to maintain large-scale GPU clusters, significantly reducing long-term operating costs. It should be noted that when the fast analysis model 101 receives a fast analysis request, it outputs the corresponding fast analysis result at a sub-second speed.

[0030] Optionally, the slice analysis device 10 further includes a precise analysis model 102, which is deployed on a cloud server. Compared with the related technologies that deploy the precise analysis model 102 on the front end, the precise analysis model 102 in this embodiment can reduce the computational overhead of the front end processor while outputting precise analysis results faster.

[0031] It should be noted that the precise analysis results output by the precise analysis model 102 are more accurate and precise than those output by the rapid analysis model 101. For example, when the analysis task is pathological slide analysis, the differences between the rapid analysis results output by the rapid analysis model 101 and the precise analysis results output by the precise analysis model 102 include: Spatial boundary precision differences. Based on the rapid analysis results, due to the priority of processing speed, a coarse detection algorithm is usually used, and the output lesion boundaries may appear blurry, with insufficiently sharp boundary lines and unclear edge pixel transitions. In contrast, the precise analysis results, based on high-resolution images and refined algorithm processing, output clear and sharp lesion boundaries, accurately delineating the boundary between lesions and normal tissue. There are also differences in detection precision. Based on the rapid analysis results, to ensure real-time performance, a simplified model or reduced resolution processing may be used, resulting in relatively low detection precision and a certain risk of missed or false detections. For the precise analysis results, a complete pathological slide is used... The analysis process includes paraffin section preparation, various staining techniques, and molecular detection, resulting in high detection accuracy and providing more accurate diagnostic information. However, there are semantic classification differences between rapid and precise analysis results. For rapid analysis results, the rapid analysis model 101 typically provides broad classification labels, such as "suspected tumor" or "inflammatory area," with a coarser classification granularity. In contrast, for precise analysis results, the precise analysis model 102 provides detailed pathological classifications, such as specific diagnoses like "carcinoma in situ," "invasive carcinoma," and "benign hyperplasia," with a finer classification granularity. The differences between rapid and precise analysis results also include differences in confidence levels. For rapid analysis results, the confidence score is usually lower to reflect the uncertainty of the initial judgment, while for precise analysis results, the confidence score is higher, providing stronger diagnostic persuasiveness.

[0032] It should be noted that the rapid analysis results output by the rapid analysis model 101 and the precise analysis results output by the precise analysis model 102 have certain similarities. For example, when the analysis task is pathological slide analysis, both the rapid analysis model 101 and the precise analysis model 102 serve the same diagnostic goal, rely on a unified data foundation, and together constitute an organic whole for hierarchical diagnosis. Specifically, the diagnostic goals of the rapid analysis model 101 and the precise analysis model 102 are consistent, both pointing to the identification results of pathological features in the pathological slides. In other words, whether it is rapid analysis or precise analysis, their fundamental purpose is to identify key pathological features in the slides, such as tumor cells, inflammatory areas, or tissue atypia, and to spatially locate them, providing a basis for the final clinical diagnosis.

[0033] It is understood that the dual-level pathological analysis model construction technology proposed in this application constructs a differentiated dual-model architecture for the same pathological analysis task (such as tumor detection and tissue segmentation): a lightweight fast-response model is deployed on the front-end and / or edge devices to achieve sub-second preliminary analysis feedback; a high-precision accurate calculation model is deployed on the cloud server to ensure the accuracy of the analysis results, decoupling the contradiction between "fast" and "accurate" at the model level and laying the foundation for real-time interaction.

[0034] In one feasible implementation, the slide analysis device 10 further includes a dynamic scheduling engine 103. The dynamic scheduling engine 103 establishes communication with the fast analysis model 101 and the precise analysis model 102 respectively to schedule the corresponding models to analyze the pathological slides. Specifically, when the slide analysis device 10 detects a browsing operation on the slide to be analyzed, it determines that the user has made a pathological diagnosis based on the slide. At this time, a communication connection is established between the user terminal and the fast analysis model 101 and the precise analysis model 102 to trigger the fast analysis model 101 and the precise analysis model 102 to analyze the slide. When the dynamic scheduling engine 103 detects a browsing operation, based on the communication connection with the fast analysis model 101 and the precise analysis model 102, it triggers the fast analysis model 101 to generate and output the fast analysis results in real time at a sub-second speed. The fast analysis results include a rough bounding box of suspected regions in the slide to be analyzed.

[0035] While the user browses the quick analysis results for a preliminary diagnosis, the system waits for the precise analysis model 102 to generate precise analysis results. These results are then cached in a pre-set cache area 105, allowing for timely output when the user needs to view the precise analysis results, without requiring a waiting period. Alternatively, after the precise analysis model 102 generates the precise analysis results, they can be directly output. Simultaneously, the precise analysis results are cached in the pre-set cache area so that when browsing the same analysis slice again, the cached results can be directly accessed without repeated analysis. This reduces the inference overhead of the precise analysis model 102 and improves the efficiency of analysis result output.

[0036] It should be noted that the preset cache 105 establishes a communication connection with the precise analysis model 102 to receive the precise analysis results generated by the precise analysis model 102, and then performs a caching operation on the received precise analysis results. The preset cache 105 can be deployed on the front end or the back end; this is not limited here. In this embodiment, the preset cache is deployed on the front end for example.

[0037] In one feasible implementation, the preset cache 105 caches the precise analysis results by establishing a corresponding cache index for each received precise analysis result. The cache index is associated with at least the digital pathological slide identifier and viewport coordinates corresponding to the precise analysis result. When the user browses to the same area of ​​the same slide again, the result can be retrieved and displayed from the preset cache 105 based on the cache index, without triggering a new precise calculation task.

[0038] Specifically, the dynamic scheduling engine 103 establishes communication with the fast analysis model 101 and the precise analysis model 102 respectively to schedule the corresponding models to analyze the pathological slides. When a browsing operation is detected, the fast analysis model 101 and the precise analysis model 102 are simultaneously scheduled to generate corresponding analysis results based on the slide to be analyzed. The fast analysis results generated by the fast analysis model 101 are output first for the user to browse, and the precise analysis results are waited for by the precise analysis model 102. After receiving the precise analysis results, the precise analysis results are output directly.

[0039] In another embodiment, the dynamic scheduling engine 103 establishes communication with the fast analysis model 101 and the precise analysis model 102 respectively to schedule the corresponding models to analyze the pathological slides. Alternatively, when a browsing operation is detected, the fast analysis model 101 and the precise analysis model 102 are simultaneously scheduled to generate corresponding analysis results based on the slides to be analyzed. The fast analysis results generated by the fast analysis model 101 are output first for the user to browse, and the precise analysis results are waited for by the precise analysis model 102. After receiving the precise analysis results, the precise analysis results are cached. It is determined whether the user is still browsing the same area. If so, the precise analysis results are output. If not, the precise analysis results are stored in the preset cache area 105 in the form of an index for the user to view later.

[0040] In another embodiment, the dynamic scheduling engine 103 establishes communication with the fast analysis model 101 and the precise analysis model 102 respectively to schedule the corresponding models to analyze the pathological slides. Alternatively, when a browsing operation is detected, the engine acquires the user's first interactive operation while browsing the slide to be analyzed. The first interactive operation includes at least one of the following: viewport movement speed, zoom level change rate, mouse / touch trajectory, and dwell time in the current viewport. Based on the first interactive operation, the activity level of the currently browsed area is determined. When the activity level is low, it indicates that the user is carefully observing the area and has a high level of attention to it. When the activity level is high, it indicates that the user is not paying attention to the area. Then, based on the activity level of the area, the target scheduling model is determined. When the activity level is high, no analysis model is scheduled, or only the fast analysis model 101 is scheduled, and the fast analysis result of the fast analysis model 101 is output. When the activity level is low, both the fast analysis model 101 and the precise analysis model 102 are scheduled simultaneously, outputting the fast analysis result for the user to browse while waiting to receive the precise analysis result output by the precise analysis model 102, and then outputting the precise analysis result. It should be noted that when the activity level is low, the fast analysis model 101 and the precise analysis model 102 are scheduled simultaneously. While outputting the fast analysis results for the user to browse, the system waits to receive the precise analysis results output by the precise analysis model 102. During this process, it monitors whether the current viewport is still in the current browsing area. If so, the precise analysis results are output; otherwise, the precise analysis results are cached for the next call.

[0041] In another embodiment, the dynamic scheduling engine 103 establishes communication with the fast analysis model 101 and the precise analysis model 102 respectively to schedule the corresponding models to analyze the pathological slides. Alternatively, when a browsing operation is detected, the region of interest of the slide to be analyzed is determined, and the fast analysis model 101 and the precise analysis model 102 are triggered at the same time. The fast analysis results are output for the user to browse, while waiting to receive the precise analysis results output by the precise analysis model 102, and then outputting the precise analysis results. It can also output quick analysis results for user browsing while waiting to receive precise analysis results generated by precise analysis model 102. After receiving the precise analysis results, the precise analysis results are cached in a preset cache area, and the browsing time of the user on the area of ​​interest is monitored in real time. If the browsing time exceeds the preset time, it is determined that the user is still performing pathological diagnosis based on the quick analysis results. At this time, it means that the user needs precise analysis results, and then the cached precise analysis results are output. Alternatively, it can output quick analysis results for user browsing while waiting to receive precise analysis results output by precise analysis model 102. After receiving the precise analysis results, the browsing time of the user on the area of ​​interest is obtained. If the browsing time exceeds the preset time, the precise analysis results are output. Alternatively, it can output quick analysis results for user browsing while waiting to receive precise analysis results output by precise analysis model 102. After receiving the precise analysis results, it is determined whether the current viewport is still on the area of ​​interest. If the current viewport has left the area of ​​interest, the precise analysis results are cached. When the current viewport is detected to have returned to the area of ​​interest, the cached precise analysis results are output.

[0042] It is understood that the output of analysis results using the dynamic scheduling engine 103 scheduling analysis model includes, but is not limited to, the above embodiments. This application does not make specific limitations on how to schedule the model and how to output the results.

[0043] In another embodiment, the slice analysis device 10 further includes a resource scheduling engine 104, which monitors the cloud task queue and network bandwidth in real time, determines the system load in real time, and adaptively adjusts the activity threshold for triggering precise calculation and / or reduces the priority of precise calculation tasks based on the system load, so as to realize on-demand analysis and efficient utilization of computing resources and improve the efficiency of analysis results.

[0044] The slice analysis device 10 proposed in this application constructs analysis models with different analysis accuracies in a layered manner, schedules the corresponding analysis models based on user needs, and consumes waiting time in a balanced manner so that the system does not consume waiting time, thereby achieving “seamless” output of analysis results and improving the output efficiency and diagnostic efficiency of analysis results.

[0045] Second Embodiment Based on the first embodiment, this application also provides a slice analysis method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the slicing method of this application.

[0046] In this embodiment, the slice analysis method includes steps S10 to S40: Step S10: When a browsing operation on the slice to be analyzed is detected, the region of interest of the slice to be analyzed is determined.

[0047] In this embodiment, it is applied to the front end. When a browsing operation on a slide to be analyzed is detected, it is determined that there is a need for disease diagnosis based on the analysis results of the slide. The analysis results are used to assist the user in generating a diagnostic result, which includes information such as suspected pathological areas, pathological types, and pathological degrees.

[0048] Therefore, once a user's need is identified, the doctor's priority areas can be further identified, namely the areas of interest in the slide to be analyzed. These areas are where the doctor needs to obtain precise analytical results. For example, in a pathological slide containing a tumor, the doctor's priority areas could be the boundary between the tumor and normal tissue to determine whether the tumor is invasive; or the area surrounding blood vessels and / or nerves to determine whether tumor cells have the characteristic of spreading to these areas. Different pathological slides correspond to different areas of interest, while the same type of pathological slide may have the same areas of interest.

[0049] For ease of understanding, this embodiment provides four optional implementation methods for determining the region of interest.

[0050] In the first method of determining regions of interest, a prior model based on similar pathological slides can be established first. For certain pathological types (such as breast cancer and lung cancer), the system can pre-train a lightweight region importance prediction model and deploy it at the front end. This prediction model can output which regions are most likely to contain key lesions based on the panoramic thumbnail or low-resolution features of the input pathological slide, and identify these regions as regions of interest. For example, after a user loads a new slide, the system first runs this lightweight prediction model to identify high-confidence candidate regions, and then selects these high-confidence candidate regions as regions of interest.

[0051] It should be noted that the prediction model can be pre-trained by annotating the regions of interest on different pathological slides in advance, and then inputting the annotation information and pathological slides into the prediction model so that the prediction model can be iteratively trained based on the input information to form a prediction model with stable output.

[0052] In the second method of determining areas of interest, the system can record the frequency of viewport access, dwell time, zoom level, etc., for each pathological slide across different users and sessions, generating a heatmap of area interest. This heatmap displays the level of interest in each area. For example, for high-interest areas (such as tumor boundaries or ductal regions), the system immediately initiates a background precision calculation task after the user loads the slide and caches the results on the edge or at the end. When the user actually views that area, they can directly retrieve the precise results from the cache without waiting.

[0053] The third method for determining regions of interest involves analyzing the user's browsing behavior when viewing the slice to be analyzed. Users typically browse non-interested areas quickly with short dwell times, while viewing interest areas tends to involve longer dwell times, or even zoom in. Therefore, determining the regions of interest in real-time based on user interactions with the slice allows for accurate identification of these regions, improving the accuracy of region labeling.

[0054] For example, in this embodiment, reference is made to Figure 3 Step S10 includes S11~S12: Step S11: Monitor the user's first interactive operation while browsing the slice to be analyzed, and determine the current activity level of the currently browsed area based on the first interactive operation; Existing systems typically employ static strategies, either performing full-map calculations (which are resource-intensive and latency-intensive) or simple viewport analysis (which is low-precision and lacks context). These systems cannot dynamically adjust their calculation strategies based on real-time user interactions (such as browsing speed and areas of interest), resulting in wasted computing resources for high-precision analysis during rapid browsing or low-quality results when users are examining the data closely. Therefore, this application's embodiments design a method to determine highly relevant areas by detecting interactive operations on the slice to be analyzed.

[0055] In one optional implementation, when a user browses the slice to be analyzed, the system records the first interactive operation generated during the browsing process in real time. This first interactive operation includes at least one of the following: viewport movement speed, zoom level change rate, mouse / touch trajectory, and dwell time in the current viewport. The first interactive operation may also include adjusted viewpoint position and viewport coordinates, user-annotated information, and number of visits. Based on the real-time recorded interactive operations, the system generates the activity level of different regions within the slice to be analyzed. The activity level characterizes the user's attention to that region; higher attention corresponds to lower activity, and vice versa.

[0056] For example, when the first interactive operation includes viewport movement speed, zoom level change rate, mouse / touch trajectory, and dwell time in the current viewport, the corresponding activity level can be quantified and generated using the following example scheme: It should be noted that viewport movement speed is used to represent the displacement distance of the field of view on the slice per unit time (e.g., pixels / second), reflecting the user's browsing rhythm; zoom level change rate is used to represent the number and magnitude of zooming in or out per minute, reflecting the intensity of attention to details in highly focused areas; mouse / touch trajectory is used to represent the coordinate sequence of the user's operation path, used to analyze browsing modes (targetless browsing and targeted browsing) and focused areas; current viewport dwell time is used to represent the length of time the user remains stationary within the same field of view, which is a direct basis for determining whether there is deep observation behavior.

[0057] After the first interaction is collected, it is semantically processed to convert it into corresponding behavioral features. Then, a weighted summation operation is performed based on the numerical value of the behavioral features and their corresponding weights to obtain a total weighted score. Finally, the corresponding activity level is generated based on the total weighted score.

[0058] Understandably, different behavioral characteristics correspond to different numerical values. For example, a long browsing time corresponds to a value of 10, while a short browsing time corresponds to a value of 1. Different behavioral characteristics also have different weights. For instance, slow scanning behavior with a dwell time exceeding a preset duration is assigned the highest weight (e.g., 30%), while fast scanning behavior is assigned a lower weight (1%); high-frequency zooming behavior is assigned a relatively high weight (e.g., 25%), and hotspot access behavior is assigned the second-highest weight (e.g., 20%). A higher weighted score indicates lower activity, and vice versa.

[0059] In a specific scenario, when the viewport of a certain area used for initial screening moves quickly, stays for a short time (less than 10 seconds), and has a low zoom level, the system will classify the behavior in that area as "fast scanning behavior." If the user stays in a certain area for more than 30 seconds, accompanied by multiple fine-tuning zooms (e.g., from 20x to 40x), it will be marked as "key analysis behavior." When it detects that the user returns to the same area multiple times in a short period of time (high revisit density), the behavior in that area will be marked as hotspot access behavior. The path entropy value is calculated through mouse trajectory. The more chaotic the path, the higher the entropy value, indicating that the user has not yet locked onto the target, and it will be marked as fast panning behavior; conversely, a low entropy value represents a clear target and efficient interpretation.

[0060] Step S12: When the current activity level is less than or equal to a preset activity level threshold, the current browsing area is designated as the area of ​​interest.

[0061] After determining the current activity level, the current activity level is compared with a preset activity level threshold, and the current browsing area that is less than or equal to the preset activity level threshold is designated as the area of ​​interest.

[0062] Optionally, determining the region of interest based on real-time interactive behavior can be achieved by dividing the slice to be analyzed into multiple regions, determining the activity level of each region based on interactive operations targeting each region, and marking regions with activity levels less than or equal to a preset activity threshold as regions of interest, while marking regions with activity levels higher than the preset activity threshold as regions of non-interest. Alternatively, the region of interest can be determined in real-time by identifying the currently viewed region, collecting interactive operations targeting the currently viewed region in real-time, calculating the total weighted score of the currently viewed region based on the interactive operations, and determining the current activity level of the currently viewed region based on the total weighted score. Then, when the current activity level is less than or equal to the preset activity threshold, the currently viewed region is determined as a region of interest; when the current activity level is higher than the preset activity threshold, the currently viewed region is determined as a region of non-interest.

[0063] In the fourth method of determining the region of interest, it is also possible to determine the region of interest based on real-time interactive behavior and prediction models.

[0064] For example, while the user is still moving, the possible location where he / she might stop is predicted, and the region of interest is determined based on the predicted possible location. The client continuously collects the user's viewport motion trajectory (speed, direction, acceleration), uses Kalman filtering or a lightweight LSTM model to predict the viewport center coordinates 0.5 to 1 second later, and determines the region of interest based on the predicted viewport center coordinates.

[0065] After identifying the region of interest, a precise analysis is performed based on the precise analysis model for scheduling within that region. This ensures that when the user actually stops, the predicted precise analysis results for the region of interest have already been generated and cached in a pre-set cache, achieving "zero-wait" display of precise analysis results.

[0066] It should be noted that the methods for determining the area of ​​interest include, but are not limited to, the above-described implementation methods.

[0067] Step S20: Obtain the rapid analysis results of the region of interest, and output the rapid analysis results at the target location corresponding to the region of interest.

[0068] Furthermore, after determining the region of interest, a rapid analysis result of the region of interest is obtained.

[0069] Reference Figure 4 Step S20 includes steps S21 to S22: Step S21: Generate a quick analysis request based on the region of interest, and send the quick analysis request to the quick analysis model, wherein the quick analysis model is a lightweight model deployed on the front end; Step S22: Receive the fast analysis result from the fast analysis model.

[0070] The dynamic scheduling engine generates a fast analysis request based on the region of interest. The fast analysis request includes a coarse-resolution image of the region of interest, the viewport center coordinates of the region of interest, etc. Then, the fast analysis request is sent to the fast analysis model through the communication connection between the dynamic scheduling engine and the fast analysis model, so that the fast analysis model can perform fast analysis operations, generate fast analysis results for the region of interest, and return the fast analysis results to the client. The front end outputs the received fast analysis results to the location of the region of interest, i.e., the target location.

[0071] It should be noted that the rapid analysis results include a coarse analysis of the area of ​​interest, such as vague lesion boundaries and broad classification labels like "suspected tumor" or "inflammatory area." Experienced doctors can make a preliminary pathological diagnosis based on these coarse analysis results. Understandably, pathologists typically examine pathological slides manually under low magnification to observe the entire slide from a macroscopic perspective, filtering out suspicious lesion areas. They then use high magnification to carefully observe the pathological features of these lesion areas from a microscopic perspective, completing a pathological diagnosis based on more detailed pathological characteristics. In other words, pathologists analyze pathological slides step-by-step, from macroscopic to microscopic and from overall to local analysis. This application utilizes a rapid analysis model to act as a "macroscopic observer," directly pointing out suspicious lesion areas to the doctor, improving observation and diagnostic efficiency. In another optional implementation, the rapid analysis results may also include a candidate lesion list, which includes at least one candidate lesion, its classification label, location, and confidence level, allowing pathologists to make a pathological diagnosis based on the candidate lesion list without requiring manual screening of suspicious lesions.

[0072] Furthermore, after obtaining the rapid analysis results, the target location corresponding to the area of ​​interest is determined, and the rapid analysis results are output at the target location.

[0073] Specifically, the target location of the area of ​​interest can be the location of the candidate lesion, one side of the area of ​​interest, or a location pre-set by the system for outputting analysis results, such as the left side of the current display page showing the slice to be analyzed and the right side showing the analysis results.

[0074] For ease of understanding, the following provides two exemplary schemes for outputting rapid analysis results: In the first exemplary output scheme, the rapid analysis results can be displayed as a semi-transparent overlay on the area of ​​interest. Doctors can click on the overlayed area to view the rapid analysis results above and the structural diagram of the pathological slide below. For example, based on the rapid analysis results, the location and boundary of suspected lesions within the area of ​​interest are determined. The generated lesion boundary is then highlighted and overlaid on the suspected lesion location (e.g., rendered in red). Alternatively, a semi-transparent overlay of a different colored image (e.g., red) can be placed above the suspected lesion area to highlight it, allowing doctors to visually correlate the suspicious area (red area) indicated by the model with the actual tissue morphology.

[0075] In the second exemplary output scheme, the rapid analysis results can also be output in a way that displays a list of candidate lesions on one side of the area of ​​interest. When a doctor clicks on any candidate lesion, that candidate lesion in the area of ​​interest is highlighted (marked in red). Alternatively, the image can be displayed on the left side, where the view automatically pans and zooms to center the area of ​​the clicked candidate lesion and marks it with a bright rectangle. The rapid analysis results of the clicked candidate lesion, such as classification labels and pathological features, can also be output simultaneously, allowing doctors to flexibly view different candidate lesions.

[0076] Understandably, due to the varying complexity of each pathological slide, doctors cannot achieve a faster pathological diagnosis based solely on the output rapid analysis results and the slide itself for some more complex slides. In such cases, it is necessary to output more pathological features for suspicious lesions, allowing doctors to use more precise analysis results to assist in generating a more accurate pathological diagnosis. Meanwhile, rapid analysis results provide valuable preliminary screening results for pathological diagnosis, but these results are essentially probabilistic predictions based on algorithmic models, which may contain false positives and may not fully capture the nuances of a specific case.

[0077] Based on this, after outputting the rapid analysis results, perform the following steps: Step S30: If the browsing time of the area of ​​interest is greater than or equal to a preset time, obtain the accurate analysis result of the area of ​​interest from the preset cache area, wherein the quick analysis result is generated by the front end and the accurate analysis result is generated by the cloud.

[0078] Specifically, based on browsing time, it can be determined that the user is still browsing the area of ​​interest, and more accurate analysis results can be output to the user to improve diagnostic accuracy. Based on browsing time, it can also be determined that the area of ​​interest is the area that the user pays high attention to. That is, the longer the browsing time of the area of ​​interest, the higher the user's attention to it. In this case, more accurate analysis results can be output to the user to meet the user's diagnostic needs.

[0079] Specifically, in order to meet users' diagnostic needs, refer to Figure 5 Step S30 includes steps S31 to S32: Step S31: Generate a precise analysis request based on the region of interest, and send the precise analysis request to the precise analysis model deployed in the cloud. The precise analysis model responds to the precise analysis request, obtains the slice data corresponding to the region of interest, and generates the precise analysis result based on the slice data. Step S32: Receive and cache the precise analysis results fed back by the precise analysis model.

[0080] Optionally, a precise analysis model is used to generate precise analysis results. Compared to rapid analysis results, precise analysis results have higher precision and accuracy. For example, the lesion boundaries output by precise analysis results are clear and sharp, accurately delineating the boundary between lesions and normal tissues. It can also output more detailed pathological classifications, such as specific diagnoses like "carcinoma in situ," "invasive carcinoma," and "benign hyperplasia."

[0081] Specifically, the precise analysis request includes slice data corresponding to the region of interest. This slice data includes a high-resolution analytical image of the region of interest and the precise location of the target within the region. Based on the high-resolution analytical image, more accurate analysis results can be generated. For example, to accurately determine the nature of a lesion, such as differentiating between benign and malignant lesions and determining tumor grade, it is necessary to mimic the meticulous observation of microscopic details such as cell morphology, nuclear features, and tissue structure under high magnification. This high-resolution information is the cornerstone of diagnosing lesions by mimicking high-magnification observation.

[0082] It should be noted that the precise analysis model and the fast analysis model are invoked simultaneously by the dynamic scheduling engine. That is, while the fast analysis model outputs a rapid analysis result and waits for the doctor to browse, the precise analysis model continues to perform precise analysis operations on the area of ​​interest. After detecting that the precise analysis model has returned a corresponding precise analysis result, the precise analysis result can be directly output. In another optional embodiment, it can also be determined whether the doctor is still observing the area of ​​interest. This is done by obtaining the browsing time of the area of ​​interest and comparing it with a preset time. If the browsing time is greater than or equal to the preset time and / or the current window still displays the area of ​​interest, it is determined that the doctor is still observing the area of ​​interest, and the precise analysis result is directly output. If the browsing time is less than the preset time and / or the current window has exited displaying the area of ​​interest, it is determined that the doctor has exited observing the area of ​​interest, and the precise analysis result is cached in a preset cache. Another implementation method is to first cache the precise analysis result in a preset cache after generating it. When it is determined that the doctor is still observing the area of ​​interest, the precise analysis result of the area of ​​interest is retrieved from the preset cache and output at the target location. It should be noted that regardless of whether the precise analysis results are currently output or not, the precise analysis results will be cached in a preset cache area for direct use later.

[0083] Further, after obtaining accurate analysis results, perform the following steps: Step S40: Output the precise analysis result at the target location.

[0084] Specifically, the precise analysis results can be output either by directly replacing the rapid analysis results with the precise analysis results, or by overlaying the precise analysis results with the rapid analysis results and outputting them to the target location.

[0085] In one optional implementation, to ensure a good user browsing experience, refer to... Figure 6 Step S40 includes: Step S41, obtain alignment rules, which are used to define the alignment rules between the fast analysis results of the fast analysis model and the accurate analysis results of the accurate analysis model in terms of spatial location and semantic category; Step S42: According to the alignment rules, perform alignment operations on the precise analysis results and the rapid analysis results. The alignment operations include smooth replacement operations and / or enhancement operations.

[0086] In this embodiment, after obtaining the accurate analysis results, a method of outputting the accurate analysis results in a smooth switching manner is proposed to achieve seamless browsing. Specifically, alignment rules are obtained, which are used to define the alignment rules between the fast analysis results of the fast analysis model and the accurate analysis results of the accurate analysis model in terms of spatial location and semantic category.

[0087] Specifically, the spatial alignment rules are used to characterize the output position of the precise analysis result relative to the rapid analysis result. It can be that the precise analysis result is output at the same position, in which case the output method of the precise analysis result is to replace the rapid analysis result with the precise analysis result. Alternatively, the rapid analysis result and the precise analysis result can be superimposed and output at the same position. In this case, the rapid analysis result and the precise analysis result are on different layers, and the layer of the precise analysis result can be above the rapid analysis result. In this case, the user can click on the same position to view the analysis results of different precision at the same time.

[0088] Semantic category alignment rules are used to ensure semantic consistency between the outputs of the fast analysis model and the precise analysis model. For example, if the semantic category output by the fast analysis model (e.g., "suspected tumor") is the same as or has a parent-child relationship with the category output by the precise analysis model (e.g., "high-confidence tumor") (e.g., "abnormal" → "tumor"), the semantic category alignment rule is to directly overwrite the output of the fast analysis model with the result of the precise analysis model, while inheriting its spatial position. If the two categories differ (e.g., the fast analysis model classifies it as "inflammation," while the precise analysis model classifies it as "tumor"), the semantic category alignment rule is for the system to trigger a difference comparison mechanism, caching the two results side-by-side and highlighting them on the user interface with different colors or layers to alert doctors and avoid misjudgments due to automatic overwriting.

[0089] In a special case, for scenarios containing multiple detection targets (such as multiple cell clusters), there may be situations where the precise analysis model detects a target but the fast analysis model does not. In this case, the alignment rule is that the system performs instance-level association through an IoU (Intersection over Union) matching algorithm. Specifically, it calculates the IoU between each detection box output by the fast analysis model and all boxes output by the precise analysis model. If the maximum IoU value exceeds a threshold (such as 0.5), it is considered that the two correspond to the same target, and then spatial and semantic fusion is performed. Unmatched fast analysis results are regarded as missed detections or false detections, and the results output by the precise analysis model dominate the final output.

[0090] Optionally, after obtaining the alignment rules, an alignment operation is performed on the precise analysis results and the fast analysis results according to the alignment rules. The alignment operation includes a smooth replacement operation and / or an enhancement operation. For example, the results of the fast analysis model are updated to the results of the precise analysis model by means of smooth animation or direct replacement.

[0091] In an alternative embodiment, refer to Figure 7 , Figure 7The technical architecture diagram of slice analysis is shown. Specifically, it acquires user interaction operations based on the slice to be analyzed, generates the current activity level of the currently viewed area based on the interaction operations, and performs no processing when the current activity level is higher than a preset activity level threshold, or generates a quick analysis request based on the current viewed area, invokes the quick analysis model to generate a quick analysis result for the current viewed area, and outputs the quick analysis result; when the current activity level is less than or equal to the preset activity level threshold, it generates both a quick analysis request and a precise analysis request, outputs the quick analysis result for the user to browse while waiting to receive the precise analysis result output by the precise analysis model, and then outputs the precise analysis result. It should be noted that when the activity level is low, the quick analysis model and the precise analysis model are simultaneously scheduled. While outputting the quick analysis result for the user to browse and waiting to receive the precise analysis result output by the precise analysis model, it monitors whether the current viewport is still in the current viewed area. If so, the precise analysis result is output; if not, the precise analysis result is cached for future use. Understandably, this embodiment adjusts the scheduling strategy based on activity level to balance the contradictions between "response speed", "analysis accuracy" and "computation cost", thereby achieving a seamless experience of "instant initial feedback followed by precise correction".

[0092] In an optional implementation, the slice analysis method further includes: Obtain the runtime load parameters of the precise analysis model; The preset activity threshold is adjusted based on the operating load parameters.

[0093] The slice analysis device in this embodiment also includes a resource scheduling engine, which achieves efficient resource utilization based on reasonable resource allocation, while avoiding computational blockage and affecting the output of analysis results.

[0094] Optionally, the runtime load parameters include at least one of the following: cloud task queue, network status, and cloud computing resources. The cloud task queue is derived from the cloud task manager and includes at least one of the following: current queue task count (pending_count), longest task wait time (max_wait_ms), and average task processing time. The network status is estimated using the current round-trip time (RTT) and current uplink bandwidth. The cloud computing resources can be obtained based on GPU availability (gpu_util).

[0095] Specifically, after obtaining the runtime load parameters, the current comprehensive load score is calculated based on these parameters, and the preset activity threshold is adjusted based on the comprehensive load score. For example, the above indicators are combined into a comprehensive load score L (range 0-100) to characterize the current system busyness level. The calculation formula is as follows: L = w1 * f1(pending_count) + w2 * f2(max_wait_ms) + w3 * f3(gpu_util)+ w4 * f4(RTT) Where f1-f4 are normalization functions that map each index to the range of 0 to 100, and w1-w4 are weighting coefficients. The weighting coefficients can be configured by the system and can be w1=0.35, w2=0.25, w3=0.25, w4=0.15.

[0096] Optionally, after calculating the comprehensive load score based on the operating load parameters, the load level is classified based on the comprehensive load score. For example, when the comprehensive load score is 0-25, the load level is determined to be idle; when the comprehensive load score is 25-50, the load level is determined to be normal; when the comprehensive load score is 50-75, the load level is determined to be busy; and when the comprehensive load score is 75-100, the load level is determined to be overloaded.

[0097] In one optional embodiment, after determining the current load level corresponding to the current comprehensive load score, a resource scheduling strategy is adjusted based on the current load level. The resource scheduling strategy includes adjusting a preset activity threshold. For example, when the load level is overloaded, the preset activity threshold is lowered to dynamically reduce the number of precision analysis tasks input to the precision analysis model; when the load level is idle, the preset activity threshold is raised to dynamically increase the number of precision analysis tasks input to the precision analysis model.

[0098] In another optional embodiment, the resource scheduling strategy may further include processing the current cloud task queue. This processing can involve adjusting the priority of tasks in the queue. For example, when the load level is overloaded, tasks that have been waiting for a long time are prioritized to lower priority, while newly added tasks are prioritized to higher priority, so as to output the analysis results desired by the user in a timely manner based on user needs. The processing can also include task deduplication. If a task for precise calculation of the same area of ​​interest already exists in the queue, it is not submitted again, and the task identifier of the existing task is reused. Alternatively, it can involve task merging. When the user's viewport moves and the area of ​​interest corresponding to an old task is covered by a new area of ​​interest, the old task is marked as "cancellable," and the new task covers a larger area of ​​interest. Finally, it can involve task cancellation. When the current activity level suddenly increases (e.g., rapid dragging begins), a cancellation command is sent directly to the cloud to release the allocated computing resources. It should be noted that the processing methods for the current cloud task queue include, but are not limited to, the methods described above.

[0099] Optionally, refer to Figure 8 , Figure 8The flowchart illustrates the slice analysis method. It acquires user interaction actions based on the slice to be analyzed, generates the current activity level of the currently viewed area based on these actions, and performs no action if the current activity level is higher than a preset activity threshold. Alternatively, it generates a quick analysis request based on the current viewed area, invokes a quick analysis model to generate a quick analysis result for the current viewed area, and outputs the quick analysis result. If the current activity level is less than or equal to the preset activity threshold, it simultaneously generates a quick analysis request and a precise analysis request, outputting the quick analysis result for user viewing while waiting to receive the precise analysis result from the precise analysis model, and then outputting the precise analysis result. While outputting the quick analysis result for user viewing and waiting to receive the precise analysis result from the precise analysis model, it monitors whether the current viewport is still within the currently viewed area, and caches the precise analysis result for future use. When cloud overload is detected, it waits for the analysis result and / or lowers the priority of tasks in the task queue.

[0100] In this embodiment, a rapid analysis model and a precise analysis model are constructed in layers. The rapid analysis model is deployed on the front end, and the precise analysis model is deployed in the cloud. After the area of ​​interest is determined based on interactive operations, different analysis models are simultaneously activated to generate analysis results. The rapid analysis results are output in a timely manner for users to browse. While the user is browsing, the precise analysis model generates precise analysis results. When it is determined that the user is viewing the precise analysis results, the precise analysis results are output in a timely manner. This solves the problem of untimely output of analysis results and improves the accuracy of the analysis results.

[0101] Third Embodiment Based on the above embodiments, referring to Figure 9 Step S31 includes: Step S311: Obtain the rapid analysis results of the region of interest, the rapid analysis results including the target pathological region and / or target pathological features; Step S312: Generate the precise analysis request based on the rapid analysis results.

[0102] In this embodiment of the application, in order to further improve the efficiency of the accurate analysis model in generating accurate analysis results, the rapid analysis results are used as the basis to generate a rapid analysis request, so that when the accurate analysis model receives a rapid analysis request containing rapid analysis results, it can combine the rapid analysis results to perform accurate analysis, thereby improving the efficiency of accurate analysis.

[0103] Understandably, using the results of the fast analysis model as prior input or feature hints for the precise analysis model essentially changes the two levels from "sequential independent execution" to "sequential dependent execution." This allows the precise model to perform calculations based on the fast analysis results from the fast analysis model, significantly improving its inference efficiency. It's important to note that when the fast and precise analysis models are completely independent, the fast analysis model outputs a result (such as a rough lesion bounding box), while the precise model receives the original high-resolution image patch and calculates from scratch. This leads to computational redundancy, as the precise analysis model needs to re-analyze features already identified by the fast model (such as edges, textures, cell nuclear density, etc.). Based on this, if the intermediate features or output results of the fast analysis model can be injected into the precise analysis model as prior information, the search space of the precise analysis model can be reduced. For example, if the fast analysis model has already provided a region of a candidate lesion, the precise model only needs to perform fine segmentation within that region without searching the entire image. In addition, the fast analysis results can also provide an initial feature map for the precise analysis model. For example, the deep features of the fast model can be used as part of the input features of the first layer of the precise model. At the same time, the fast analysis results can also make the precise analysis model pay more attention to the candidate lesion regions already marked by the fast analysis model.

[0104] In one feasible implementation, the fast analysis model outputs several candidate regions (bounding boxes) of the slice to be analyzed, along with the confidence score of each candidate region. The precise analysis model no longer processes the entire slice, but instead performs fine-grained reasoning only on high-resolution local image patches of these candidate regions. Thus, assuming the total pixel area of ​​the candidate regions output by the fast analysis model accounts for only 10% of the viewport, the computational cost of the precise analysis model can theoretically be reduced by 90%. This is suitable for sparse target detection tasks (such as tumor cell clusters and mitotic images). Since most regions in pathological images are normal or irrelevant, the fast analysis model first filters out abnormal regions, and the precise analysis model can then perform precise analysis based on these abnormal regions, reducing computational cost. Optionally, when the fast analysis model outputs candidate regions, the candidate regions need to be expanded outward to a certain extent before being input into the precise analysis model to avoid boundary truncation that could cause the precise model to lose local context.

[0105] In another feasible implementation, a lightweight fast analysis model performs forward propagation on low-resolution slices to extract intermediate layer feature maps. These intermediate feature maps are then upsampled to align their spatial dimensions with the input dimensions of the precise analysis model. Thus, the first layer of the precise analysis model no longer directly convolves from the original pixels but receives a concatenated input. For example, the original high-resolution image patch is combined with the aligned feature map from the fast analysis model, and the precise analysis model continues forward propagation based on this enhanced input. Therefore, the precise analysis model does not need to learn low-level features (such as edges and textures) from scratch but utilizes features already extracted by the fast analysis model, reducing training convergence time and inference computation.

[0106] In another feasible implementation, the precise analysis model is decomposed into multiple cascaded refinement modules. Each module utilizes the output features of the previous level. Specifically, the fast analysis model outputs a coarse segmentation mask or heatmap (with dimensions consistent with the low-resolution viewport). This coarse result is upsampled to high resolution as an initial semantic prior. Based on the initial semantic prior, a medium-precision result is output, and the medium-precision analysis result is output. Then, based on the medium-precision result, the precise analysis result is iteratively generated. This achieves the output of progressively precise analysis results. Users can first see the medium-precision result (within 100ms), and then obtain and output the final high-precision result (precise analysis result) based on background calculations. This reduces the user's waiting time and achieves a smoother distribution of computational load, avoiding computational spikes caused by the precise analysis model calculating the precise analysis result all at once.

[0107] Optionally, to focus more deeply on areas that pathologists are truly interested in, refer to Figure 10 Step S312 includes: Step S313: Obtain the second interactive operation of the user browsing the quick analysis results, and determine the target area of ​​interest based on the second interactive operation; Step S314: Generate a precise analysis request containing the target region of interest.

[0108] Understandably, in clinical practice, the final diagnostic authority rests with the pathologist. Simply relying on automated output cannot meet the clinical requirements for rigor and reliability. Therefore, to achieve more accurate diagnoses, this application's embodiments establish a crucial interactive bridge between automated analysis and the physician's professional judgment. A precise analysis request is generated based on the physician's professional judgment. This request includes the area requiring precise analysis, which is designated as the target region of interest. Specifically, after outputting the rapid analysis results, the physician's interactive data in response to the results is monitored. The physician can provide interactive data in various ways that align with their operational intuition to pinpoint the areas they deem requiring further precise analysis. Furthermore, after outputting the rapid analysis results, the physician's second interactive operation based on the results is monitored, generating physician interaction data. A precise analysis request is then generated based on this data, and the precise analysis results are obtained. This precise analysis request includes the target region of interest generated based on the physician's interactive data. Introducing physician interactive data for precise analysis of the slides improves the accuracy of the analysis.

[0109] In one feasible implementation, the doctor's interactive data includes: the doctor performing zoom operations, click operations, clicking on an item in the candidate lesion list, and inputting text commands in the area of ​​interest after the results of rapid analysis are overlaid.

[0110] Specifically, one interaction method involves the doctor clicking on an item in the output list of candidate lesions with the mouse. For example, if the doctor clicks on candidate lesion 1 with the highest confidence level in the list, this click event is captured by the system.

[0111] Another interaction method allows doctors to operate directly on the overlaid pathology slides, without relying on lists. Doctors can use the mouse to select areas of any shape or size on the image based on their judgment of the overlaid image. For example, a doctor might find an area on the heatmap that is not significantly highlighted in red but has a suspicious shape, and manually select that area. The coordinate information of this selection operation (coordinates of the top-left vertex and width and height) is captured by the system in real time.

[0112] In addition, more advanced text-based command interaction is supported. Doctors can enter commands conforming to medical terminology in the text input boxes of the interaction area, such as "analyze this vascular tumor thrombus." This interaction method can be used in conjunction with the previous two methods. Regardless of the interaction method used by the doctor, their actions (clicking, selecting, entering text) will be parsed by the system as doctor interaction data. This module converts this interaction data into precise coordinate information. If the doctor clicks a list item, the coordinates of the bounding box of the lesion associated with that list item are extracted. If the doctor manually selects a bounding box, the coordinates of the selected area are directly adopted. These coordinates, obtained directly or indirectly from the doctor's interaction, are defined as the coordinates of the confirmed lesion. For example, if the doctor clicks on candidate lesion 1, its corresponding bounding box coordinates under the slice to be analyzed, such as x:10240, y:20480, width:512, height:512, are locked, and candidate lesion 1 is identified as the area requiring further analysis.

[0113] Furthermore, after generating a precise analysis request based on the rapid analysis results of the region of interest and / or doctor interaction data, the precise analysis request is sent to the precise analysis model so that the precise analysis model can generate and feed back the precise analysis results to the front end.

[0114] This application's embodiments improve the efficiency of the precise analysis model by using the rapid analysis results as intermediate features as input. Based on the doctor's actual interaction data, the target area of ​​interest that needs to be focused on is determined. Only the rapid analysis results of the target area of ​​interest are input into the precise analysis model, reducing the amount of data required for precise analysis and further improving the model's efficiency.

[0115] In the embodiments of this application, a cascaded collaboration mechanism can be established between the fast analysis model and the precise analysis model. After the fast analysis model performs a preliminary analysis on the slice to be analyzed and outputs the fast analysis result, the precise analysis model receives the original high-resolution image of the slice to be analyzed and uses the output of the fast analysis model as prior information or guiding features, thereby reducing the search space, avoiding repeated calculation of low-level features, significantly reducing the computational delay of precise inference, and improving the efficiency of outputting analysis results.

[0116] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the slice analysis method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0117] This application provides a slice analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the slice analysis method in Embodiment 1 above.

[0118] The following is for reference. Figure 11 The diagram illustrates a structural schematic of a slice analysis device suitable for implementing embodiments of this application. The slice analysis device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 11 The slice analysis device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0119] like Figure 11 As shown, the slice analysis device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the slice analysis device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the slice analysis device to communicate wirelessly or wiredly with other devices to exchange data. Although slice analysis devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0120] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0121] The slice analysis device provided in this application, employing the slice analysis method described in the above embodiments, can solve the technical problem of untimely output of slice analysis results. Compared with the prior art, the beneficial effects of the slice analysis device provided in this application are the same as those of the slice analysis method provided in the above embodiments, and other technical features of this slice analysis device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0122] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0124] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the slice analysis method in the above embodiments.

[0125] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0126] The aforementioned computer-readable storage medium may be included in the slice analysis device; or it may exist independently and not assembled into the slice analysis device.

[0127] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the slice analysis device, enable the slice analysis device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0130] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described slice analysis method, thereby solving the technical problem of untimely output of slice analysis results. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the slice analysis method provided in the above embodiments, and will not be repeated here.

[0131] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the slice analysis method described above.

[0132] The computer program product provided in this application can solve the technical problem of untimely output of slice analysis results. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the slice analysis method provided in the above embodiments, and will not be repeated here.

[0133] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A slice analysis method, characterized in that, When applied to the front end, the slice analysis method includes the following steps: When a browsing operation on a slide to be analyzed is detected, the region of interest for the slide to be analyzed is determined, and the region of interest is determined according to the pathological type corresponding to the slide to be analyzed. Obtain the rapid analysis results of the region of interest, and output the rapid analysis results at the target location corresponding to the region of interest. The rapid analysis results include candidate lesions and the fuzzy boundaries of the candidate lesions. If the browsing time of the area of ​​interest is greater than or equal to the preset time, the precise analysis result of the area of ​​interest is obtained from the preset cache. The fast analysis result is generated by the front end, and the precise analysis result is generated by the cloud. The precise analysis result includes lesions and clear boundaries of lesions. The precise analysis results are output at the target location.

2. The slice analysis method as described in claim 1, characterized in that, The step of determining the region of interest in the slice to be analyzed includes: Monitor the user's first interactive operation while browsing the slice to be analyzed, and determine the current activity level of the currently browsed area based on the first interactive operation; When the current activity level is less than or equal to a preset activity level threshold, the currently browsed area is designated as the area of ​​interest.

3. The slice analysis method as described in claim 1, characterized in that, The steps for obtaining the rapid analysis results of the region of interest include: A quick analysis request is generated based on the region of interest, and the quick analysis request is sent to the quick analysis model, wherein the quick analysis model is a lightweight model deployed on the front end. Receive the rapid analysis results in response to the rapid analysis model.

4. The slice analysis method as described in claim 1, characterized in that, The step of obtaining the accurate analysis results of the region of interest from the preset cache includes: A precise analysis request is generated based on the region of interest, and the precise analysis request is sent to a precise analysis model deployed in the cloud. The precise analysis model responds to the precise analysis request, obtains the slice data corresponding to the region of interest, and generates the precise analysis result based on the slice data. Receive and cache the precise analysis results fed back by the precise analysis model.

5. The slice analysis method as described in claim 4, characterized in that, The step of generating a precise analysis request based on the region of interest includes: Obtain rapid analysis results for the region of interest, including the target pathological region and / or target pathological features; The precise analysis request is generated based on the rapid analysis results.

6. The slice analysis method as described in claim 5, characterized in that, The step of generating the precise analysis request based on the rapid analysis results includes: Obtain the second interactive operation of the user when browsing the quick analysis results, and determine the target area of ​​interest based on the second interactive operation; Generate a precise analysis request that includes the target region of interest.

7. The slice analysis method as described in claim 1, characterized in that, The step of outputting the precise analysis result at the target location includes: Obtain alignment rules, which are used to define the alignment rules between the fast analysis results of the fast analysis model and the accurate analysis results of the accurate analysis model in terms of spatial location and semantic category; Based on the alignment rules, an alignment operation is performed on the precise analysis results and the rapid analysis results. The alignment operation includes a smooth replacement operation and / or an enhancement operation.

8. The slice analysis method according to any one of claims 1-7, characterized in that, The slice analysis method also includes: Obtain the runtime load parameters of the precise analysis model; Adjust the preset activity threshold based on the aforementioned operating load parameters.

9. A slice analysis device, characterized in that, The slice analysis device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the slice analysis method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the slice analysis method as described in any one of claims 1 to 8.

Images