Pathology image transmission method, device, and storage medium

Abstract

This application discloses a method, device, and storage medium for transmitting pathological images. The method relates to the field of image processing technology. The method includes: receiving a view request from a client for a target digital pathological whole-slice image, the view request including coordinate position and zoom level; determining a window image within the target digital pathological whole-slice image selected by the coordinate position and zoom level; inputting the window image into a preset analysis model to generate a diagnostic importance heatmap; dividing the window image into image blocks based on the diagnostic importance heatmap and the network status from the server to the client; marking the diagnostic importance level of each image block according to the diagnostic importance heatmap; obtaining the client's network status; and assigning a transmission control strategy to each image block based on the network status and diagnostic importance level; processing each image block according to the transmission control strategy and sending it to the client. This application can achieve the technical effect of improving the efficiency of remote pathological diagnosis.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device and storage medium for transmitting pathological images. Background Technology

[0002] Currently, the main method for cross-hospital pathology consultations is to view pathology images on the client side. Specifically, the client requests the corresponding image tile from the server based on the current window position. However, pathology images are generally digital whole-slice images, characterized by high resolution and therefore large image data volume. Therefore, in low-bandwidth or high-latency networks, each user interaction requires waiting for the network to transmit new tiles round trip, resulting in severe interaction lag and impacting diagnostic efficiency. Summary of the Invention

[0003] The main objective of this application is to provide a method, device, and storage medium for transmitting pathological images, aiming to solve the technical problem of low efficiency in remote pathological diagnosis.

[0004] To achieve the above objectives, this application provides a pathological image transmission method applied on a server side. The pathological image transmission method includes: Receive a view request from the client for the target digital pathology whole slide image. The view request includes the coordinate position and zoom level. Determine the coordinate position and zoom level of the selected window image in the target digital pathology whole slice image; Input the window image into the preset analysis model to generate a diagnostic importance heatmap. The diagnostic importance heatmap is a grayscale image or probability image that quantifies the diagnostic importance of each position in the window image. Based on the diagnostic importance heatmap and the network status from server to client, the window image is divided into image blocks. According to the diagnostic importance heatmap, the diagnostic importance level of each image block is marked. The network status of the client is obtained. Based on the network status and the diagnostic importance level, a transmission control strategy is assigned to each image block. The diagnostic importance level is a discrete level based on the diagnostic importance heatmap. The transmission control strategy includes encoding parameters, transmission priority and error control scheme. According to the transmission control strategy, each image block is processed and sent to the client.

[0005] In one embodiment, processing each image block and sending it to the client according to a transmission control policy includes: Generate transmission scheduling instructions based on the transmission control strategy; According to the transmission scheduling instructions, each image block is encoded based on the diagnostic importance level to obtain the encoded image block; Based on the transmission priority, the encoded image blocks and their corresponding diagnostic importance levels are encapsulated into data packets and sent to the client.

[0006] In one embodiment, the pathological image transmission method further includes: Based on continuous view requests, the user's current browsing position and movement trend on the client are analyzed, including movement direction and movement speed. Predict the candidate block to be viewed next and send the candidate block to the client in the background.

[0007] In one embodiment, predicting the candidate block to be viewed next includes: Real-time prediction is performed using a sector as the prediction area. The central axis of the sector is determined based on the direction of movement, and the radius of the sector is determined based on the speed of movement. For each image patch within the fan-shaped prediction region, calculate its average heatmap value based on the diagnostic importance heatmap; Image blocks with an average heatmap value greater than a preset threshold are marked as candidate blocks.

[0008] Furthermore, to achieve the above objectives, this application also provides a pathological image transmission method applied to a client. The pathological image transmission method includes: In response to the selection event of the image viewing control, determine the target digital pathology whole slide image, coordinate position, and zoom level corresponding to the selection event; Generate and send a view request to the server based on the target digital pathology whole slide image, coordinate position, and zoom level; The server receives encoded image blocks and diagnostic importance levels for each image block in response to the view request. The diagnostic importance levels are discrete levels based on the diagnostic importance heatmap, which is a grayscale or probability map that quantifies the diagnostic importance of each location in the window image. Based on the image patches and their diagnostic importance levels, the images are decoded and stitched together to form the target image. The image viewing control renders the target image.

[0009] In one embodiment, the pathological image transmission method further includes: The client receives candidate blocks sent by the server and stores them in the client's cache. Candidate blocks are image blocks whose heatmap values ​​are greater than a preset threshold. The heatmap values ​​are the quantized values ​​of the image blocks based on the diagnostic importance heatmap. When a user browses to the area where a candidate block is located, the candidate block is loaded from the cache.

[0010] In addition, to achieve the above objectives, this application also provides a pathological image transmission device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pathological image transmission method described above.

[0011] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the pathological image transmission method described above.

[0012] This application provides a method for transmitting pathological images. First, it receives a view request from a client for a target digital pathological whole-slice image, the view request including coordinate position and zoom level. Then, it determines a window image within the target digital pathological whole-slice image, bounded by the coordinate position and zoom level. The window image is input into a preset analysis model to generate a diagnostic importance heatmap. Based on the diagnostic importance heatmap and the network status from the server to the client, the window image is divided into image blocks. According to the diagnostic importance heatmap, the diagnostic importance level of each image block is marked. The client's network status is obtained, and a transmission control strategy is assigned to each image block based on the network status and diagnostic importance level. According to the transmission control strategy, each image block is processed and sent to the client. By locating the window image through the view request, generating the diagnostic importance heatmap, and formulating a differentiated transmission strategy based on the network status, high-diagnostic-value image blocks are prioritized for processing and transmission. This ensures high-definition, low-latency presentation of key areas under low bandwidth conditions, reducing interaction stuttering and waiting time, and effectively improving the efficiency of remote pathological diagnosis. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the overall architecture of the pathological image transmission method of this application; Figure 2 This is a flowchart illustrating an embodiment of the pathological image transmission method of this application. Figure 3 This is a schematic diagram of a window image provided in Embodiment 1 of the pathological image transmission method of this application; Figure 4 This is a schematic diagram of the diagnostic importance heatmap provided in Embodiment 1 of the pathological image transmission method of this application; Figure 5 This is a schematic diagram illustrating the effect framework of the pathological image transmission method in Embodiment 1 of this application; Figure 6This is a schematic diagram illustrating the effect of the pathological image transmission method in Embodiment 1 of this application; Figure 7 This is a schematic diagram of the data packet sending queue provided in Embodiment 3 of the pathological image transmission method of this application; Figure 8 A schematic flowchart illustrating the pathological image transmission method provided in this application embodiment; Figure 9 This is a schematic diagram of the module structure of the control device of the tracking camera according to an embodiment of this application.

[0016] Explanation of icon numbers: 11. Image viewing control on the client side; 12. Target digital pathology whole slide image; 13. Window image; 14. Image patch; 15. Diagnostic importance heatmap; 16. Corresponding area of ​​image patch 14 in the diagnostic importance heatmap; 17. High importance image patch.

[0017] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0019] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0020] Currently, the main method for cross-hospital pathology consultations is to view pathology images on the client side. Specifically, the client requests the corresponding image tile from the server based on the current window position. However, pathology images are generally digital whole-slice images, characterized by high resolution and therefore large image data volume. Therefore, in low-bandwidth or high-latency networks, each user interaction requires waiting for the network to transmit new tiles round trip, resulting in severe interaction lag and impacting diagnostic efficiency.

[0021] The main solution of this application is as follows: Receiving a view request from a client for a target digital pathology whole-slice image, the view request including coordinate position and zoom level; determining the window image selected by the coordinate position and zoom level within the target digital pathology whole-slice image; inputting the window image into a preset analysis model to generate a diagnostic importance heatmap; based on the diagnostic importance heatmap and the network status from the server to the client, dividing the window image into image blocks, marking the diagnostic importance level of each image block according to the diagnostic importance heatmap, obtaining the client's network status, and assigning a transmission control strategy to each image block based on the network status and diagnostic importance level; processing each image block according to the transmission control strategy and sending it to the client. By locating the window image through the view request, generating the diagnostic importance heatmap, and formulating a differentiated transmission strategy based on the network status, high-diagnostic-value image blocks are prioritized for processing and transmission, ensuring high-definition, low-latency presentation of key areas under low bandwidth conditions, reducing interaction stuttering and waiting time, and thus improving the technical efficiency of remote pathology diagnosis.

[0022] It should be noted that the executing entity in this embodiment can be a pathological image transmission device, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a pathological image transmission device capable of performing the above functions. This embodiment does not specifically limit it in this way. The following uses a pathological image transmission device as the executing entity as an example to describe this embodiment and the following embodiments.

[0023] First, please refer to Figure 1 , Figure 1 This is a schematic diagram of the overall architecture of the pathological image transmission method of this application. It includes a server and a client, which achieve intelligent collaboration through bidirectional data flow. On the server side, digital pathological whole-slice images are first stored and managed, and then processed in parallel by a lightweight AI analysis module to generate a diagnostic importance heatmap. Subsequently, the transmission control engine implements differentiated scheduling based on the heatmap and client requirements, guiding the image encoder to perform priority encoding. Finally, the network transceiver module completes the intelligent distribution of data. On the client side, the user initiates a view request through the interactive interface. After the network transceiver module receives the data, it is processed by the decoder. At the same time, a metadata-based rendering engine is used to perform layered rendering using the diagnostic importance level information transmitted by the encoder. Combined with a local caching mechanism, this achieves the visual effect of priority decoding and high-definition display of core lesion areas, thereby achieving efficient and accurate remote pathological diagnosis support.

[0024] Based on this, Embodiment 1 of this application proposes a method for transmitting pathological images. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the pathological image transmission method of this application. The pathological image transmission method includes steps S10-S50: Step S10: Receive a view request for the target digital pathology whole slide image sent by the client. The view request includes coordinate position and zoom level.

[0025] In this embodiment, a whole-slide digital pathology image (WSI) refers to a high-resolution digital pathology slide image. These images contain a massive amount of data, typically ranging from several gigabytes to tens of gigabytes, and include everything from low-magnification overviews to cellular-level details under high magnification. A view request is a standard request initiated by a client, such as a doctor's browser, to the server to obtain a specific local area of ​​the target whole-slide digital pathology image for display. The coordinate position is a key parameter in the view request, represented by (x, y). The zoom level, represented by z, is another key parameter included in the view request and defines the requested magnification or resolution level.

[0026] As an optional implementation, a view request for a target digital pathology whole slide image is received from a client, and the identifier, coordinate position (x, y), and zoom level z of the target digital pathology whole slide image are parsed from the view request.

[0027] Specifically, the client and diagnostic server exchange connection information through a separate signaling server to negotiate and establish a direct peer-to-peer (P2P) connection. With the assistance of the signaling server, a WebRTC (Web Real-Time Communication) peer-to-peer connection is established between the client and diagnostic server. The client sends its current view request as a media request or via a data channel to the diagnostic server, which captures this view request through its VideoTrackSource interface. This enables low-latency, high-quality real-time streaming of critical diagnostic regions in pathological images under low-bandwidth network conditions.

[0028] As an alternative implementation, the server receives two view requests from the client.

[0029] Specifically, the initial view request retrieves the lowest resolution overview map and its corresponding overview heatmap of the target WSI. Upon receiving the request, the server directly retrieves two pre-generated lowest resolution overview maps (typically the highest downsampled version of the entire WSI) and their corresponding overview heatmaps from preprocessed storage. These heatmaps are diagnostic importance heatmaps of the same resolution, generated after analyzing the aforementioned overview maps. The server then combines this overview map with the overview heatmaps... Figure 1The data is then sent to the client. The second progressive transfer view request is initiated by the client after the user has circled the area of ​​interest on the overview map. The client sends a request to the server for that area, including its coordinates on the overview heatmap in the HTTP request header. By pre-calculating on the server, high-definition images are loaded quickly, significantly reducing the server's real-time load. This is suitable for large-scale, rapid screening scenarios where real-time performance requirements are not high.

[0030] Step S20: Determine the window image with coordinate position and zoom level in the target digital pathology whole slice image.

[0031] In this embodiment, the window image refers to the specific image region extracted from the target digital pathology whole slide image based on the coordinate position (x,y) and zoom level z.

[0032] As an optional implementation, the window image is determined by using the coordinate position (x,y) as the center point reference and scaling according to the scaling level z.

[0033] Specifically, the view request is parsed to extract the target WSI identifier, coordinate position (x, y), and zoom level z. If the viewport image display size is a preset fixed value, the viewport boundary coordinates are calculated using the coordinate position (x, y) as the viewport center point reference. After scaling according to the zoom level z, a rectangular crop is performed along the viewport boundary to obtain a fixed-size viewport image. If the viewport image display size is a dynamically variable value, the viewport image is scaled according to the zoom level z, using the coordinate position (x, y) as the viewport center point reference. By determining the viewport image based on the coordinate position and zoom level, a more specific viewport image can be quickly extracted.

[0034] As another alternative implementation, the corresponding region of the pre-stored high-resolution full-map heatmap is read based on the coordinate information of the region of interest on the overview heatmap in the progressive transport view request.

[0035] Specifically, after the server parses the progressive transport view request to obtain the coordinates of the region of interest on the overview heatmap, it performs coordinate mapping based on the coordinate information in the request, according to the hierarchical relationship between the current lowest and highest resolution. The overview map is a low-resolution pyramid-shaped image, which is generated by regularly and uniformly downsampling the high-resolution image, for example, taking 1 pixel out of every 128 pixels. For example, the server generates two key levels of overview maps during preprocessing: Level 0 is the lowest resolution overview layer, obtained by downsampling the target WSI by 1:128, meaning that every 128x128 pixel block in the original image is merged into 1 pixel in the overview map; Level N is the highest resolution or full resolution layer, meaning that the target WSI is sampled at a 1:1 resolution. The scaling factor from the full resolution layer to the overview layer is 128, meaning that 1 pixel in the overview map corresponds to a 128x128 pixel area in the full resolution map. The specific coordinates of the region of interest on the overview heatmap are multiplied by a scaling factor corresponding to the full resolution. The resulting coordinates are the corresponding location in the high-resolution heatmap. Based on these coordinates, the corresponding region of interest is retrieved from the pre-stored high-resolution full-map heatmap. By pre-processing all time-consuming calculations, only rapid coordinate conversion and file retrieval are needed upon user request, reducing the real-time computing load on the server and enabling the region of interest to be obtained in a very short time after a request is initiated.

[0036] Step S30: Input the window image into the preset analysis model to generate a diagnostic importance heatmap. The diagnostic importance heatmap is a grayscale image or probability map that quantifies the diagnostic importance of each position in the window image.

[0037] In this embodiment, the preset analysis model refers to an artificial intelligence model, such as a lightweight convolutional neural network, deployed on the server side for real-time analysis of pathological images and generation of diagnostic importance heatmaps. The diagnostic importance heatmap refers to a grayscale or probability map generated by the preset analysis model that quantifies the diagnostic importance of each location in the input window image space. The value of each pixel or region in the diagnostic importance heatmap is a continuous value between [0,1]. The higher the value, the greater the probability that the location contains key diagnostic features, and the higher the diagnostic importance.

[0038] As an optional implementation, the size of the window image is adjusted and then input into a preset analysis model to generate a diagnostic importance heatmap.

[0039] Specifically, the window image is downsampled to a scale suitable for real-time analysis; for example, the window image is scaled to a preset size of 512x512 and input into a preset analysis model deployed on a server GPU, such as a lightweight ResNet-18 variant model. This model is specifically trained for a particular disease, such as breast pathology, to identify key diagnostic features. The preset analysis model performs forward inference and outputs a diagnostic importance heatmap of the same size as the input in a very short time. By scaling the image to a fixed, smaller size, the amount of data that needs to be fed into the AI ​​model for forward computation is reduced, enabling the rapid generation of diagnostic importance heatmaps.

[0040] As an alternative implementation, the server preprocesses the target digital pathology whole slice image into multiple resolution levels, runs a preset analysis model on the image at each level, and generates a corresponding full-image diagnostic importance heatmap.

[0041] Specifically, the server uses digital pathology image processing libraries such as OpenSlide to perform multi-level downsampling on the entire target WSI, generating an image pyramid where each level represents a different scaling level, i.e., resolution. For example, level 0 is full resolution at 1:1, level 1 is downsampled at 1:2, level 2 is downsampled at 1:4, and so on down to the lowest resolution overview image at the highest level, such as 1:128 downsampled. For each resolution level in the pyramid, the server runs a pre-defined AI analysis model to perform forward inference on the entire image at that level, outputting a diagnostic importance heatmap corresponding to the spatial size of the input image. The server associates and stores the generated full-image diagnostic importance heatmap for each level with the image data of the corresponding level. By associating and storing the full-image diagnostic importance heatmap for each level with the image data of the corresponding level, it is convenient to directly read the pre-calculated heatmap data when receiving subsequent requests from clients for a specific region and level, achieving rapid data retrieval.

[0042] Step S40: Based on the diagnostic importance heatmap and the network status from server to client, divide the window image into image blocks. According to the diagnostic importance heatmap, mark the diagnostic importance level of each image block. Obtain the network status of the client. Based on the network status and diagnostic importance level, assign a transmission control strategy to each image block. The transmission control strategy includes encoding parameters, transmission priority, and error control scheme.

[0043] Among them, network status refers to the real-time quality indicators of the network path from the server to the specific client that initiated the request. It mainly includes bandwidth, which is the currently available network transmission rate and determines the amount of coding rate to be allocated; latency refers to the round-trip time of data packets from the server to the client, which affects the speed of interactive response and is one of the factors to be considered in transmission control strategies; packet loss rate refers to the proportion of data packets lost in network transmission and determines the strength of error control protection that needs to be applied.

[0044] In this embodiment, the transmission control engine divides the window image into non-overlapping image blocks of fixed size. Taking the top left corner of the window image as the origin, the entire window image is evenly divided into a regular grid in space. For example, a 512x512 pixel window image is divided into 32x32 pixel blocks, resulting in 16x16=256 image blocks. For each image patch, a region perfectly corresponding to its spatial location and size is found on the diagnostic importance heatmap. All pixel values ​​of the corresponding heatmap region for each image patch are extracted. The average heatmap value of the image patch is calculated using one of the following methods: arithmetic mean, Gaussian weighted average, or quantile average. For example, the arithmetic mean of the heatmap values ​​of all pixels within the corresponding region is calculated by adding the heatmap values ​​of all pixels within that region and dividing by the total number of pixels in that region. For instance, for a 32x32 pixel patch with a total of 1024 pixels, the average value obtained by adding the heatmap values ​​of the 1024 pixels and dividing by 1024 is the average heatmap value of the image patch. This value is normalized to the [0,1] interval, representing the overall diagnostic importance of the image patch. The higher the average heatmap value, the higher the diagnostic importance. The calculated average heatmap value is compared with a preset threshold to assign a discrete diagnostic importance level to the image patch. For example, if the average heatmap value is >0.7, the image patch is marked as high importance; if the average heatmap value is between 0.3 and 0.7, it is marked as medium importance; and if the average heatmap value is <0.3, it is marked as low importance, thus obtaining the diagnostic importance level for each image patch. Simultaneously, the server's network detection module measures the end-to-end network status of the current client in real time, including available bandwidth, round-trip time, and packet loss rate. Based on the importance level list of each image patch and the real-time network status, a transmission control strategy is dynamically assigned to each image patch through a built-in mapping strategy, including encoding parameters, transmission priority, and error control scheme. The coding parameters are determined based on importance level and available bandwidth, assigning different coding qualities. For example, with 800Kbps bandwidth, high-importance blocks are assigned high-quality coding parameters, using VP9 encoding (Video codec 9), with --cq-level=20 (Constant Quality Level), i.e., near-lossless coding. A lower --cq-level value indicates higher coding quality, while a higher value indicates higher compression ratio. Medium-importance blocks use VP9 encoding with --cq-level=32, and low-importance blocks are assigned high-compression coding parameters, using VP9 encoding with --cq-level=45. Transmission priorities are assigned to each image block; for example, high-importance blocks are set to a maximum priority of 3, medium-importance blocks to a priority of 2, and low-importance blocks to a minimum priority of 1.Based on importance level and network packet loss rate, different error control schemes are allocated. For example, for high-importance blocks, 20% FEC (Forward Error Correction) redundancy is enabled, meaning 20% ​​more error correction information is sent; for medium-importance blocks, 10% FEC is enabled; and for low-importance blocks, FEC is not enabled, tolerating packet loss. By allocating high-quality coding bitrate, priority transmission timing, and stronger packet loss resistance to critical diagnostic areas when bandwidth is limited, the image clarity and loading speed of critical areas are improved.

[0045] As an optional implementation, the window image is divided into multiple image blocks, the diagnostic importance level of each image block is determined based on the average heatmap value of each image block, and the transmission control strategy of each image block after window image segmentation is determined based on the diagnostic importance level and the network status from server to client.

[0046] Specifically, the RTP (Real-time Transport Protocol) transmission control module divides the window image into non-overlapping image blocks of fixed size. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a schematic diagram of a window image provided in Embodiment 1 of the pathological image transmission method of this application. For example, a 512x512 window image is divided into 16x16 32x32 pixel blocks. For each image block, a region that completely corresponds to the position and size of that block is found on the diagnostic importance heatmap. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the diagnostic importance heatmap provided in Embodiment 1 of the pathological image transmission method of this application. The average heatmap value of an image block is calculated using one of the following methods: arithmetic mean, Gaussian weighted average, or quantile average. For example, the heatmap values ​​of all pixels in the region are summed, and this sum is divided by the total number of pixels in the region. The result is the average heatmap value of the image block, normalized to the [0,1] interval. The average heatmap value represents the overall diagnostic importance of the image block. After calculating the average heatmap value of each block, its diagnostic importance level is determined according to a preset threshold, and each block is marked as high, medium, or low diagnostic importance. Based on the current network status from the server to the client, according to the diagnostic importance level of each image block, data packets of high-importance blocks are subject to stronger forward error correction redundancy, or an ARQ strategy with shorter retransmission timeouts and more retransmissions is used. For low-importance blocks, a higher packet loss rate can be tolerated, or a no-retransmission strategy can be adopted. By calculating the average heatmap value of each image block and thus determining the diagnostic importance level, a basis is provided for the differentiated transmission control strategy for each image block.

[0047] As another alternative implementation, a transmission control strategy is determined based on the heatmap data of the corresponding region of interest read from the pre-stored full-map diagnostic importance heatmap.

[0048] Specifically, the transmission order of encoded data packets is rearranged. For high-importance portions within the area of ​​interest, high-quality data packets are sent first, while for low-importance portions or portions outside the area of ​​interest, low-quality data packets are sent first. This allows for the acquisition of high-resolution images of suspected lesion areas within seconds, without waiting for the entire high-resolution slice to load. By rearranging the encoded data packets, high-resolution localization and prioritized loading of suspected lesion areas in WSI are achieved, improving the efficiency and user experience of large-scale image rapid screening.

[0049] Step S50: Process each image block according to the transmission control strategy and send it to the client.

[0050] As an optional implementation, each image block is processed to obtain a corresponding data packet, which is then sent to the client according to the sending priority.

[0051] Specifically, a transmission control strategy is determined for image blocks of different importance levels. For example, high-importance blocks use lossless or near-lossless coding parameters, medium-importance blocks use standard coding parameters, and low-importance blocks use high-compression-ratio coding parameters. The encoded image blocks are then packetized, and data packets of higher importance are sent first according to transmission priority. By allocating optimal image quality and transmission priority to critical diagnostic areas within limited network bandwidth, the overall browsing smoothness is ensured while improving the diagnostic clarity and interactive experience of lesions.

[0052] As an alternative implementation, the target digital pathology whole slide image is reorganized and sent to the client in a format that supports quality stratification encoding.

[0053] Specifically, the server uses a format that supports quality-layered encoding, such as JPEG2000. During encoding, the JPEG2000 encoder automatically divides the image into encoded blocks, for example, 64x64 pixels. After encoding, the data of each encoded block at different quality levels is encapsulated into data packets. Each data packet contains the spatial coordinates, resolution level, and quality level of the corresponding encoded block. No differentiated encoding methods are applied to the encoded data packets; only the transmission order is reorganized. High-quality data packets from high-importance areas are prioritized for transmission. For example, the first priority is to send high-importance, high-quality data packets within the area of ​​interest; subsequent priority is to send lower-quality data packets within the high-importance part of the area of ​​interest, as well as data packets from the low-importance part of the area of ​​interest; and the lowest priority is to send data packets outside the area of ​​interest. These data packets are then transmitted to the client via HTTP / 2 streaming. By not performing differentiated processing during the encoding stage and only optimizing at the transmission scheduling layer, high-definition priority loading of suspected lesion areas is achieved without increasing the server's real-time encoding burden, thus improving browsing efficiency in rapid screening scenarios.

[0054] This embodiment provides a method for transmitting pathological images. It generates a diagnostic importance heatmap using a preset analysis model, identifies important regions in the image, and, based on network conditions, assigns differentiated transmission control strategies to image regions of varying importance. Please refer to... Figure 5 , Figure 5 This is a schematic diagram illustrating the effect framework of the pathological image transmission method provided in Embodiment 1 of this application. High-importance image blocks 17 are loaded preferentially, and under the same low bandwidth conditions, higher image clarity and faster loading speed can be obtained within this image block area. Please refer to... Figure 6 , Figure 6 The schematic diagram provided in Embodiment 1 of the pathological image transmission method of this application shows that, under limited network bandwidth, high-definition images of critical diagnostic areas can be prioritized and transmitted quickly, avoiding wasting bandwidth and loading time in non-critical areas, thus improving the efficiency of remote pathological diagnosis.

[0055] Based on any of the above embodiments of this application, Embodiment 2 of this application proposes a method for transmitting pathological images, which can be referred to the above description and will not be repeated hereafter. Based on this, according to the transmission control strategy, each image block is processed and sent to the client, including: Step S51: Generate transmission scheduling instructions according to the transmission control strategy.

[0056] As one implementation method, the encoding parameters, transmission priority, and error control scheme parameters in the transmission control strategy are read, and the transmission parameters of all image blocks are integrated to generate a structured transmission scheduling instruction. This generated transmission scheduling instruction is then used for subsequent encoding of each image block.

[0057] Step S52: According to the transmission scheduling instruction, each image block is encoded based on the diagnostic importance level to obtain the encoded image block.

[0058] In one implementation, the server's image encoder reads transmission scheduling instructions, specifying the importance level of each image block and its corresponding transmission control strategy parameters. For example, high-importance blocks are encoded using high-quality parameters to preserve diagnostic details to the maximum extent; low-importance blocks are encoded using high compression ratio parameters to significantly reduce data volume with acceptable quality loss. Based on the transmission scheduling instructions, image blocks of different diagnostic importance levels are encoded. This differentiated encoding achieves the effect of preserving detail in critical areas and efficiently compressing non-critical areas, meeting diagnostic requirements.

[0059] Step S53: According to the sending priority, the encoded image block and its corresponding diagnostic importance level are encapsulated into a data packet and sent to the client.

[0060] In one implementation, the packetizer encapsulates the encoded image patch data into data packets, specifically RTP packets. During encapsulation, the diagnostic importance level of the image patch in the data packet is added to the packet, such as by writing it into the RTP extension header. The encapsulated data packets are then sent to a transmission queue and sorted according to the transmission priority corresponding to the diagnostic importance level carried in the data packets. Please refer to [reference needed]. Figure 7 , Figure 7 This diagram illustrates the data packet transmission queue provided in Embodiment 3 of the pathological image transmission method of this application. For example, data packets for all high-importance blocks are placed at the head of the queue. The network transmission module retrieves data packets from the priority queue sequentially for transmission, while simultaneously performing importance-aware error control according to scheduling instructions. For instance, FEC redundancy packets are added to data packets in the high-priority queue before transmission. Through priority scheduling, low network transmission latency of diagnostically critical image data is ensured, allowing it to arrive at the client first and improving the real-time response experience during interactive browsing.

[0061] In this embodiment, by using differentiated coding and priority scheduling based on diagnostic importance, the image quality and transmission speed of key diagnostic areas are prioritized under limited network bandwidth, thereby improving the image clarity and smoothness of remote pathological diagnosis.

[0062] Based on any of the above embodiments of this application, Embodiment 3 of this application proposes a pathological image transmission method, which can be referred to the above description and will not be repeated hereafter. In addition, the pathological image transmission method further includes: Step S60: Based on the continuous view requests, analyze and obtain the user's current browsing position and movement trend on the client, including the movement direction and movement speed.

[0063] In one implementation, the server continuously records and caches a sequence of consecutive view requests from the same client, arranged chronologically, obtaining the coordinates (x, y) and timestamp of each view request. By calculating the coordinate difference between the two most recent requests or within a time window, a displacement vector (Δx, Δy) is obtained. The angle between this displacement vector and the positive direction is calculated, and the vector is converted into an angle to determine the user's movement direction. Based on the magnitude of the displacement vector, i.e., the displacement distance, and the corresponding time interval, the user's movement speed is calculated. By transforming discrete user interactions into continuous, quantifiable browsing behavior data, it is possible to determine whether the doctor is observing statically, browsing slowly, or scanning rapidly.

[0064] Step S70: Predict the candidate blocks to be viewed next and send the candidate blocks to the client in the background.

[0065] As one implementation method, candidate blocks for the next viewing are predicted based on the user's current browsing position, movement direction, and movement speed on the client. These candidate block data packets are then sent to the client with a lower priority than real-time window data transmission. By predicting the candidate block results, data that the user is likely to need is delivered locally in advance during network idle time, improving the user experience.

[0066] In this embodiment, by analyzing the user's browsing behavior trends in real time, the system predicts the high diagnostic value areas the user may view next, and uses network idle time to send the image data of these areas to the client in advance. This enables instant image loading when browsing key areas, improving the smoothness of interaction and diagnostic efficiency.

[0067] Based on any of the above embodiments of this application, Embodiment 4 of this application proposes a method for transmitting pathological images, which can be referred to the above description and will not be repeated hereafter. Based on this, the method predicts the candidate blocks to be viewed next, including: Step S71: Perform real-time prediction using a sector as the prediction area, determine the central axis of the sector based on the direction of movement, and determine the radius of the sector based on the speed of movement.

[0068] As one implementation method, starting from the user's current browsing position on the client in the view request, the movement direction is used as the central axis of the fan-shaped area. The movement speed is multiplied by a preset time lookahead window, such as 1 second in the future, to calculate the radius of the fan. A preset angle value is used as the fan angle, such as ±30°, thus forming a fan-shaped prediction area with a certain angle and radius centered on the current direction. This fan-shaped area better reflects the uncertainty of user operations than simple straight line or rectangular predictions, ensuring a high probability of hitting the area the user is about to browse.

[0069] Specifically, when the movement speed is below a preset first speed threshold (i.e., the user's movement speed is close to zero), the coordinate changes in the continuous view request are minimal or nonexistent. At this time, the user may be carefully analyzing a specific area within the current viewport and is not predicting the next candidate block to view. When the movement speed is above the preset first speed threshold but below the preset second speed threshold (i.e., the movement speed is low and the direction is stable, the displacement vector changes smoothly, and the user is in a slow browsing state), a smaller preset time look-ahead window, such as 0.5 seconds, can be selected based on the lower movement speed to calculate the radius of the fan-shaped prediction area. Simultaneously, due to the smooth movement and low probability of sudden directional changes, a narrower fan-shaped angle, such as ±15°, can be used. When the movement speed is above the preset second speed threshold, the movement speed is high, the displacement vector is large, and the direction may change; the user may be browsing the general structure. Based on the higher movement speed, a larger preset time look-ahead window, such as 2 seconds, can be selected to calculate the radius of the fan-shaped prediction area to cover more distant future locations. Since directional control is relatively inaccurate during rapid movement, a wider fan-shaped angle, such as ±45°, is used to cover a larger range of possible turns and avoid prediction omissions.

[0070] Step S72: Calculate the average heatmap value based on the diagnostic importance heatmap for each image patch within the fan-shaped prediction area.

[0071] As one implementation method, for each image patch that falls completely within the sector prediction region, the corresponding diagnostic importance heatmap is queried based on its coordinate position. All pixel values ​​of the corresponding heatmap region for each image patch are extracted, and the average heatmap value of the image patch is calculated using one of the following methods: arithmetic mean, Gaussian weighted mean, or quantile mean, and then normalized to the [0,1] interval. Through the quantitative calculation of the average heatmap value, the criticality of the corresponding image patch is intuitively represented.

[0072] Step S73: Image blocks with average heatmap values ​​greater than a preset threshold are marked as candidate blocks.

[0073] As one implementation, image blocks with average heatmap values ​​greater than a preset threshold, for example, 0.7, are marked as high-value candidate blocks. For candidate blocks within the prediction region, their priority is determined based on their average heatmap value; the higher the average heatmap value, the higher the priority. When the moving speed exceeds a preset second speed threshold (i.e., the moving speed is high), the heatmap screening threshold is appropriately reduced, for example, to 0.5. This allows more potentially important areas to be included in the prefetch range, avoiding missing valuable targets during rapid scanning.

[0074] In this embodiment, the user's browsing trend is captured by the fan-shaped prediction region, and image blocks with high diagnostic value are selected as prefetching candidates, thereby achieving accurate and efficient prefetching of key lesion areas that the user is about to browse.

[0075] Based on any of the above embodiments of this application, Embodiment 5 of this application proposes a method for transmitting pathological images, which can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 3 The pathological image transmission method also includes: Step S100: In response to the selection event of the image viewing control, determine the target digital pathology whole slide image, coordinate position, and zoom level corresponding to the selection event.

[0076] In this embodiment, the image viewing control is a visual interactive component in the client user interface, which allows pathologists to operate, select, and browse digital pathology whole slide images.

[0077] As an optional implementation, the client-side image viewing control captures the user's interactive operations on the pathology image browsing interface in real time, and determines the target digital pathology whole slide image, coordinate position, and zoom level corresponding to the selected event.

[0078] Specifically, for example, a pathologist opens a breast biopsy WSI image in a browser and zooms in to 20x objective view. The client monitors front-end interaction events such as mouse dragging and scrolling in real time, obtaining the corresponding target digital pathology whole-slice image identifier, coordinate position (x, y), and zoom level (z). Through simple interactive actions, the coordinate position and zoom level corresponding to the selected event are quickly determined. This enables real-time analysis and transmission of the doctor's current operating window under low bandwidth conditions, ensuring low-latency, high-quality presentation of critical diagnostic areas.

[0079] As another optional implementation, the user views a low-resolution overview map sent by the server, identifies a region of interest in the overview map, and determines the coordinate position of the region of interest in the lowest resolution overview map.

[0080] Specifically, the overview image is a low-resolution pyramid-level image. This low-resolution image is generated by regularly and uniformly downsampling a high-resolution image, for example, taking one pixel out of every 128 pixels. Users can define a region of interest in the lowest-resolution overview image by clicking and dragging to draw a rectangle, or by using AI for initial analysis. The client determines the coordinates of this region of interest within the lowest-resolution overview image based on the position and size of the defined rectangle. By defining the region of interest on the overview image, the current area is identified as a critical area, allowing for the priority loading of high-diagnostic-value components within that area.

[0081] Step S200: Generate and send a view request to the server based on the target digital pathology whole slice image, coordinate position, and zoom level.

[0082] As an alternative implementation, a view request is generated and sent to the server based on the coordinate position and zoom level of the target digital pathology whole slice image.

[0083] Specifically, through a signaling server, the client establishes a WebRTC peer-to-peer connection with the diagnostic server. Based on the coordinates and zoom level of the target digital pathology whole-slice image, the client generates a view request and sends it to the server. Whenever the view undergoes a significant change, such as a change in zoom level or a panning exceeding a certain threshold, the client automatically generates and sends a new view request containing the aforementioned parameters to the server. By sending view requests in real time, highly interactive, low-latency real-time remote consultations are achieved, ensuring that the area corresponding to the view request receives a fast and high-quality response.

[0084] As another alternative implementation, a progressive transmission request is generated and sent to the server based on the region of interest and its coordinates in the target digital pathology whole slice image.

[0085] Specifically, after identifying the area of ​​interest, the client generates a progressive transmission request (e.g., an HTTP request) for that area, appending the coordinates of the area of ​​interest to the request and sending it to the server. By identifying the area of ​​interest and sending the corresponding request, efficient and rapid location and initial screening are achieved.

[0086] Step S300: Receive the encoded image blocks and the diagnostic importance level of each image block sent by the server in response to the view request. The diagnostic importance level is a discrete level divided based on the diagnostic importance heatmap, which is a grayscale map or probability map corresponding to the window image that quantifies the diagnostic importance of each position.

[0087] As an alternative implementation, the client receives data packets from the server, each data packet containing differentially encoded image patch data and metadata identifying its diagnostic importance level.

[0088] Specifically, the client receives real-time transport protocol data packets from the server through an established WebRTC peer-to-peer connection. The client parses the extended header of each data packet to extract key metadata, which clearly identifies the diagnostic importance level of the corresponding image patch. By prioritizing different data packets, low-latency, high-quality streaming transmission of key diagnostic areas in the current real-time browsing window is achieved.

[0089] As an alternative implementation, data packets are received from an HTTP / 2 stream, wherein the order of the data packets has been reorganized by the server based on the diagnostic importance level derived from a pre-stored heatmap.

[0090] Specifically, the client receives data packets from the server via an HTTP / 2 connection. The server's transmission control engine has reorganized the packet delivery order based on a pre-stored diagnostic importance heatmap, prioritizing the delivery of high-quality layer data packets corresponding to the highest diagnostic importance within the region of interest. In other words, high-quality layer data packets for high-importance areas are sent first. The client does not need to parse additional explicit importance metadata; it decodes and reconstructs strictly according to the order in which the data packets arrive, enabling it to obtain high-resolution data from high-importance areas first, achieving progressive image sharpening.

[0091] Step S400: Decode and stitch the image blocks into a target image based on the image blocks and their diagnostic importance levels.

[0092] As an alternative implementation, the client establishes a decoding queue based on the diagnostic importance level in the metadata, decodes and stitches the data into the target image.

[0093] Specifically, data packets of high-importance image blocks are prioritized for decoding, while data from medium- and low-importance blocks are temporarily uncoded. After decoding, the original pixel data and associated coordinate positions of each image block are obtained. Based on the coordinate position information of each image block, the decoded image blocks are stitched together block by block according to the window layout to construct the complete target image data. For low-importance image blocks that have not yet been decoded, interpolation with adjacent image blocks or preset low-quality placeholders are used for filling. At the same time, an error hiding mechanism is activated for high-importance image blocks that have suffered packet loss during transmission to ensure the integrity of the target image stitching. Decoding is prioritized according to the diagnostic importance level, with high-importance image blocks being decoded first, quickly generating images of the core diagnostic region and significantly shortening the waiting time for obtaining key information.

[0094] As another optional implementation, the data packets are decoded and stitched together into the target image according to the order in which they arrive.

[0095] Specifically, the client receives data packets from the server via an HTTP / 2 connection, sorted in descending order of importance and quality. Therefore, the high-resolution data of the most important areas is decoded first. After decoding, the resolution and coordinates are obtained. The decoder then uses these earliest obtained pixel data, based on their coordinates, to fill the corresponding precise locations in the image buffer. The image of the target area is gradually constructed based on the data arriving first. When the image is initially displayed, the highly important parts are almost completely clear, while other parts are gradually filled and clarified as subsequent data arrives and is decoded. By decoding sequentially, the client can see high-resolution details of suspected lesions within the designated area first, greatly accelerating the initial screening and lesion localization process. This also avoids the complex scheduling overhead of the client, making it suitable for fast browsing scenarios where real-time performance is not critical but efficiency is paramount.

[0096] Step S500: Render the target image using the image viewing control.

[0097] As an optional implementation, the client-side rendering engine acquires the stitched target image data and renders it.

[0098] Specifically, the client-side rendering engine acquires the stitched target image data and matches it with the canvas size, current coordinate position, and zoom level of the image viewing control. The target image is then rendered onto the display canvas of the image viewing control according to the coordinate mapping relationship. Prioritizing the refresh of high-importance areas, it quickly presents key diagnostic information. As medium- and low-importance image blocks are gradually decoded and stitched together, the control's rendering screen is progressively updated, synchronously responding to the panning and zooming interaction commands of the image viewing control. The rendering position and display ratio of the target image are adjusted in real time to keep the control rendering synchronized with user operations. By prioritizing the rendering of high-importance areas, doctors can quickly view lesions and other core diagnostic sites, significantly improving the response speed and efficiency of remote pathology diagnosis.

[0099] In this embodiment, by prioritizing the processing and decoding of key image regions based on their diagnostic importance and rendering the target image, a rapid and clear presentation of lesions is achieved with limited resources, thus optimizing the diagnostic efficiency and experience of remote pathology browsing.

[0100] Based on any of the above embodiments of this application, Embodiment Six of this application proposes a pathological image transmission method, which can be referred to the above description and will not be repeated hereafter. In addition, the pathological image transmission method further includes: Step S600: Receive candidate blocks sent by the server and store the candidate blocks in the client's cache. The candidate blocks are image blocks with heatmap values ​​greater than a preset threshold. The heatmap values ​​are the quantized values ​​of the image blocks based on the diagnostic importance heatmap.

[0101] In one implementation, the client's network module continuously listens for data from the server in the background. Upon receiving candidate blocks predicted by the server based on the user's movement trends, it parses the image block data and coordinates. The client's local cache uses the image block's coordinates as a unique key and stores the decoded image block data locally. This process is completed silently in the background and does not affect the user's current screen display. By transmitting candidate blocks in the background with low priority, network idle time is utilized, without impacting the user experience.

[0102] Step S700: When the user browses to the area where the candidate block is located, the candidate block is loaded from the cache.

[0103] As one implementation method, when a user browses to the area containing a candidate block, a quick query is performed in the client's local cache based on the user's current browsing coordinates. It is determined whether the horizontal and vertical coordinates of the current location fall within the coordinate range of the same candidate block in the cache. If a cached block exists, it is considered a cache hit, and the client immediately reads its data from the local cache and loads it as the image block corresponding to the current coordinates. By pre-stored candidate blocks in the client cache, network round-trip time is eliminated, enabling rapid image loading.

[0104] In this embodiment, by pre-caching high-diagnostic-value image blocks fetched by the server to the client and quickly loading them when the user browses and hits them, the browsing smoothness and diagnostic efficiency are improved.

[0105] For example, to help understand the technical concept or principle of the pathological image transmission method of this embodiment, please refer to... Figure 8 , Figure 8 A flowchart illustrating the pathological image transmission method provided in this application embodiment is shown below: The client sends a request to the server to view a target digital pathology whole-slice image. Upon receiving the request, the server retrieves the corresponding raw image data from storage. The server uses AI algorithms to analyze the image, generating a diagnostic importance heatmap while simultaneously monitoring the current network transmission status. The raw image is segmented into multiple small blocks, and different transmission priorities are assigned to these blocks based on the generated heatmap; for example, core lesion areas have high priority, while background areas have low priority. Based on the network status and the diagnostic importance heatmap, encoding parameters, transmission priorities, and error control schemes are dynamically configured for image transmission, and scheduling instructions are generated. Following the generated scheduling instructions, different compression and encoding strategies are applied to image blocks of different priorities. The encoded image data is queued according to a preset priority order and sent as encapsulated data packets over the network to the client, ensuring that critical pathological details are presented with priority and clarity.

[0106] Specifically, in real-time pathological image transmission scenarios, the server is equipped with a GPU, deploying a lightweight ResNet-18 variant AI model for generating heatmaps, integrating the OpenSlide WSI library, and deeply customizing it based on the WebRTC Native API; the client is a browser-based PWA application. In a network simulation environment, such as 1Mbps bandwidth, 100ms RTT, and 2% packet loss rate, the pathologist clicks and selects on the client, zooming in on the WSI to 20x magnification, triggering a view request. The client establishes a WebRTC peer-to-peer connection with the diagnostic server via a signaling server. The server captures the requested window image, scales it to 512x512 pixels, and sends it to the AI ​​model, generating a diagnostic importance heatmap within 10ms. The transmission control engine divides the window into multiple image blocks, calculates the average heatmap value for each block, and classifies them into high, medium, and low importance levels. Simultaneously, the network detection module reports the currently available bandwidth, such as 800Kbps. The control engine integrates the heatmap and network status, assigning differentiated VP9 encoding parameters, transmission priorities, and FEC redundancy strategies to blocks of different importance levels. Encoded image blocks are encapsulated into RTP packets with importance-based metadata and sent according to priority. Upon receiving the RTP packets, the client decodes and renders high-importance blocks based on the metadata. Furthermore, the intelligent prefetching module, based on the doctor's browsing trends, pre-transmits high-importance blocks along the predicted path to the client cache with low priority. High-importance areas quickly achieve diagnostic-level clarity, while browsing low-importance areas results in slightly blurred initial images but smooth, stutter-free playback.

[0107] Specifically, in non-real-time rapid preliminary screening scenarios, the server preprocesses the entire WSI or a specific magnification level into multiple quality layers, such as downsampling from 1:128 to full resolution (1:1). For each quality layer, an AI model is pre-run to generate a full-image diagnostic importance heatmap, which is stored along with the image data. When a client first requests a WSI, the server first sends the lowest resolution overview image and its corresponding overview heatmap. The pathologist delineates a region of interest on the client's overview image, and the client then sends a progressive transmission request for that region to the server, including the region's coordinates on the overview heatmap in the HTTP request header. Upon receiving the request, the server, without real-time AI calculation, directly reads the corresponding region from the pre-stored high-resolution heatmap and uses quality-layered encoding methods such as JPEG2000. The transmission control engine reorganizes the transmission order of the encoded data stream packets based on the pre-stored heatmap. For highly important parts within the region of interest, high-quality layer data packets are sent first; for less important parts or parts outside the region, low-quality layer data packets are sent first. It can greatly reduce the real-time computing pressure on the server, allowing doctors to obtain high-definition images of suspected lesion areas within seconds, without having to wait for the entire high-resolution slice to load completely.

[0108] This application provides a pathological image transmission device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the pathological image transmission method in Embodiment 1 above.

[0109] The following is for reference. Figure 9 The diagram illustrates a structural schematic of a pathological image transmission device suitable for implementing embodiments of this application. The pathological image transmission device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablets, and in-vehicle terminals, as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The pathological image transmission device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0110] like Figure 9 As shown, the pathological image transmission device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the pathological image transmission device. The processing unit 1001, the read-only memory 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the pathology image transmission device to communicate wirelessly or wiredly with other devices to exchange data. Although a pathology image transmission device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0111] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0112] The pathological image transmission device provided in this application, employing the pathological image transmission method described in the above embodiments, can solve the technical problem of low efficiency in remote pathological diagnosis. Compared with the prior art, the beneficial effects of the pathological image transmission device provided in this application are the same as those of the pathological image transmission device provided in the above embodiments, and other technical features of this pathological image transmission device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0113] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0114] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0115] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the pathological image transmission method in the above embodiments.

[0116] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), or any suitable combination thereof.

[0117] The aforementioned computer-readable storage medium may be included in the pathological image transmission device; or it may exist independently and not be assembled into the pathological image transmission device.

[0118] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the pathological image transmission device, the pathological image transmission device causes the following: It receives a view request from a client for a target digital pathological whole-slice image, the view request including coordinate position and zoom level; determines a window image within the target digital pathological whole-slice image whose coordinate position and zoom level are selected; inputs the window image into a preset analysis model to generate a diagnostic importance heatmap; based on the diagnostic importance heatmap and the network status from the server to the client, it divides the window image into image blocks, marks the diagnostic importance level of each image block according to the diagnostic importance heatmap, obtains the client's network status, and assigns a transmission control strategy to each image block according to the network status and diagnostic importance level; and processes each image block according to the transmission control strategy and sends it to the client.

[0119] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0121] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0122] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described pathological image transmission method, thereby solving the technical problem of low efficiency in remote pathological diagnosis. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the pathological image transmission method provided in the above embodiments, and will not be repeated here.

[0123] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the pathological image transmission method described above.

[0124] The computer program product provided in this application can solve the technical problem of low efficiency in remote pathological diagnosis. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the pathological image transmission method provided in the above embodiments, and will not be repeated here.

[0125] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A method for transmitting pathological images, characterized in that, Applied to the server side, the pathological image transmission method includes: Receive a view request for a target digital pathology whole slide image sent by a client, the view request including coordinate position and zoom level; Determine the window image within the target digital pathology whole slice image, including its coordinate position and zoom level; The window image is input into a preset analysis model to generate a diagnostic importance heatmap, which is a grayscale image or probability image corresponding to the window image that quantifies the diagnostic importance of each position. Based on the diagnostic importance heatmap and the network status from server to client, the window image is divided into image blocks. According to the diagnostic importance heatmap, the diagnostic importance level of each image block is marked. The network status of the client is obtained. According to the network status and the diagnostic importance level, a transmission control strategy is assigned to each image block. The diagnostic importance level is a discrete level based on the diagnostic importance heatmap. The transmission control strategy includes encoding parameters, transmission priority, and error control scheme. According to the transmission control strategy, each image block is processed and sent to the client.

2. The pathological image transmission method as described in claim 1, characterized in that, The step of processing each image block according to the transmission control strategy and sending it to the client includes: Based on the transmission control strategy, a transmission scheduling instruction is generated; According to the transmission scheduling instruction, each image block is encoded based on the diagnostic importance level to obtain encoded image blocks; According to the sending priority, the encoded image block and its corresponding diagnostic importance level are encapsulated into a data packet and sent to the client.

3. The pathological image transmission method as described in claim 1, characterized in that, The pathological image transmission method further includes: Based on the continuous view requests, the user's current browsing position and movement trend on the client are analyzed, and the movement trend includes movement direction and movement speed; Predict the candidate block to be viewed next, and send the candidate block to the client in the background.

4. The pathological image transmission method as described in claim 3, characterized in that, The predicted candidate blocks to be viewed next include: Real-time prediction is performed using a fan-shaped area as the prediction region. The central axis of the fan-shaped area is determined based on the direction of movement, and the radius of the fan-shaped area is determined based on the speed of movement. For each image patch within the fan-shaped prediction region, calculate its average heatmap value based on the diagnostic importance heatmap; Image blocks whose average heatmap value is greater than a preset threshold are marked as candidate blocks.

5. A method for transmitting pathological images, characterized in that, Applied to the client, the pathological image transmission method includes: In response to a selection event of an image viewing control, the target digital pathology whole slide image, coordinate position, and zoom level corresponding to the selection event are determined; A view request is generated and sent to the server based on the target digital pathology whole slide image, coordinate position, and zoom level. The system receives encoded image blocks and diagnostic importance levels of each image block sent by the server in response to the view request. The diagnostic importance levels are discrete levels divided based on a diagnostic importance heatmap, which is a grayscale image or probability map that quantifies the diagnostic importance of each position in the window image. Based on the image patches and their diagnostic importance levels, the images are decoded and stitched together to form a target image. The target image is rendered using the image viewing control.

6. The pathological image transmission method as described in claim 5, characterized in that, The pathological image transmission method further includes: The client receives a candidate block sent by the server and stores the candidate block in the client's cache. The candidate block is an image block whose heatmap value is greater than a preset threshold. The heatmap value is the quantized value of the image block based on the diagnostic importance heatmap. When a user browses to the area where the candidate block is located, the candidate block is loaded from the cache.

7. A pathological image transmission device, characterized in that, The pathological image transmission device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pathological image transmission method as described in any one of claims 1-6.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the pathological image transmission method as described in any one of claims 1-6.

Images