Image transmission method, device, equipment and medium
By loading the pathological image tile with the highest compression rate on the consultation and diagnosis client and preferentially uploading and downloading tile on the transit server, the problem of slow remote pathological image transmission is solved, low-latency image transmission is achieved, and diagnostic efficiency is improved.
Patent Information
- Application Number
- CN202110714914.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-25
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-06-25
AI Technical Summary
The existing remote pathological image transmission speed is slow, which may take dozens of minutes or even longer, affecting the diagnostic efficiency.
Through consultation and diagnosis, the client loads the pathological image tiles and basic information with the highest compression rate, calculates the tiles of the pixel area to be viewed according to the layer-by-layer upward lending method, and uploads and downloads the tiles on the transit server to achieve low latency Image transmission.
It speeds up the transmission speed of pathological images, allows doctors to view and upload images at the same time, reduces waiting time and improves diagnostic efficiency.
Smart Images

Figure CN113613036B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image transmission method, device, equipment and medium. Background Art
[0002] In the field of medical diagnosis, it is usually necessary to take a predetermined size of pathology from the patient's diseased part to make a pathology image. Usually, the pathology image is a very large file with G (Gigabyte, gigabyte) as the unit.
[0003] The existing remote pathology image transmission method is that the pathologist uploads the pathology image to the pathology information server at one time, and then the diagnosing doctor downloads the pathology image to the local for analysis and viewing. Since the pathology image is relatively large, the remote pathology image transmission process may require waiting for tens of minutes or longer to complete the transmission of the pathology image, resulting in the problem of slow pathology image transmission speed. Summary of the invention
[0004] The present invention provides an image transmission method, device, equipment and medium to solve the problem of slow transmission speed of pathological images.
[0005] In order to solve the above problems, an embodiment of the present invention discloses an image transmission method, which is applied to a consultation diagnosis client and includes:
[0006] The consultation diagnosis client loads the tiles of the pathological image with the highest compression rate and the basic information of the pathological image sent by the consultation application client;
[0007] The consultation diagnosis client displays the tile in the pathological image with the highest compression rate;
[0008] Determine the pixel area to be checked selected by the consultation diagnosis client in the pathological image, and calculate the tiles to be checked of the pixel area to be checked in a layer-by-layer upward manner;
[0009] If the tile to be viewed exists in the consultation diagnosis client, the pathological image is directly displayed.
[0010] Optionally, the method further comprises:
[0011] If the consultation diagnosis client does not exist for the tile to be viewed, obtaining the three-dimensional coordinates of the tile to be viewed when the shape is a square, the three-dimensional coordinates including: the number of layers of the tile to be viewed, the horizontal coordinate of the tile to be viewed, and the vertical coordinate of the tile to be viewed;
[0012] Sending the three-dimensional coordinates of the tile to be viewed to the relay server, so that the relay server searches whether the tile to be viewed exists in the relay server according to the three-dimensional coordinates of the tile to be viewed;
[0013] If it does not exist, the relay server sends a priority upload instruction to the consultation application client, and the consultation application client inserts the tile to be viewed into the head of the queue and uploads the tile to be viewed to the relay server first, so that the relay server sends the tile to be viewed to the consultation diagnosis client.
[0014] Optionally, the method further comprises:
[0015] Generating a layer table according to the basic information of the pathological image, wherein the layer table stores the corresponding relationship between the basic information of the pathological sub-image, the number of layers and the reduction rate;
[0016] The step of determining the pixel area to be viewed selected by the consultation diagnosis client in the pathological image and calculating the tiles to be viewed of the pixel area to be viewed in a layer-by-layer upward manner comprises:
[0017] When the consultation diagnosis client is operated to adjust the current zoom-out ratio to the first zoom-out ratio, obtaining the first coordinate of the current position of the mouse in the pixel area to be viewed;
[0018] Obtaining a second coordinate of the pixel area to be viewed according to the first coordinate and the screen resolution;
[0019] Searching the layer table according to the first reduction ratio to obtain the layer corresponding to the first reduction ratio and the second reduction ratio corresponding to the layer;
[0020] Calculate a first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio, and the second reduction ratio;
[0021] Calculate the second pixel corresponding to the first coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio
[0022] A tile to be viewed corresponding to the pixel area to be viewed in a layer is calculated according to the first pixel and the second pixel.
[0023] Optionally, the step of calculating the first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio and the second reduction ratio includes:
[0024] Divide the horizontal coordinate of the first coordinate by the first reduction ratio and multiply by the second reduction ratio to obtain the horizontal coordinate of the first pixel;
[0025] The vertical coordinate of the first coordinate is divided by the first reduction ratio and multiplied by the second reduction ratio to obtain the vertical coordinate of the first pixel.
[0026] Optionally, the step of calculating the second pixel corresponding to the second coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio includes:
[0027] Divide the horizontal coordinate of the second coordinate by the first reduction ratio and multiply by the second reduction ratio to obtain the horizontal coordinate of the second pixel;
[0028] The vertical coordinate of the second pixel is obtained by dividing the vertical coordinate of the second pixel by the first reduction ratio and multiplying by the second reduction ratio.
[0029] Optionally, the step of calculating the tile to be checked of the pixel area to be checked in the layer according to the first pixel and the second pixel comprises:
[0030] Dividing the horizontal coordinate and the vertical coordinate of the first pixel by the set pixel to obtain the starting row and the starting column of the tile;
[0031] Dividing the horizontal coordinate and the vertical coordinate of the second pixel by the set pixel respectively to obtain the end row and the end column of the tile;
[0032] The tile to be viewed corresponding to the pixel area to be viewed in the layer is obtained according to the starting row, the starting column, the ending row, and the ending column.
[0033] Optionally, before the consultation diagnosis client loads the tiles in the pathological image with the highest compression rate and the basic information of the pathological image sent by the consultation application client, the method further includes:
[0034] Layering the basic information of the pathological image according to different reduction ratios to obtain a plurality of pathological sub-images;
[0035] Cutting the pathological sub-image to generate a plurality of tiles;
[0036] Uploading the tiles to a pre-established queue in sequence according to the compression rates of the pathological sub-images from high to low;
[0037] The basic information of the tiles and the pathological images in the queue is uploaded to a transfer server, so that the consultation diagnosis client downloads the basic information of the tiles and the pathological images from the transfer server.
[0038] Optionally, the step of cutting the pathological sub-image to generate a plurality of tiles includes:
[0039] The pathological sub-image is cut according to the set pixels to generate several tiles.
[0040] In order to solve the above problems, an embodiment of the present invention further discloses an image transmission device, which is applied to a consultation diagnosis client and includes:
[0041] A loading module, used to load tiles of the pathological image with the highest compression rate sent by the consultation application client and basic information of the pathological image;
[0042] A display module, used for displaying the tile in the pathological image with the highest compression rate;
[0043] A calculation module, used to determine the pixel area to be checked selected by the consultation diagnosis client in the pathological image, and calculate the tiles to be checked of the pixel area to be checked in a layer-by-layer upward manner;
[0044] The query module is used to directly display the pathological image if the tile to be viewed exists in the consultation diagnosis client.
[0045] In order to solve the above problem, an embodiment of the present invention further discloses an electronic device, including:
[0046] one or more processors; and
[0047] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to perform the image transmission method.
[0048] In order to solve the above problem, an embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the image transmission method. The embodiment of the present invention has the following advantages:
[0049] In the embodiment of the present invention, the consultation diagnosis client preferentially loads tiles in the pathological image with the highest compression rate, that is, preferentially loads tiles in the pathological image with the smallest data volume, so that the original pathological image becomes smaller and the transmission speed is accelerated. In addition, the consultation diagnosis client can download pathological images with other compression rates in the background while viewing the pathological image. In this way, the pixel area to be viewed selected by the consultation diagnosis client in the pathological image is determined according to the doctor's operation, and the tiles to be viewed in the pixel area to be viewed are calculated in a layer-by-layer upward manner; if the tiles to be viewed exist in the consultation diagnosis client, the pathological image is directly displayed. In this way, by displaying a relatively blurred pathological image and continuously downloading tiles according to the doctor's operation, the pathological image is continuously corrected and clarified. In other words, the consultation diagnosis client can download, view, and upload at the same time, thereby achieving low latency and accelerating the transmission speed of the pathological image. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 is a flowchart of the steps of an image transmission method according to Embodiment 1 of the present invention;
[0051] Figure 2 It is a structural block diagram of an image transmission method implemented by the present invention;
[0052] Figure 3 It is a structural block diagram of an image transmission device described in Embodiment 2 of the present invention. DETAILED DESCRIPTION
[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0054] Embodiment 1
[0055] Reference Figure 1 , which shows a flow chart of an image transmission method according to Embodiment 1 of the present invention, the method is applied to a consultation diagnosis client, and specifically includes:
[0056] Step 101: The consultation diagnosis client loads the tiles of the pathological image with the highest compression rate sent by the consultation application client and the basic information of the pathological image.
[0057] In actual applications, the consultation diagnosis client gives priority to loading the pathological images with the highest compression rate sent by the consultation application client through the relay server. In other words, the tiles in the pathological images with the smallest data volume are loaded first. This ensures that the consultation diagnosis client can view the pathological images at any time and reduces the waiting time of the consultation diagnosis client.
[0058] The consultation diagnosis client can also be understood as the consultation accepting party, and the consultation application client can also be understood as the consultation applying party, that is, the user end that uploads the pathological image and initiates the remote consultation application.
[0059] Pathological images are generally used by pathologists to view and analyze pathological images, and are generally two-dimensional images.
[0060] The basic information of the pathological image refers to the resolution of the pathological image, that is, the width and height of the pathological image, that is, the pixels of the width and height of the pathological image.
[0061] In actual applications, the consultation application client can use a pathology scanner to scan the pathology image to obtain the pathology image and basic information of the pathology image, and then the consultation application client sends the obtained pathology image and basic information of the pathology image to the consultation diagnosis client through the relay server.
[0062] In actual applications, after receiving the remote consultation application, the consultation diagnosis client can generate a stratification table based on the basic information of the pathological image, or the consultation application client can generate a stratification table based on the basic information of the pathological image, and then send the stratification table to the consultation diagnosis client through the transfer server. The specific method of generating the stratification table is as follows:
[0063] The basic information of the pathological image is layered according to different reduction ratios to obtain a plurality of pathological sub-images.
[0064] The basic information of the pathological image is layered according to different reduction ratios into several layers, each layer corresponds to a pathological sub-image, thereby obtaining several pathological sub-images.
[0065] The reduction rate generally refers to the multiple by which the pathological image is reduced.
[0066] The pathological sub-image refers to an image obtained after the pathological image is layered, that is, it is a pathological image obtained after the original pathological image is layered.
[0067] In a specific application, the basic information of the pathological image is layered in descending order of reduction ratio, and the layer with the highest reduction ratio is taken as the top layer, and then descends in order. In other words, by layering the basic information of the pathological image, the size of the pathological image is reduced, and then a number of digital pathological sub-images are obtained.
[0068] It should be noted that in actual applications, the present invention is based on the original pathological image and is layered according to the zoom rate. Layering is performed based on the original pathological image, and the image loss rate is minimized. Layering can also be performed based on the layered pathological sub-images, in which case the image loss rate will be higher. The specific layering method can be selected according to actual needs.
[0069] In actual use, the pathological image is obtained by scanning the sample slide with the scanner's magnifying lens to observe more details. For example, the pathological image obtained by scanning the sample slide with the scanner's 40x lens is called the original pathological image. In the consultation, the pathological image that needs to be transmitted is the original pathological image.
[0070] If the reduction rate of a layer is N, the length and width of the pathological sub-image of this layer are reduced to 1 / N of the original pathological image, that is, the area of the pathological sub-image of this layer is reduced to 1 / (N 2 ), a layered embodiment of the present invention is described in detail below, taking the original pathological image with a resolution of 80,000*80,000 and the pathological image being divided into 7 layers as an example.
[0071] The first pathological sub-image is the original pathological image, with a resolution of 80,000*80,000, as layer 7. At this time, the reduction rate is 1, that is, the length and width of the first pathological image are reduced to 1 / 1 of the original pathological image respectively.
[0072] The second pathological sub-image is the sixth layer, and the reduction rate is 2, so the resolution of the second pathological sub-image is 40,000*40,000.
[0073] The third pathological sub-image is used as the fifth layer, and the reduction rate is 4, so the resolution of the third pathological sub-image is 20,000*20,000.
[0074] The fourth pathological sub-image is used as the fourth layer, and the reduction rate is 8, so the resolution of the fourth pathological sub-image is 10,000*10,000.
[0075] The fifth pathological sub-image is the third layer, and the reduction ratio is 16, so the resolution of the fifth pathological sub-image is 5 thousand*5 thousand.
[0076] The sixth pathological sub-image is used as the second layer, and the reduction ratio is 32, so the resolution of the sixth pathological sub-image is 2500*2500.
[0077] The seventh pathological sub-image is used as the first layer, and the reduction ratio is 64, so the resolution of the seventh pathological sub-image is 1250*1250.
[0078] It is determined whether the resolution of the seventh pathological sub-image is greater than the preset pathological image resolution, and if so, the basic information of the pathological image is continued to be layered. If less than, the layering of the pathological image is terminated.
[0079] Among them, the resolution of the pathological image is generally much higher than the screen resolution of the terminal display device. Therefore, the preset pathological image resolution can be set according to the screen resolution of the terminal, or the preset pathological image resolution can be set according to the specific application scenario. For example, the preset pathological image resolution is set to 1500*1500 pixels, or it can be other pixels. The present invention does not make any specific limitations on this.
[0080] After the above stratification, in order to facilitate subsequent use, a stratification table can be generated according to the stratification results. The stratification table is as follows:
[0081] Table 1:
[0082] Basic Information Number of layers Reduction rate 80,000*80,000 7 1 40,000*40,000 6 2 20,000*20,000 5 4 10,000*10,000 4 8 5k*5k 3 16 2500*2500 2 32 1250*1250 1 64
[0083] In a specific application, the pathological image is stratified according to different reduction ratios, divided into several layers, and several pathological sub-images are obtained. Then, the pathological sub-images are stored according to different reduction ratios, different layers, and different pathological sub-images corresponding to the reduction ratios, and saved in a stratification table. That is to say, the stratification table stores the basic information of the pathological sub-images, the number of layers, and the corresponding relationship between the reduction ratios.
[0084] The number of layers generally refers to the number of layers into which the pathological image is divided according to different reduction ratios. Generally, the number of layers of the pathological image is 7, that is, the number of layers may be the 1st to the 7th layers.
[0085] It should be noted that for ease of understanding, the present invention uses the basic information of the pathological image as 80,000*80,000 for explanation. In actual applications, when performing the above calculations, if there is a situation where it cannot be divided evenly, it can be rounded up and then continued to be stratified.
[0086] After the pathological image is layered, the pathological sub-image is cut to generate a number of tiles. The tiles are uploaded to a pre-established queue in order of compression rate of the pathological sub-image from high to low, and the tiles in the queue and the basic information of the pathological image are uploaded to a relay server, so that the consultation diagnosis client downloads the tiles and the basic information of the pathological image from the relay server.
[0087] The pathological sub-image is cut according to the set pixels to generate a number of tiles.
[0088] After the pathological image is layered, the present invention obtains a plurality of pathological sub-images. The plurality of pathological sub-images can be cut in sequence according to the number of layers according to set pixels to generate a plurality of tiles.
[0089] The set pixels are generally cut based on pixels of fixed width and fixed height. Preferably, the predetermined pixels are 256*256 pixels. The following takes 256*256 pixels as an example to illustrate the process of cutting the pathological sub-image.
[0090] In actual application, the pathology slice scanner cuts the pathology slices in order from left to right and from top to bottom, with 256*256 pixels in sequence. In the embodiment of the present invention, the pathology slice scanner cuts the pathology slices in order from left to right and from top to bottom, with 256*256 pixels in sequence, and obtains tiles with the size of the pixels. These tiles are combined to finally obtain a picture of the entire pathology sub-image. The advantage of this is that the transit server or the consultation diagnosis client can obtain a complete pathology image more quickly during the subsequent parsing and splicing process.
[0091] It should be noted that in actual applications, after cutting according to 256*256 pixels, there may be some tiles that do not meet the 256*256 pixel cutting. For those that do not meet the requirements, just keep the remaining tiles and upload them directly to the queue. There is no need to perform white-filling processing to make the remaining tiles meet the 256*256 pixels.
[0092] In actual applications, the consultation client can establish an upload queue at the same time as submitting an application for remote consultation on pathological images. It can also be that the application for remote consultation on pathological images is submitted first and then the upload queue is established. It can also be that the upload queue is established first and then the application for remote consultation on pathological images is submitted. It can also be that the upload queue is established before generating a hierarchical table. The present invention does not impose any specific limitations on this.
[0093] In the present invention, since the pathological image is relatively large, it takes a certain amount of time to upload it after cutting. Therefore, the present invention proposes to establish an upload queue on the consultation client. Since the time for cutting the pathological image is less than the time for uploading the pathological image, the pathological image can be cut while the cut pathological image is uploaded to the queue. In addition, the queue can also realize the function of pre-loading the switched pathological image. In other words, the queue interruption mechanism can also be understood as a jamming mechanism. Specifically, the implementation details of the queue jamming mechanism will be described in detail in the following embodiments.
[0094] In a specific application, tiles received in the pathological sub-image are uploaded to the queue in order from high to low compression rates of the pathological sub-images. The compression rate also reflects the corresponding different layers, that is, the higher the compression rate, the lower the layer.
[0095] The compression rate of the pathological sub-image adopts lossy compression to reduce the resolution of the pathological sub-image. The higher the compression rate, the smaller the pathological sub-image is and the lower the definition is.
[0096] Because the tiles of the pathology sub-images are previously uploaded to the pre-established queue in order of compression rate from high to low, the consultation diagnosis client also loads the pathology sub-images with high compression rate.
[0097] The consultation application client uploads the tiles received in the queue to the transit server according to the first-in-first-out principle, so that the consultation diagnosis client downloads the tiles from the transit server.
[0098] After the relay server receives the tiles uploaded by the consultation application client, the relay server will store them in order of compression rate from high to low, that is, the tiles are stored in the cache space in layers.
[0099] In actual applications, the relay server marks each tile in the received tiles according to the layer number, horizontal coordinate, and vertical coordinate, and stores them in the cache space. For example, 1-1-1 represents the first layer, the horizontal coordinate is 1, and the vertical coordinate is 1.
[0100] Step 102: The consultation diagnosis client displays the tile in the pathological image with the highest compression rate.
[0101] Among them, the pathological image with the highest compression rate is the pathological image after the pathological image is compressed. The definition of the pathological image is low, and the space occupied by the pathological image is also small.
[0102] Since the consultation application client previously transmitted the pathological images in order of the compression rate of the pathological images from high to low, after the consultation diagnosis client is opened, it will prioritize loading the basic information of the pathological images and all tiles in the pathological images with the highest compression rate and display them to the consultation diagnosis client. At the same time, the tiles in the pathological images with other compression rates are downloaded in the background of the consultation diagnosis client and cached in the consultation diagnosis client.
[0103] Step 103: determine the pixel region to be viewed selected by the consultation diagnosis client in the pathological image, and calculate the tiles to be viewed of the pixel region to be viewed in a layer-by-layer upward manner.
[0104] Since the tiles in the pathological image with the highest compression rate previously loaded by the consultation diagnosis client have low clarity, the clear image of the pixel area that the consultation diagnosis client wants to view has not been transmitted yet. At this time, the tiles of the pixel area that the consultation diagnosis client needs to view can be transmitted first, thereby meeting the needs of the consultation diagnosis client.
[0105] The consultation diagnosis client is operating on the pathology image with the highest compression rate. For example, it wants to view the image of a certain pixel area, but the image of the certain pixel area that the consultation diagnosis client wants to view has not been transmitted yet. At this time, the pixel area to be viewed can be selected in the pathology image, and the tiles to be viewed of the pixel area to be viewed can be calculated in a layer-by-layer upward manner, so that the consultation diagnosis client gives priority to transmitting the tiles to be viewed of the pixel area to be viewed, thereby meeting the needs of the consultation diagnosis client.
[0106] Step 103 includes the following sub-steps:
[0107] Sub-step 1031: when the consultation diagnosis client is operated to adjust the current zoom-out ratio to the first zoom-out ratio, the first coordinate of the current position of the mouse in the pixel area to be viewed is obtained.
[0108] When the consultation diagnosis client is operated to adjust the current zoom ratio to the first zoom ratio, it means that the consultation diagnosis client wants to view certain pixel areas of the pathological image. At this time, when the user operates the mouse to move and click on the pathological image, the consultation diagnosis client obtains the coordinate information of the current position of the mouse, and thus can obtain the first coordinate of the current position of the mouse in the pixel area to be viewed.
[0109] Sub-step 1032: Obtain second coordinates of the pixel area to be viewed according to the first coordinates and the screen resolution.
[0110] When the consultation diagnosis client is viewing the pathological image with the highest compression rate, when using the mouse to zoom in or out on a certain position in the pathological image, it is necessary to obtain the first coordinate and the second coordinate of the current position of the mouse in the pixel area to be viewed, which is equivalent to determining the pixel area that the consultation diagnosis client needs to view. Generally, the first coordinate is the coordinate of the pixel area to be viewed in the upper left corner of the pathological image, and then the second coordinate can be obtained according to the screen resolution, that is, the second coordinate is the coordinate of the pixel area to be viewed in the lower right corner of the pathological image, so that the pixel area to be viewed is determined, but the tile to be viewed corresponding to the pixel area to be viewed has not been transmitted yet, so it is necessary to find the tile of the pathological image close to the pixel area by looking up the layered table to obtain the tile to be viewed of the pixel area to be viewed.
[0111] Sub-step 1033: searching the layer table according to the first reduction ratio to obtain the layer corresponding to the first reduction ratio and the second reduction ratio corresponding to the layer.
[0112] Since the layer table has been generated before and saves the corresponding relationship between the basic information of the pathological sub-image, the number of layers and the reduction ratio, the layer table can be searched through the first reduction ratio. If the first reduction ratio exists, the basic information of the layer corresponding to the first reduction ratio and the pathological image corresponding to the layer can be directly obtained.
[0113] If the first reduction ratio does not exist in the hierarchical table, find the reduction ratio closest to the first reduction ratio, and use the method of borrowing from the upper layer to find the second reduction ratio closest to the first reduction ratio. In other words, the smallest reduction ratio in the hierarchical table that is larger than the first reduction ratio is used as the second reduction ratio.
[0114] Sub-step 1034: Calculate the first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio, and the second reduction ratio.
[0115] Sub-step 1035: Calculate the second pixel corresponding to the first coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio.
[0116] Since the first reduction ratio does not exist in the layered table, the first reduction ratio is to find the closest second reduction ratio. Therefore, the first coordinate and the second coordinate of the pixel area to be viewed need to be converted to obtain the first pixel and the second pixel corresponding to the second reduction ratio.
[0117] Sub-step 1036: Calculate the tile to be viewed in the layer corresponding to the pixel area to be viewed according to the first pixel and the second pixel.
[0118] As one implementation, sub-step 1034 includes:
[0119] The horizontal coordinate of the first coordinate is divided by the first reduction ratio and multiplied by the second reduction ratio to obtain the horizontal coordinate of the first pixel.
[0120] The vertical coordinate of the first coordinate is divided by the first reduction ratio and multiplied by the second reduction ratio to obtain the vertical coordinate of the first pixel.
[0121] As one implementation, sub-step 1035 includes:
[0122] The horizontal coordinate of the second coordinate is divided by the first reduction ratio and multiplied by the second reduction ratio to obtain the horizontal coordinate of the second pixel.
[0123] The vertical coordinate of the second pixel is obtained by dividing the vertical coordinate of the second pixel by the first reduction ratio and multiplying by the second reduction ratio.
[0124] Further, sub-step 1036 includes:
[0125] The horizontal coordinate and the vertical coordinate of the first pixel are divided by the set pixel to obtain the starting row and the starting column of the tile.
[0126] The horizontal coordinate and the vertical coordinate of the second pixel are divided by the set pixel to obtain the end row and the end column of the tile.
[0127] The tile to be viewed corresponding to the pixel area to be viewed in the layer is obtained according to the starting row, the starting column, the ending row, and the ending column.
[0128] Step 104: If the tile to be viewed exists in the consultation diagnosis client, the pathological image is directly displayed.
[0129] Step 105: If the consultation diagnosis client does not exist for the tile to be viewed, the three-dimensional coordinates of the tile to be viewed when the shape is a square are obtained.
[0130] The square may be a square or a rectangle, and the three-dimensional coordinates include: the layer number of the tile to be viewed, the horizontal coordinate of the tile to be viewed, and the vertical coordinate of the tile to be viewed;
[0131] Step 106: Send the three-dimensional coordinates of the tile to be viewed to the relay server, so that the relay server searches whether the tile to be viewed exists in the relay server according to the three-dimensional coordinates of the tile to be viewed.
[0132] Step 107: If it does not exist, the relay server sends a priority upload instruction to the consultation application client, and the consultation application client inserts the tile to be viewed into the head of the queue, and uploads the tile to be viewed to the relay server in priority, so that the relay server sends the tile to be viewed to the consultation diagnosis client.
[0133] In the embodiment of the present invention, the consultation diagnosis client preferentially loads tiles in the pathological image with the highest compression rate, that is, preferentially loads tiles in the pathological image with the smallest data volume, so that the original pathological image becomes smaller and the transmission speed is accelerated. In addition, the consultation diagnosis client can view the pathological image while the background is downloading pathological images with other compression rates. In this way, the pixel area to be viewed selected by the consultation diagnosis client in the pathological image is determined according to the doctor's operation, and the tiles to be viewed of the pixel area to be viewed are calculated in a layer-by-layer upward manner; if the tiles to be viewed exist in the consultation diagnosis client, the pathological image is directly displayed. In this way, by displaying a relatively blurred pathological image and continuously downloading tiles according to the doctor's operation, the pathological image is continuously corrected and clarified. In other words, the consultation diagnosis client can download, view, and upload at the same time, thereby achieving low latency and accelerating the transmission speed of the pathological image.
[0134] In order for those skilled in the art to better understand the solution of the present invention, Figure 2 Taking the example, the image transmission scheme of the present invention is described in detail.
[0135] At 12:00:00, the doctor of the consultation application client selects a pathology image file in the desktop application. The program identifies and parses the pathology image file to obtain the basic information of the pathology image (80,000*80,000), and at the same time, it is layered according to 7 layers to calculate the pathology sub-images corresponding to each layer. The reduction rates corresponding to the 7 layers are (7-1X, 6-2X, 5-4X, 4-8X, 3-16X, 2-32X, 1-64X), and the generated layered table is as shown in Table 1 in Example 1.
[0136] Among them, the first layer is the layer with the highest compression rate, and the compression rate decreases with the increasing number of layers, that is, the seventh layer is the layer with the lowest compression rate. The meaning of 7-1X is: the reduction factor corresponding to the seventh layer is 1, and the meaning of 6-2X is: the reduction factor corresponding to the sixth layer is 2. The meanings of other layers can be deduced by referring to the above meanings.
[0137] About 2 seconds later, at 12:00:02, the doctor of the consultation application client uploads the patient information, basic information of the pathology image, and the first layer, which is the tile of the pathology image with the highest compression rate, to the queue pre-established by the consultation application client.
[0138] According to the actual network situation, about 3 seconds later, at 12:00:05, the tile is transmitted from the consultation application client to the relay server. After receiving the tile, the relay server stores the received tile according to the layer (which can also be understood as the compression rate), notifies the diagnosis doctor of the consultation diagnosis client, opens the storage space, and starts receiving and caching the tiles uploaded by the consultation application client.
[0139] About 5 seconds later, at 12:00:10, the diagnosing doctor receives the remote diagnosis request and opens the software to view the pathological image. The consultation diagnosis client downloads and displays the patient information together with the basic information of the pathological image and the tiles in the first layer, which is the layer with the highest compression rate. At this time, the consultation diagnosis client asynchronously downloads the tiles of other layers in the background in the order of compression rate from high to low (layers 2-7).
[0140] When the doctor's zoom ratio is reduced from 1 to 13.5, the coordinates of the mouse position of the pixel area to be viewed in the upper left corner of the pathological image are 2000*2000 (first coordinate). Since the screen resolution is 1028*720, the pixel coordinates of the pixel area to be viewed in the lower right corner of the pathological image are 3028*2720 (second coordinate), thereby determining the pixel area that the doctor needs to view.
[0141] According to the reduction ratio of 13.5, the hierarchical table is queried. The reduction ratio of 13.5 is closest to the third layer of the hierarchical table. That is to say, the smallest layer in the hierarchical table that is greater than 13.5 is the third layer, and the reduction ratio corresponding to the third layer is 16. The pixel area that the doctor needs to view is converted to the pixels corresponding to the third layer, that is, the pixels 2000*2000 to 3028*2720 are converted to 16, and the corresponding pixels are 2370*2370 to 3589*3234 (calculation method: pixel / 13.5*16, rounded up or down).
[0142] Then, according to the tile block size 256*256, it is calculated that the tile range of the pixel area to be viewed is row 9-column 9 to row 14-column 12 (calculation method: pixel / 256, rounded up or down). According to the starting row, starting column, end row, and end column, it can be seen that the rows are 6 and the columns are 4, so a total of 24 tiles (6*4) are required. Then, 24 tiles are taken out from the third layer as the tiles to be viewed for the pixel area to be viewed.
[0143] According to the layer number, horizontal coordinate, and vertical coordinate of the tile to be viewed, check whether the tile has been downloaded in the local cache of the consultation diagnosis client segment. If all hits, it will be drawn directly. If a certain coordinate tile has not been downloaded, its layer number (number of layers), horizontal coordinate number, and vertical coordinate number are sent to the relay server to apply for priority download and set a trigger. Once the drawing of the map is completed, the display will be refreshed immediately. If the current tile does not exist on the relay server, the relay server will initiate a priority upload instruction to the consultation application client. The consultation application client will prioritize transmitting tiles in the pixel area to the relay server until the consultation diagnosis client is hit when polling the relay server.
[0144] At the same time, the drawing program determines whether the tiles that are not satisfied by the current layer (3rd layer) can be supplemented by the tiles of the previous layer (4th layer). If the same position of the 4th layer can be supplemented, it will be recorded. If the 4th layer cannot be satisfied, it will be judged by the upper layer (5th layer) until the top layer (7th layer), which will definitely hit (preloaded). The image is drawn in order from top to bottom so that there are no blank spaces in all positions of the image. The missing piece code is recorded in the cache. When the receiving queue receives the tile, it can determine whether it is missing. If it is missing, it will trigger a redraw to make up for the part that is not in the current layer (3rd layer) until it is finally displayed as the content of the current layer (3rd layer).
[0145] According to the embodiment of the present invention, when the consultation diagnosis client views a tile, it can be downloaded directly from the relay server, thereby ensuring that the consultation diagnosis client can view the pathological image faster. If the consultation diagnosis client views a tile that does not exist in the relay server, the three-dimensional coordinates of the tile are obtained and the three-dimensional coordinates of the tile are sent to the relay server, so that the relay server searches whether the tile exists in the relay server according to the three-dimensional coordinates of the tile; if not, the relay server sends a priority upload instruction to the consultation application client, and the consultation application client inserts the tile into the head of the queue and uploads the tile to the relay server with priority, so that the relay server sends the tile to the consultation diagnosis client. That is to say, when the consultation diagnosis client applies to download an unprocessed tile, the tile is put into the front end of the queue for priority processing and the file is returned by queuing the tile, thereby realizing instant and low-latency remote viewing of pathological images.
[0146] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0147] Embodiment 2
[0148] Reference Figure 3 , shows an image transmission device according to the second embodiment of the present invention, the device is applied to a consultation diagnosis client, and includes:
[0149] The loading module 301 is used to load the tiles of the pathological image with the highest compression rate sent by the consultation application client and the basic information of the pathological image.
[0150] The display module 302 is used to display the tiles in the pathological image with the highest compression rate.
[0151] The calculation module 303 is used to determine the pixel area to be checked selected by the consultation diagnosis client in the pathological image, and calculate the tiles to be checked of the pixel area to be checked in a layer-by-layer upward manner.
[0152] The query module 304 is used to directly display the pathological image if the tile to be viewed exists in the consultation diagnosis client.
[0153] Optionally, the query module is further used to obtain the three-dimensional coordinates of the tile to be viewed when the shape of the tile to be viewed is a square if the consultation diagnosis client does not exist for the tile to be viewed, and the three-dimensional coordinates include: the number of layers of the tile to be viewed, the horizontal coordinate of the tile to be viewed, and the vertical coordinate of the tile to be viewed;
[0154] A judgment module, used for sending the three-dimensional coordinates of the tile to be viewed to the relay server, so that the relay server searches whether the tile to be viewed exists in the relay server according to the three-dimensional coordinates of the tile to be viewed;
[0155] The judgment result of the judgment module is that if it does not exist, the relay server sends a priority upload instruction to the consultation application client, and the consultation application client inserts the tile to be viewed into the head of the queue, and uploads the tile to the relay server in priority, so that the relay server sends the tile to be viewed to the consultation diagnosis client.
[0156] Optionally, the device further comprises:
[0157] The layer table module is used to generate a layer table according to the basic information of the pathological image, wherein the layer table stores the corresponding relationship between the basic information of the pathological sub-image, the number of layers and the reduction rate.
[0158] The calculation module comprises:
[0159] A first coordinate submodule, for obtaining a first coordinate of a current position of a mouse in the pixel area to be viewed when the consultation diagnosis client is operated to adjust the current zoom-out ratio to a first zoom-out ratio;
[0160] A second coordinate submodule, used for obtaining a second coordinate of the pixel area to be checked according to the first coordinate and the screen resolution;
[0161] A search submodule, configured to search the layer table according to the first reduction ratio to obtain the layer corresponding to the first reduction ratio and the second reduction ratio corresponding to the layer;
[0162] A first calculation submodule, configured to calculate a first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio, and the second reduction ratio;
[0163] The second calculation submodule is used to calculate the second pixel corresponding to the first coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio.
[0164] The third calculation submodule is configured to calculate a tile to be checked corresponding to the pixel area to be checked in a layer according to the first pixel and the second pixel.
[0165] Optionally, the first computing submodule is specifically used for:
[0166] Divide the horizontal coordinate of the first coordinate by the first reduction ratio and multiply by the second reduction ratio to obtain the horizontal coordinate of the first pixel;
[0167] The vertical coordinate of the first coordinate is divided by the first reduction ratio and multiplied by the second reduction ratio to obtain the vertical coordinate of the first pixel.
[0168] Optionally, the second calculation submodule is specifically used to divide the horizontal coordinate of the second coordinate by the first reduction ratio multiplied by the second reduction ratio to obtain the horizontal coordinate of the second pixel; divide the vertical coordinate of the second coordinate by the first reduction ratio multiplied by the second reduction ratio to obtain the vertical coordinate of the second pixel.
[0169] Optionally, the third calculation submodule is specifically used to divide the horizontal coordinate and vertical coordinate of the first pixel by the set pixel to obtain the starting row and starting column of the tile; divide the horizontal coordinate and vertical coordinate of the second pixel by the set pixel to obtain the ending row and ending column of the tile; and obtain the tile to be viewed corresponding to the pixel area to be viewed in the layer according to the starting row, starting column, ending row and ending column.
[0170] The device also includes:
[0171] A stratification module, used for stratifying the basic information of the pathological image according to different reduction ratios to obtain a plurality of pathological sub-images;
[0172] A cutting module, used for cutting the pathological sub-image to generate a plurality of tiles;
[0173] A sorting module, used for uploading the tiles to a pre-established queue in sequence according to the compression rate of the pathological sub-images from high to low;
[0174] The uploading module is used to upload the basic information of the tiles and the pathological images in the queue to the transfer server, so that the consultation diagnosis client can download the basic information of the tiles and the pathological images from the transfer server.
[0175] Optionally, the cutting module is specifically used to cut the pathological sub-image according to set pixels to generate a plurality of tiles.
[0176] According to the embodiment of the present invention, when the consultation diagnosis client views a tile, it can be downloaded directly from the relay server, thereby ensuring that the consultation diagnosis client can view the pathological image faster. If the consultation diagnosis client views a tile that does not exist in the relay server, the three-dimensional coordinates of the tile are obtained and the three-dimensional coordinates of the tile are sent to the relay server, so that the relay server searches whether the tile exists in the relay server according to the three-dimensional coordinates of the tile; if not, the relay server sends a priority upload instruction to the consultation application client, and the consultation application client inserts the tile into the head of the queue and uploads the tile to the relay server with priority, so that the relay server sends the tile to the consultation diagnosis client. That is to say, when the consultation diagnosis client applies to download an unprocessed tile, the tile is put into the front end of the queue for priority processing and the file is returned by queuing the tile, thereby realizing instant and low-latency remote viewing of pathological images.
[0177] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The embodiment of the present invention also provides an electronic device, including:
[0178] one or more processors; and
[0179] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to perform the image transmission method as described.
[0180] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the image transmission method.
[0181] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0182] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0183] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0184] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0186] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0187] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0188] The above is a detailed introduction to an image transmission method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. An image transmission method, characterized in that: The method is applied to a consultation diagnosis client, comprising: The consultation diagnosis client loads the tiles of the pathological image with the highest compression rate and the basic information of the pathological image sent by the consultation application client; The consultation diagnosis client displays the tile in the pathological image with the highest compression rate; Determine the pixel area to be checked selected by the consultation diagnosis client in the pathological image, and calculate the tiles to be checked of the pixel area to be checked in a layer-by-layer upward manner; If the tile to be viewed exists in the consultation diagnosis client, the pathological image is directly displayed; The method further comprises: Generating a layer table according to the basic information of the pathological image, wherein the layer table stores the corresponding relationship between the basic information of the pathological sub-image, the number of layers and the reduction rate; The step of determining the pixel area to be viewed selected by the consultation diagnosis client in the pathological image and calculating the tiles to be viewed of the pixel area to be viewed in a layer-by-layer upward manner comprises: When the consultation diagnosis client is operated to adjust the current zoom-out ratio to the first zoom-out ratio, obtaining the first coordinate of the current position of the mouse in the pixel area to be viewed; Obtaining a second coordinate of the pixel area to be viewed according to the first coordinate and the screen resolution; Searching the layer table according to the first reduction ratio to obtain the layer corresponding to the first reduction ratio and the second reduction ratio corresponding to the layer; Calculate a first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio, and the second reduction ratio; Calculate the second pixel corresponding to the first coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio A tile to be viewed corresponding to the pixel area to be viewed in a layer is calculated according to the first pixel and the second pixel.
2. The method according to claim 1, characterized in that The method further comprises: If the consultation diagnosis client does not exist for the tile to be viewed, obtaining the three-dimensional coordinates of the tile to be viewed when the shape is a square, the three-dimensional coordinates including: the number of layers of the tile to be viewed, the horizontal coordinate of the tile to be viewed, and the vertical coordinate of the tile to be viewed; Sending the three-dimensional coordinates of the tile to be viewed to the relay server, so that the relay server searches whether the tile to be viewed exists in the relay server according to the three-dimensional coordinates of the tile to be viewed; If it does not exist, the relay server sends a priority upload instruction to the consultation application client, and the consultation application client inserts the tile to be viewed into the head of the queue and uploads the tile to be viewed to the relay server first, so that the relay server sends the tile to be viewed to the consultation diagnosis client.
3. The method according to claim 1, characterized in that The step of calculating the first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio and the second reduction ratio comprises: Divide the horizontal coordinate of the first coordinate by the first reduction ratio and multiply by the second reduction ratio to obtain the horizontal coordinate of the first pixel; The vertical coordinate of the first coordinate is divided by the first reduction ratio and multiplied by the second reduction ratio to obtain the vertical coordinate of the first pixel.
4. The method according to claim 1, characterized in that: The step of calculating the second pixel corresponding to the second coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio comprises: Divide the horizontal coordinate of the second coordinate by the first reduction ratio and multiply by the second reduction ratio to obtain the horizontal coordinate of the second pixel; The vertical coordinate of the second pixel is obtained by dividing the vertical coordinate of the second pixel by the first reduction ratio and multiplying by the second reduction ratio.
5. The method according to claim 3 or 4, characterized in that: The step of calculating the tile to be checked of the pixel area to be checked in the layer according to the first pixel and the second pixel comprises: Dividing the horizontal coordinate and the vertical coordinate of the first pixel by the set pixel to obtain the starting row and the starting column of the tile; Dividing the horizontal coordinate and the vertical coordinate of the second pixel by the set pixel respectively to obtain the end row and the end column of the tile; The tile to be viewed corresponding to the pixel area to be viewed in the layer is obtained according to the starting row, the starting column, the ending row, and the ending column.
6. The method according to claim 1, characterized in that Before the consultation diagnosis client loads the tiles in the pathological image with the highest compression rate and the basic information of the pathological image sent by the consultation application client, the method further includes: Layering the basic information of the pathological image according to different reduction ratios to obtain a plurality of pathological sub-images; Cutting the pathological sub-image to generate a plurality of tiles; Uploading the tiles to a pre-established queue in sequence according to the compression rates of the pathological sub-images from high to low; The basic information of the tiles and the pathological images in the queue is uploaded to a transfer server, so that the consultation diagnosis client downloads the basic information of the tiles and the pathological images from the transfer server.
7. The method according to claim 6, characterized in that The step of cutting the pathological sub-image to generate a plurality of tiles comprises: The pathological sub-image is cut according to the set pixels to generate several tiles.
8. An image transmission device, characterized in that: The device is applied to a consultation and diagnosis client, and comprises: A loading module, used to load tiles of the pathological image with the highest compression rate sent by the consultation application client and basic information of the pathological image; A display module, used for displaying the tile in the pathological image with the highest compression rate; A calculation module, used to determine the pixel area to be checked selected by the consultation diagnosis client in the pathological image, and calculate the tiles to be checked of the pixel area to be checked in a layer-by-layer upward manner; A query module, used for directly displaying the pathological image if the tile to be viewed exists in the consultation diagnosis client; The device also includes: A layer table module, used to generate a layer table according to the basic information of the pathological image, wherein the layer table stores the corresponding relationship between the basic information of the pathological sub-image, the number of layers and the reduction rate; The calculation module comprises: A first coordinate submodule, for obtaining a first coordinate of a current position of a mouse in the pixel area to be viewed when the consultation diagnosis client is operated to adjust the current zoom-out ratio to a first zoom-out ratio; A second coordinate submodule, used for obtaining a second coordinate of the pixel area to be checked according to the first coordinate and the screen resolution; A search submodule, configured to search the layer table according to the first reduction ratio to obtain the layer corresponding to the first reduction ratio and the second reduction ratio corresponding to the layer; A first calculation submodule, configured to calculate a first pixel corresponding to the first coordinate in the layer according to the first coordinate, the first reduction ratio, and the second reduction ratio; The second calculation submodule is used to calculate the second pixel corresponding to the first coordinate in the layer according to the second coordinate, the first reduction ratio and the second reduction ratio. The third calculation submodule is configured to calculate a tile to be checked corresponding to the pixel area to be checked in a layer according to the first pixel and the second pixel.
9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to perform the image transmission method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the image transmission method as claimed in any one of claims 1 to 7.
