A WSI image display method and system based on a B / S structure

Abstract

This invention provides a WSI image display method and system based on a B / S architecture. The display method includes the following steps: creating a pyramid-structured WSI image of a pathological slide and storing it on a server; obtaining a query request initiated by a browser, the query request including the current field of view location and manufacturer identifier; and, based on the query request, encapsulating the manufacturer's SDK, querying the image, and sending it to the browser. The display method provided by this invention can open images with large memory requirements and is compatible with displaying WSI images of pathological slides from different manufacturers.

Description

Technical Field

[0001] This invention belongs to the field of computer vision image processing technology, and particularly relates to a WSI image display method and system based on a B / S architecture. Background Technology

[0002] Wholeslide image (WSI) technology is crucial in bioimaging research, such as in cancer analysis and disease prediction. It allows traditional microscopic slides to be acquired as digital images, which are easy to store and access from any computer, and can be transmitted remotely between researchers and doctors. WSI typically involves two steps: (1) scanning pathological images sequentially by sub-region and then stitching them together to produce a complete full-field pathological slide image; (2) using specialized software to identify and analyze these digital images. It utilizes a digital scanner to scan traditional pathological slides, acquiring high-resolution digital images, and then seamlessly stitches and integrates the fragmented images using a computer to create a visualized digital image. Compared to traditional glass slides, this technology effectively solves the problems of traditional glass slides being easily damaged, faded, lost, and difficult to retrieve.

[0003] Currently, WSI format images are too large, ranging from 600Mb to 10Gb per image, and cannot be opened by most software. Furthermore, there are many WSI manufacturers, and no single display software is compatible with most WSI formats on the market. Summary of the Invention

[0004] The main objective of this invention is to propose a WSI image display method and system based on a B / S architecture, aiming to solve the technical problem that current WSI format pathological slide digital images cannot be displayed in software compatibility.

[0005] To achieve the above objectives, this invention proposes a WSI image display method based on a B / S architecture, which includes the following steps:

[0006] Create WSI images of pathological slides in a pyramid structure and store them on the server;

[0007] Obtain the query request initiated by the browser, which includes the current view location and the manufacturer's identifier;

[0008] Based on the query request, the packaging manufacturer's SDK queries the image and sends it to the browser.

[0009] Optionally, the step of creating pyramid-structured WSI images of pathological sections and storing them on a server includes:

[0010] Construct a three-dimensional coordinate system, which includes several layers of grid regions in a pyramid structure;

[0011] Based on the hierarchy, the scaled WSI images of pathological slides are distributed in a grid area;

[0012] The image and its corresponding 3D coordinates within each grid area are stored on the server.

[0013] Optionally, before obtaining the query request initiated by the browser, which includes the current view location and manufacturer identification, the method further includes: determining the layer and view of the WSI image required by the user based on the scrolling of the mouse wheel.

[0014] Optionally, in the step of obtaining the query request initiated by the browser, wherein the query request includes the current view location and the manufacturer's identifier, the current view location is the coordinates (x, y, z) of the image pyramid where the current view is located.

[0015] Optionally, in the step of obtaining the query request initiated by the browser, wherein the query request includes the current field of view location and the manufacturer's identifier, the coordinates (x, y, z) represent the x-th row and y-th column of the current field of view in the WSI image grid region, and z represents the level of the image pyramid in which the WSI image of the current field of view is located.

[0016] Optionally, the step of encapsulating the manufacturer's SDK, querying the image, and sending it to the browser according to the query request includes:

[0017] Based on the manufacturer's identification, the manufacturer's SDK is packaged;

[0018] Based on the current field of view location, read the image from the server;

[0019] Call the packaged SDK and send the image to the browser.

[0020] This invention also proposes a WSI image display system based on a B / S architecture, which includes:

[0021] Browser;

[0022] Resource servers are used to store WSI images of pathological slides from different manufacturers; and,

[0023] An image query device is used to obtain a query request initiated by the browser, query WSI images in the resource server, and send the WSI images to the browser.

[0024] Optionally, the image query device is also used to encapsulate SDKs that support WSI image formats from various manufacturers.

[0025] In the technical solution of this invention, pyramid-structured WSI images of pathological slides are stored on a server, and a query request is sent through a browser. Based on the content of the query request, the server calls the manufacturer's SDK and sends the requested WSI image to the browser. The display method provided by this invention can open images with large memory requirements and is compatible with displaying WSI images of pathological slides from different manufacturers. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating an embodiment of the WSI image display method based on a B / S architecture proposed in this invention.

[0028] Figure 2 A full-field image of a pathological slide in WSI format, provided by an embodiment of the WSI image display method based on a B / S architecture according to the present invention.

[0029] Figure 3 A partial WSI image of a pathological slide, as provided by an embodiment of the WSI image display method based on a B / S architecture according to the present invention;

[0030] Figure 4 This is a schematic diagram of an embodiment of the WSI image display system based on the B / S architecture provided by the present invention.

[0031] Labeling Explanation: WSI Image Display System Based on B / S Architecture - 100, Browser - 101, Resource Server - 102, Image Query Device - 103. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] To better describe and illustrate the embodiments of this application, reference may be made to one or more accompanying drawings, but the additional details or examples used to describe the drawings should not be considered as limiting the scope of any of the inventive creations of this application, the embodiments or preferred methods described herein.

[0034] In the description of this invention, it should be noted that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and do not indicate that the device referred to must have a specific orientation or operate in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0036] Currently, WSI format images are too large, ranging from 600Mb to 10Gb per image, and cannot be opened by most software. Furthermore, there are many WSI manufacturers, and no single display software is compatible with most WSI formats on the market.

[0037] In view of this, this invention proposes a WSI image display method based on a B / S architecture. B / S architecture stands for Browser / Server, where a small amount of business logic is implemented on the front end, but the main business logic is implemented on the server side. In the B / S architecture, the display logic is handled by the web browser, while the business logic is placed on the web application. This avoids a large, heavy client and reduces the pressure on the client. Because the client contains very little logic, it is also called a thin client.

[0038] Figure 1 For a flowchart illustrating an embodiment of the proposed WSI image display method based on a B / S architecture, please refer to [link / reference]. Figure 1-3 The demonstration method includes the following steps:

[0039] Step S10: Create WSI images of pathological sections with a pyramid structure and store them on the server.

[0040] Image pyramids are a type of multi-scale representation of images, an efficient yet conceptually simple structure for interpreting images at multiple resolutions. An image pyramid is a series of image resolutions derived from the same original image, progressively decreasing in a pyramid shape (from bottom to top). It is obtained through stepwise downsampling until a certain termination condition is met.

[0041] In this embodiment, to enable WSI images to be scaled and displayed at different resolutions, a pyramid model is used to store WSI images. Different layers of the pyramid correspond to different resolutions. The bottom layer of the pyramid contains the highest resolution image data captured by the instrument, while the middle layer contains thumbnails of the bottom image, which are lower-resolution versions, so that pathologists can retrieve data at lower resolution.

[0042] Step S20: Obtain the query request initiated by the browser, the query request including the current view location and the manufacturer's identifier.

[0043] In this embodiment, users such as doctors determine the desired WSI image level by scrolling the mouse wheel in their browser. The mouse click or landing point determines the location of the user's desired field of view, thus generating a query request. In practice, the field of view location is converted into coordinates within the image pyramid by the backend server. .

[0044] It should be noted that this browser is written in HTML, CSS, and JavaScript, and primarily provides WSI image display and zooming functions.

[0045] Step S30: Based on the query request, package the manufacturer's SDK, query the image, and send it to the browser.

[0046] In this embodiment, the query request generates a URL: The access address is specified in the URL. Here, x and y represent the current viewpoint's position in the WSI image (row x, column y), z represents the pyramid level (the level of the WSI image where the current viewpoint is located), and filename is used to distinguish WSI images from different manufacturers. The server determines the image storage location based on the information in the access address and returns it to the browser. Simultaneously, the backend server also encapsulates the manufacturer's SDK (Assistant Toolkit) to support their WSI format, enabling the image to be opened and displayed in the browser. Furthermore, the frontend and backend agree to request images with a resolution of 1024*1024 for easier display in the browser.

[0047] It's worth noting that the backend server is programmed in Python (within the Flask framework) to handle business logic. The basic pattern of the Flask framework is to assign a view function to a URL within the program. Whenever a user accesses this URL, the system executes the assigned view function, retrieves the function's return value, and displays it in the browser.

[0048] In the technical solution of this invention, pyramid-structured WSI images of pathological slides are stored on a server, and a query request is sent through a browser. Based on the content of the query request, the server calls the manufacturer's SDK and sends the requested WSI image to the browser. The display method provided by this invention can open images with large memory requirements and is compatible with displaying WSI images of pathological slides from different manufacturers.

[0049] Specifically, step S10 includes the following steps:

[0050] Step S11: Construct a three-dimensional coordinate system, which includes several layers of grid regions in a pyramid structure.

[0051] In this embodiment, a three-dimensional coordinate system (X, Y, Z) is constructed, where X is the horizontal coordinate axis, Y is the vertical coordinate axis, and Z represents the level. Each grid cell in the grid region of each level has a side length of 1024 pixels. Smaller grid side lengths result in more accurate WPI images, but also incur additional storage overhead and a larger search space.

[0052] Step S12: Distribute the scaled pathological slide WSI images in the grid area according to the hierarchy.

[0053] In this embodiment, the image at each level of the WSI image pyramid is divided into several fixed-size image blocks ("tiles"). When browsing in a browser, the browser dynamically requests several "tiles" at the current view level and location from the server to piece together the image of the current view.

[0054] Step S13: Store the image and its corresponding 3D coordinates within each grid area in the server.

[0055] In this embodiment, each image patch in the three-dimensional coordinate system is associated with its corresponding coordinates. The images are stored on the server. When a user views the image in a browser, the server calculates the grid coordinates of the image block based on the mouse click or landing point on the WPI image, and then returns the images within these grids and their surrounding images to the browser, thus enabling the dynamic display of WPI images.

[0056] In addition, step S30 specifically includes the following steps:

[0057] Step S31: Package the manufacturer's SDK according to the manufacturer's identifier.

[0058] In this embodiment, the access address generated by the query request includes the manufacturer's identifier. After obtaining the identifier, the server encapsulates the manufacturer's SDK for supporting WPI images, i.e., the auxiliary toolkit.

[0059] Step S32: Read the image from the server based on the current field of view position.

[0060] In this embodiment, the access address generated by the query request includes the image location, and the server determines whether an image is stored based on the image location; if it is stored, the image is read.

[0061] Step S33: Call the packaged SDK and send the image to the browser.

[0062] In this embodiment, the server calls the packaged SDK and sends the image to the browser. The SDK supports opening WPI images in the browser.

[0063] This invention also proposes a WSI image display system based on a B / S architecture. Figure 4 For a schematic diagram of an embodiment of the proposed WSI image display system based on a B / S architecture, please refer to [link / reference]. Figure 4 The display system 100 includes a browser 101, a resource server 102, and an image query device 103.

[0064] The resource server 102 is used to store WSI images of pathological slides from different manufacturers. It should be noted that these WSI images are stored in an image pyramid format and include the manufacturer's identifier. The image query device 103 is used to receive query requests initiated by the browser 101 and send them to the resource server 102. The resource server 102 queries the WSI images according to the query requests and then sends the WSI images to the browser 101.

[0065] Furthermore, the image query device 103 is also used to encapsulate SDKs that support WSI image formats from various manufacturers, so as to facilitate their use and enable WSI images to be opened and displayed smoothly in the browser 101.

[0066] The WSI image display system 100 based on the B / S architecture of the present invention can display different WSI formats, has strong compatibility, displays images without lag, and provides a good user experience.

[0067] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A WSI image display method based on a B / S architecture, characterized in that, Includes the following steps: S10. Create WSI images of pathological sections with a pyramid structure and store them on the server; S11. Construct a three-dimensional coordinate system, which includes several layers of grid regions in a pyramid structure; S12. Distribute the scaled pathological slide WSI images into the grid regions according to the layers; S13. Store the image and its corresponding three-dimensional coordinates in each grid region into the server; S20. Obtain a query request initiated by the browser, the query request including the current view location and the manufacturer's identifier; the current view location is the three-dimensional coordinates (x, y, z) of the image pyramid where the current view is located. S30. Based on the query request, package the manufacturer's SDK and send the queried image to the browser; the query request generates a URL: The access address is specified in the URL; where x and y represent the current view's position in the WSI image (row x, column y), z represents the pyramid level (the level of the WSI image where the current view is located), and filename is used to distinguish WSI images from different manufacturers. The server determines the image storage location based on the information in the access address and returns it to the browser. Simultaneously, the backend server also encapsulates the SDKs (toolkits) of these manufacturers to support their WSI formats, enabling the images to be opened and displayed in the browser. Furthermore, the frontend and backend agree to request images with a resolution of 1024*1024 for easier display in the browser.

2. The WSI image display method based on B / S architecture as described in claim 1, characterized in that, Before obtaining the query request initiated by the browser, which includes the current view location and manufacturer identification, the method further includes: determining the layer and view of the WSI image required by the user based on the scrolling of the mouse wheel.

3. The WSI image display method based on B / S architecture as described in claim 1, characterized in that, In the step of obtaining the query request initiated by the browser, which includes the current field of view location and the manufacturer's identifier, the coordinates (x, y, z) represent the x-th row and y-th column of the current field of view in the WSI image grid region, and z represents the level of the image pyramid where the WSI image of the current field of view is located.

4. The WSI image display method based on B / S architecture as described in claim 1, characterized in that, The steps of encapsulating the manufacturer's SDK according to the query request, querying the image and sending it to the browser include: encapsulating the manufacturer's SDK according to the manufacturer's identifier; Based on the current field of view location, read the image from the server; Call the packaged SDK and send the image to the browser.

5. A WSI image display system for the method described in claims 1-4, characterized in that, include: Browser; Resource server, used to store WSI images of pathological slides from different manufacturers; And an image query device, used to obtain the query request initiated by the browser, query the WSI image in the resource server, and send the WSI image to the browser.

6. The WSI image display system as described in claim 5, characterized in that, The image query device is also used to encapsulate SDKs that support WSI image formats from various manufacturers.

Images