A method and apparatus for displaying a pathology slide

Abstract

The method and device for displaying pathological sections provided by the embodiment of the application can realize the display of pathological sections by obtaining the level of a target layer of a pathological section, reading picture information and target matrix information corresponding to the level of the target layer, and generating a target display section picture according to the target matrix information and a preset rule. The method and device are easy to operate, can be extended to meet medical analysis, are easy to maintain, can be repeatedly used at the front end after being generated once at the back end, and have a fast loading speed.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to a method and apparatus for displaying pathological slides. Background Technology

[0002] With the increasing integration of internet technology and medicine, there is a growing demand for displaying and analyzing medical images on web browsers. However, medical images are typically large, ranging from under 10MB to over 1GB, and are highly specialized, making direct loading into web browsers for display and analysis impractical. Currently, the main methods include using OpenSeaDragon front-end loading in conjunction with Deep Zoom, and using map front-end loading tile technology. OpenSeaDragon loads images and offers basic interactive functionality. However, tile technology itself is not designed for loading medical images due to its high technical complexity, high maintenance costs, and inconvenient application. Summary of the Invention

[0003] This application provides a method and apparatus for displaying pathological slides to solve the problem that existing professional software's customizable functions are not easily expandable and cannot meet the needs of front-end pathological image display and analysis.

[0004] Firstly, to achieve the above objectives, this application provides a method for displaying pathological slides, comprising:

[0005] Obtain the layer level of the target layer for the pathological slide;

[0006] Read the image information and target matrix information corresponding to the layer of the target layer;

[0007] Based on the target matrix information, a target display slice layer is generated according to preset rules.

[0008] Further, the step of generating target display slices according to preset rules based on the target matrix information includes:

[0009] A target layer is generated according to preset rules based on the target matrix information;

[0010] Generate a target display layer based on the target layer.

[0011] Furthermore, before or after the layer at which the target layer of the pathological slide is obtained, the method further includes:

[0012] Read the element images corresponding to the pathological slides, and the storage path of the element images.

[0013] Furthermore, after generating the target display layer according to the target matrix information and preset rules, the process further includes:

[0014] Tags are generated based on the storage path of the element image.

[0015] Furthermore, after generating the target display layer according to the target matrix information and preset rules, the method further includes: generating thumbnails based on the storage path of the element images.

[0016] Secondly, to achieve the above objectives, this application provides an apparatus for displaying pathological slides, the apparatus comprising:

[0017] The acquisition unit is used to acquire the layer of the target layer of a pathological slide;

[0018] The reading unit is used to read the image information and target matrix information corresponding to the layer of the target layer;

[0019] The slice display unit is used to generate a target display slice layer according to the target matrix information and a preset rule.

[0020] Furthermore, the slice display unit is also used for:

[0021] A target layer is generated according to preset rules based on the target matrix information;

[0022] Generate a target display layer based on the target layer.

[0023] Furthermore, the reading unit is also used for:

[0024] Read the element images corresponding to the pathological slides, and the storage path of the element images.

[0025] Furthermore, the device also includes:

[0026] The label display unit is used to generate labels based on the storage path of the element image.

[0027] Furthermore, the device also includes:

[0028] The thumbnail display unit is used to generate thumbnails based on the storage path of the element image.

[0029] Thirdly, to achieve the above objectives, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for displaying pathological slides according to any of the first aspects.

[0030] Fourthly, to achieve the above objectives, this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for displaying pathological slides according to any of the first aspects.

[0031] As can be seen from the above technical solutions, the method and apparatus for displaying pathological slides provided in this application embodiment obtain the layer level of the target layer of the pathological slide; read the image information and target matrix information corresponding to the layer level of the target layer; and generate a target display slide image according to the target matrix information and preset rules to display the pathological slide. The method and apparatus described in this application are easy to operate, can be expanded to meet medical analysis needs, are easy to maintain, generate once on the back end, and can be reused on the front end, with a fast loading speed.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart illustrating a method for displaying pathological slides provided in an embodiment of this application;

[0035] Figure 2 This application provides a schematic diagram of the structure of a small image matrix in an embodiment;

[0036] Figure 3 A schematic diagram of the processed data provided in the embodiments of this application;

[0037] Figure 4 This is a schematic diagram of the target display slice layer structure provided in an embodiment of this application;

[0038] Figure 5 This is a schematic diagram of a device for displaying pathological slides provided in an embodiment of this application. Detailed Implementation

[0039] The features and advantages of this application will become clearer and more explicit through the following detailed description.

[0040] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0041] In the description of this application, it should be noted that the terms "upper," "lower," "inner," "outer," "front," "rear," "left," and "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship in the working state of this application, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0042] Pathological slides are images scanned by an electron microscope for pathological analysis. As professional medical images, pathological slides are highly specialized, come in various formats, and are characterized by large size and high resolution. For example, the size of a single pathological slide is usually around 500MB (MByte, megabit) to 1.5GB (gigabyte); the resolution of a single pathological slide is usually above 40,000*40,000 pixels.

[0043] The applicant found that existing Deep Zoom technology uses OpenSeaDragon to load and display data on the front-end server. This not only requires front-end personnel to be proficient in OpenSeaDragon, but also makes secondary development of the front-end server difficult. The generated slice files and information at various resolutions cannot meet the needs of medical analysis and are not very intuitive. It requires understanding and analyzing the basic composition of SVS files, which requires operators to have strong professional knowledge. Moreover, the loading speed is not very fast.

[0044] Figure 1 A flowchart illustrating a method for displaying pathological slides provided in this application embodiment is shown below. Figure 1 As shown, the method includes steps S101 to S103.

[0045] S101: Obtain the layer level of the target layer of the pathological slide.

[0046] Existing pathological slides are difficult to display and analyze directly on the front end of a web browser. Therefore, a novel method for converting pathological slides is adopted, which converts original pathological image files, such as SVS format, to formats such as JPG for storage to meet the requirements of front-end display and analysis. Specifically, the original pathological image file is obtained, which includes original pathological slides, which may be in SVS format and include at least one layer; the image information of the original pathological slide is obtained and saved to a database; the image information includes the number of layers of the original pathological slide, the resolution of each layer, and the pixel size of each layer; according to the image information, each layer of the original pathological slide is cropped into several smaller images according to a preset matrix rule, and the matrix information of the path set of the smaller images is saved to the database; processing data is formed based on the image information and the matrix information of the path set of the smaller images, and the SVS format medical pathological images are cropped at various resolutions to generate an image matrix, and the basic image information is saved to the database to meet the requirements of front-end display and analysis. Figure 2 The diagram shows the structure of a small image matrix. The cropped images are small enough for regular viewing, and you can intuitively see the set of clear images corresponding to pathological slides at various resolutions without the need for professional software.

[0047] The backend server only needs to generate the image once, and the frontend directly retrieves and displays it. There are no restrictions on the loading method the frontend can use; as long as the image is displayed according to certain rules, images of various resolutions can be restored to meet the needs of different frontend loading methods. Furthermore, the converted images contain relatively complete information, allowing the frontend to perform effective analysis. Using this invention to convert pathological slides, a clear set of images corresponding to pathological slides of various resolutions can be seen intuitively.

[0048] In practice, the user inputs the target layer level of the pathological slide they want to display, and then the corresponding information is obtained to further display the target layer.

[0049] S102: Read the image information and target matrix information corresponding to the layer of the target layer.

[0050] The above method for converting pathological slides converts original pathological image files, such as SVS format, into formats like JPG for storage. The image information and the segmented image matrix information together form the processed data, which is then stored. After obtaining the target layer's hierarchy, the corresponding processed data is retrieved, such as... Figure 3 The `photo_set` parameter is used for image display.

[0051] The image information of the original pathological slides includes mpp, level_downsamples, and level_dimensions. Mpp refers to the ratio of each layer's resolution to the display device's resolution; level_downsamples is the ratio of each layer's resolution to the maximum resolution within the layer; and level_dimensions is the pixel size corresponding to each layer. The pixel size is pixel width * pixel height. Pixel width and height are affected by the camera used for taking the image and are determined after the tissue slide is photographed. The pixel width and height are obtained by converting the image using the machine's resolution coefficient during photography. This coefficient is in pixels / µm, which is the ratio of pixels to micrometers; this ratio is a value inherent to the machine during photography.

[0052] The matrix information includes the size and positional relationship of each small image in the matrix.

[0053] If the original pathological image file also includes element images, such as labels, macros, thumbnails, etc., these elements are used to further supplement the description of the pathological slides.

[0054] S103: Generate a target display slice layer according to the target matrix information and a preset rule.

[0055] Retrieve corresponding processing data from the database, including image information, matrix information, or element image information.

[0056] In practice, the image matrix can be loaded recursively, such as loading images row by row or column by column using a loop. After one row of images is loaded, the actual image size is obtained, and its aspect ratio is compared with the window's aspect ratio to determine whether it fills the width or height. The position of each loaded row is recorded, and the position is set, along with the current horizontal and vertical coordinate baseline values ​​for reference when loading the next row. Next, it's necessary to determine if the image loading is complete. If so, the scaling data between the current display size and the minimum magnification size needs to be calculated based on the size of the loaded images. If the images are not fully loaded, loading continues.

[0057] In another embodiment, generating the target display slice according to the target matrix information and a preset rule further includes:

[0058] A target layer is generated according to preset rules based on the target matrix information;

[0059] Generate a target display layer based on the target layer.

[0060] First, the images within the matrix are loaded using the matrix information to generate the target layer. Then, the overall aspect ratio of the target layer is compared with the aspect ratio of the window, and the entire layer is enlarged or reduced to generate the target display layer.

[0061] In a specific embodiment, after generating the target display layer according to the target matrix information and a preset rule, the method further includes: generating labels according to the storage path of the element images.

[0062] After obtaining the storage path, load and generate the label image. Adjust the width and height of the label image to a ratio not exceeding 160 according to its actual size. Then, set a rotation count indicator. Clicking the interaction event once will rotate the label image 90 degrees and adjust its coordinates. Adjust the position of the label image according to the number of clicks so that it always stays in a certain position on the display page, such as the upper left or upper right.

[0063] In a specific embodiment, after generating the target display layer according to the target matrix information and a preset rule, the method further includes: generating a thumbnail based on the storage path of the element image.

[0064] After obtaining the storage path, a thumbnail is generated. The image width and height are adjusted to a ratio not exceeding 160 pixels based on the actual size of the thumbnail, and a bounding box of the current display area is drawn. When the mouse drags over the thumbnail, the display area of ​​the target display layer in the moving monitor needs to be changed; the display area cannot be dragged completely off the current screen. Based on the vertex coordinates of the thumbnail, the position of the target display layer is adjusted to match its position in the thumbnail. Clicking the thumbnail achieves the same effect, ensuring that the display area of ​​the target layer corresponds dynamically to the current display area in the thumbnail. Figure 4 The image shows a schematic diagram of the target display slice layer structure, where 001 is the label and 002 is the thumbnail.

[0065] In a specific embodiment, after generating the target display layer according to the target matrix information and a preset rule, the method further includes obtaining the aspect ratio of the image.

[0066] Specifically, it's necessary to obtain the parameters between the magnification factor and mpp at each magnification level under the current display magnification, and calculate the actual size at the maximum magnification. For example, if the lowest resolution of a layer is 1000*800, and the next level magnification by 2 times is 2000*1600, and the monitor's display resolution is 100*80, which differs from the lowest layer resolution by a factor of 10, then if you magnify from the lowest magnification to the next level, the display will need to be magnified by a factor of 20, and the actual size at the maximum magnification needs to be calculated.

[0067] If a default example scale size is set as `default_w`, the actual distance `b_w` represented by `default_w` is calculated using the `mpp` parameter. `b_w` is then rounded down to an integer (e.g., 231.92 becomes 200 after rounding). This calculated example scale distance `default_w` is then divided by `mpp`. The example scale size is then drawn based on the rounded `b_w` and `default_w`. Finally, unit checks are performed on the represented data; values ​​exceeding 1000µm need to be converted to mm units.

[0068] In practice, this also includes measuring the target size. Specifically, a flag is added to indicate that the mouse is currently in measurement mode. When measuring, the response method of the left and right mouse buttons is adjusted. Clicking the left button draws a starting point or a combination of points and lines to generate a line segment. Then, the actual distance of the generated line segment is calculated and displayed at the midpoint of the line segment. By previewing the measurement function, the specific location of the measurement point can be viewed before finalizing, achieving the purpose of previewing.

[0069] In practice, it also includes drag-and-drop interaction of the target display layer. Specifically, it listens for mouse drag events, obtains the position information of the mouse movement, calculates the distance the mouse moves in the display area of ​​the monitor, performs coordinate transformation, obtains the corresponding distance on the target display layer, and then moves and adjusts it. In response to the movement of the target display layer, the current display area of ​​the thumbnail is adjusted accordingly.

[0070] In practice, this also includes scaling the target display layer up and down. Specifically, it determines whether the mouse wheel is used to zoom in or out, obtains the current display magnification based on its value, and then retrieves the scaling ratio for the previous or next level. Scaling is performed in both the X and Y directions. After scaling, the position needs to be adjusted to prevent the zoomed-in image from being outside the display area. The image matrix within the current display area is detected and replaced with an image at the corresponding magnification. Areas not in display are replaced with the lowest magnification image or simply not displayed. The reason for replacing or not displaying images is to reduce memory usage; otherwise, excessive memory consumption could lead to program crashes.

[0071] This application stores pathological slides as an image matrix after cutting and converting them. Then, by loading image information, it generates dynamic magnification / reduction data for the current image display, measurement ratio reference data, etc. After loading the image matrix, it initializes the label image, thumbnail, original image, and measurement tools respectively, and executes the stage rendering refresh function to continuously update the current display area of ​​the thumbnail, the preview line segment when using the measurement tool, and the changes in the detection window, thus completing the basic initialization display. This solves the problem of meeting the basic needs of medical analysis, restoring large images, and realizing the function of layered multi-resolution, and can be easily expanded to other functions.

[0072] Corresponding to the embodiment of a method for displaying pathological slides provided in this application, this application also provides an embodiment of an apparatus for displaying pathological slides. (Reference) Figure 5 This is a schematic diagram of a device for displaying pathological slides provided in an embodiment of this application. The device includes:

[0073] Acquisition unit 100 is used to acquire the layer of the target layer of the pathological slide.

[0074] The reading unit 200 is used to read the image information and target matrix information corresponding to the layer of the target layer.

[0075] The slice display unit 300 is used to generate a target display slice layer according to the target matrix information and a preset rule. The slice display unit 300 is also used for:

[0076] A target layer is generated according to preset rules based on the target matrix information;

[0077] Generate a target display layer based on the target layer.

[0078] The reading unit 200 is also used for:

[0079] Read the element images corresponding to the pathological slides, and the storage path of the element images.

[0080] The device also includes a label display unit 400, used to generate labels based on the storage path of the element image.

[0081] The device also includes a thumbnail display unit 500, used to generate thumbnails based on the storage path of the element images.

[0082] The device also includes a measurement unit 600 for performing the functions of acquiring and measuring the aspect ratio of the image.

[0083] The device also includes a drag-and-drop interaction unit 700 for performing the movement function of the target display layer.

[0084] The device also includes a scaling interaction unit 800 for performing zooming in or out functions on the target display layer.

[0085] In practical implementation, PIXIJS can be used to load a collection of 2D medical images in SVS format. PIXIJS itself is an open-source HTML5 2D rendering engine implemented using WebGL. Browsers that do not support it will automatically downgrade to Canvas implementation. It is a fast and lightweight 2D library compatible with all devices. Developers do not need to understand WebGL to use it. By calling PIXIJS to load the matrix image set, and developing in-depth image interaction functions according to actual needs, such as drag-and-drop, zooming, and measurement tools, can be implemented.

[0086] The device described in this application includes functions such as stage rendering, container, sprite, interaction, and loading image textures. By loading image information, it generates dynamic zoom-in / zoom-out data for the current image display, measurement ratio reference data, etc. After loading the image matrix, it performs the initialization of label images, thumbnails, original images, and measurement tools, and executes the stage rendering refresh function to continuously update the current display area of ​​the thumbnails, the preview line segments when using the measurement tools, and the changes in the detection window, thus completing the basic initialization.

[0087] This application provides a method and apparatus for displaying pathological slides. The method involves obtaining the layer level of the target layer of the pathological slide; reading the image information and target matrix information corresponding to the layer level of the target layer; and generating a target display slide image according to a preset rule based on the target matrix information to display the pathological slide. The method and apparatus described in this application are easy to operate, can be expanded to meet medical analysis needs, are easy to maintain, generate once on the backend, and can be reused on the frontend, with a fast loading speed.

[0088] According to the above-described method for displaying pathological slides, embodiments of this application also provide a readable storage medium and a computer device. The readable storage medium stores an executable program, which, when executed by a processor, implements the steps of the above-described method for displaying pathological slides. The computer device includes a memory, a processor, and an executable program stored in the memory and executable on the processor; when the processor executes the program, it implements the steps of the above-described method for displaying pathological slides.

[0089] The present application has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present application. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and implementation methods of the present application without departing from the spirit and scope of the present application, and all such modifications and improvements fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

Claims

1. A method for displaying pathological slides, characterized in that, The method includes: Obtain the layer level of the target layer for the pathological slide; Read the image information and target matrix information corresponding to the layer of the target layer; the target matrix information includes the size and positional relationship of each small image in the matrix, and the small images are generated by cropping the corresponding layers of the original pathological slides according to preset matrix rules; According to the target matrix information, a target display slice layer is generated according to a preset rule. During the generation process, the image matrix is ​​loaded in a recursive or loop manner. When loading, the position and horizontal and vertical coordinate reference values ​​of each row of images are recorded. The display size is adapted by combining the aspect ratio of the image with the aspect ratio of the window. When scaling, the image matrix in the display area is replaced with the corresponding magnification image, and the non-display area is replaced with the lowest magnification image or not displayed to reduce memory usage. It also includes generating a target layer according to preset rules based on the target matrix information, and generating a target display layer based on the target layer.

2. The method for displaying pathological sections according to claim 1, characterized in that, Before or after acquiring the target layer of the pathological slide, the method further includes: reading the element image corresponding to the pathological slide and the storage path of the element image.

3. The method for displaying pathological sections according to claim 2, characterized in that, After generating the target display layer according to the target matrix information and a preset rule, the method further includes: generating labels according to the storage path of the element images.

4. The method for displaying pathological sections according to claim 2, characterized in that, After generating the target display layer according to the target matrix information and preset rules, the method further includes: generating thumbnails based on the storage path of the element images.

5. A device for displaying pathological slides, characterized in that, The device includes: an acquisition unit for acquiring the layer of a target layer of a pathological slide; The reading unit is used to read the image information and target matrix information corresponding to the layer of the target layer; the target matrix information includes the size and positional relationship of each small image in the matrix, and the small images are generated by cropping the corresponding layers of the original pathological slides according to preset matrix rules; The slice display unit is used to generate a target display slice layer according to the target matrix information and a preset rule. This includes loading the image matrix in a recursive or loop manner during the generation process, recording the position and horizontal and vertical coordinate reference values ​​of each row of images after loading, adapting the display size by combining the image aspect ratio with the window aspect ratio, and replacing the image matrix in the display area with the corresponding magnification image when scaling, while replacing the non-display area with the lowest magnification image or not displaying it to reduce memory usage. It also includes generating a target layer according to preset rules based on the target matrix information, and generating a target display layer based on the target layer.

6. The apparatus for displaying pathological slides according to claim 5, characterized in that, The reading unit is also used to: read the element images corresponding to the pathological slides, and the storage path of the element images.

7. The apparatus for displaying pathological sections according to claim 5, characterized in that, The device further includes a label display unit, used to generate labels based on the storage path of the element image.

8. The apparatus for displaying pathological slides according to claim 5, characterized in that, The device further includes a thumbnail display unit, used to generate thumbnails based on the storage path of the element image.

Images