One-screen multi-image display system and method based on digital pathological section

Abstract

The invention provides a one-screen multi-image display system and method based on digital pathological sections. The system comprises an image preprocessing module, a layout and mode setting module and an operation result display module. The method comprises the following steps: acquiring a plurality of digital pathological sections, performing full-width digital pathological section imaging WSI format conversion, and performing preprocessing operation of image enhancement and noise removal on a first digital pathological section image after format conversion to obtain a second digital pathological section image after preprocessing operation; obtaining a second digital pathological section image, and automatically adjusting the layout of the second digital pathological section image to obtain a third digital pathological section image after layout; establishing a double-linkage function mode between the third digital pathological section images; and operating the third digital pathological section image, and displaying the observed interested area under the support of a double-linkage function mode. The invention provides a comprehensive and efficient image processing and display platform for the field of digital pathology.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and in particular to a one-screen multi-image display system and method based on digital pathological sections. Background Art

[0002] Digital pathology refers to the use of technology to speed up and optimize the workflow of pathology laboratories; in actual operation, it is achieved by using advanced digital scanners to capture digital pathology slides, which are then stored, analyzed and shared with others. The entire digital pathology system or environment consists of four key parts: capturing digital pathology slide information (images) using digital scanners, image storage and archiving, editing and other operations on captured images, reading and sharing images with other doctors or institutions. Digital pathology is a field in the medical field that uses digital image processing and analysis technology to diagnose and study pathological digital pathology slides. Traditional digital pathology systems usually use a single screen to display pathological digital pathology slides, and doctors need to manually switch between different digital pathology slides for observation and analysis. However, with the development of artificial intelligence technology, more and more AI algorithms are being used in digital pathology to provide doctors with tools for auxiliary diagnosis and analysis. Representative ones are multi-window display systems and tiled display software; among them, multi-window display systems: some digital pathology systems allow doctors to open multiple windows on a single screen, each window displays different pathological digital pathology slices; such systems can display multiple digital pathology slices at the same time, but lack the function of dual linkage, and doctors need to manually operate between different windows; tiled display software: some software provides tiled display function, which can arrange multiple pathological digital pathology slices in a grid or matrix on a single screen. In fact, it is only a single-point mode. Although doctors can view multiple digital pathology slices on the same screen at the same time, they lack the interactive ability and synchronous processing ability of dual linkage, are independent of each other, and have high implementation complexity.

[0003] Prior art 1, application number: 202110781790.1 discloses a method and device for synchronous browsing of multiple digital pathology slices, comprising the following steps: converting a digital pathology slice into a digital pathology slice; selecting a digital pathology slice as a main digital pathology slice Main, and obtaining a navigation thumbnail MT of the main digital pathology slice; selecting a digital pathology slice to be aligned that needs to be aligned with the main digital pathology slice, and obtaining a digital pathology slice of the digital pathology slice to be aligned Figure X I and Navigation Abbreviations Figure X T; determine the similarity between the digital pathology slice to be aligned and the main digital pathology slice Main and obtain the affine transformation matrix of the two; obtain the thumbnail of the digital pathology slice to be aligned and the main digital pathology slice Main according to the affine transformation matrix, and convert the navigation thumbnail Figure XT is rotated, offset and scaled to convert the digital pathology slice to be aligned Figure X Although the digital pathology slices can be browsed synchronously on multiple screens in the browsing platform, it is convenient for users to use the digital pathology slice browsing and comparison function, and improves the work efficiency of doctors; however, it is impossible to switch multiple digital pathology slice images on one display screen, resulting in cumbersome operations and increasing the cost of screen usage by an order of magnitude.

[0004] Prior art 2, application number: 201610264724.6 discloses a one-screen multi-display control system and a one-screen multi-display control method, including an LED display screen including a display module, a generation module and a sending module, and a plurality of shutter glasses including a first receiving module, a processing module and lenses. The display module displays a plurality of programs to be played alternately and cyclically in a preset cycle. The generation module generates a shutter control signal according to the frame start indication of each program to be played. The sending module sends the shutter control signal to the corresponding shutter glasses. The first receiving module receives the shutter control signal. The processing module controls the start and close of the lenses according to the preset cycle according to the shutter control signal, so that the opening of the lenses is synchronized with the display of the corresponding programs to be played. It also discloses a one-screen multi-display control method. Although the LED display screen outputs a plurality of programs to be played alternately and cyclically in sequence, the display of each program to be played is synchronized with the opening of a shutter glasses, so that multiple users watch different programs in the same time period; but it lacks the interactive capability and synchronous processing capability of dual linkage, and is independent of each other.

[0005] Prior art three, application number: 201510679414.6 discloses a method for multiple displays on one screen and its system, display device and receiving device, including: displaying a signal source corresponding to the predetermined time slot within a predetermined time slot, different predetermined time slots correspond to different signal sources, and receiving the signal source corresponding to the predetermined time slot within the predetermined time slot, thereby achieving the simultaneous display of multiple full-screen images on the same display screen while avoiding interference between users. Although it saves display screen resources and meets the viewing needs of different users, it realizes targeted user customization needs; however, it lacks the function of dual linkage, and manual switching is more complicated.

[0006] At present, the prior art 1, prior art 2 and prior art 3 have the problem that the display of multiple digital pathology sections lacks the dual linkage function, and the manual operation is cumbersome; when viewing multiple digital pathology sections on the same screen at the same time, there is a lack of dual linkage interaction and synchronization processing capabilities, and they are independent of each other and have high implementation complexity. It is possible to display multiple digital pathology sections on a single screen at the same time and provide convenient switching and interaction functions; compared with the prior art solutions, the system of the present invention is more flexible, efficient and intuitive, and can provide a better experience in digital pathology analysis and diagnosis. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a one-screen multi-image display system based on digital pathological sections, comprising:

[0008] The image preprocessing module is responsible for acquiring a number of digital pathology slices, performing full-frame digital pathology slice imaging (WSI) format conversion, and performing image enhancement and noise removal preprocessing operations on the first digital pathology slice image after the format conversion to obtain a second digital pathology slice image after the preprocessing operation;

[0009] The layout and mode setting module is responsible for acquiring the second digital pathology slice image, automatically adjusting the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtaining the third digital pathology slice image after layout; and establishing a dual linkage function mode between the third digital pathology slice images;

[0010] The operation result display module is responsible for identifying the user's identity information. After verification, it establishes a connection between the user and the screen, identifies the user's gestures, mouse or touch screen method, and performs zooming, translation and rotation operations on the third digital pathology slice image. With the support of the dual linkage function mode, it realizes the display of the area of ​​interest for observation.

[0011] Optional, layout and mode setting module, including:

[0012] The threshold setting submodule is responsible for obtaining the second digital pathology slice image to be laid out according to the screen size, and adopting two methods, namely, grid layout and tile layout, and setting the threshold for switching from grid layout to tile layout.

[0013] The image arrangement submodule is responsible for arranging the second digital pathology slice image on the screen according to the grid layout and tile layout procedures;

[0014] The image preview submodule is responsible for previewing the layout of the second digital pathology slice image on the entire screen after completing the layout of the second digital pathology slice image.

[0015] Optionally, in the threshold setting submodule, when the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds a threshold, the tile layout is started, otherwise the grid layout is performed.

[0016] Optionally, in the image arrangement submodule, the position of each second digital pathology slice image on the screen is adjusted by dragging and adjusting the margin as needed.

[0017] Optionally, the layout and mode setting module also includes:

[0018] The operation association submodule is responsible for adding an event listener to each digital pathology slice image to capture the user's operation actions on the digital pathology slice image, including zooming in, zooming out, and moving; when the user operates a digital pathology slice image, the operation action is published as an event using the event transmission mechanism, and other digital pathology slice images are subscribed as event listeners to receive and process the operation action, and the current operation action is transmitted to other synchronously displayed digital pathology slice images;

[0019] The operation execution submodule is responsible for applying the operation action to other digital pathology slice images accordingly after receiving the operation action;

[0020] The image update submodule is responsible for updating the display of the digital pathology slice image by re-adjusting the display parameters of the slice whenever an operation action is applied to other digital pathology slice images.

[0021] Optional, operation-related submodules, including:

[0022] The mechanism creation unit is responsible for creating an event listener for each digital pathology slide image to capture the user's operation actions on the image, matching a key value for each operation action, adding the response logic of the corresponding key value using the event transfer mechanism method, and moving the control logic of each operation action of each digital pathology slide image to the event transfer mechanism method to achieve adaptive operation;

[0023] The event subscription unit is responsible for encapsulating the operation action into an event object when the user operates a digital pathology slide image, and publishing the event through the event bus or event manager; by registering the corresponding event listener on each digital pathology slide image, other digital pathology slide images that need to be displayed synchronously are subscribed as event listeners;

[0024] The function calling unit is responsible for receiving and processing operation action events. After other digital pathology slice images receive the operation action events as event listeners, they call the event processing function to process the events and extract the operation action information in the events; by calling the response logic, the current operation action is passed to other synchronously displayed digital pathology slice images.

[0025] Optionally, the event processing function of the event subscription unit refers to a function used to process a specific event, and the specific event refers to operation action information.

[0026] Optional, function call unit, including:

[0027] Combined with the element preset sub-unit, it is responsible for establishing an event processing function set. Each element in the event processing function set corresponds to the user's operation action on the image, and the corresponding event processing function set is found through the operation action;

[0028] The operation action recognition subunit is responsible for analyzing the current operation action, obtaining the current operation action parameters, obtaining the pre-classified event processing function set corresponding to the current operation action parameters according to the current operation action parameters, and directly finding the corresponding event processing function set through the current operation action;

[0029] The corresponding function calling sub-unit is responsible for obtaining all event processing functions in the event processing function set that process the current operation action, and each event processing function corresponds to an event subscriber, executes the event processing functions in the event processing function set, and distributes the events to other digital pathology slice images displayed synchronously.

[0030] Optionally, the operation result display module includes:

[0031] The identity authentication submodule is responsible for verifying the user's identity information based on the pre-entered identity information. After the verification is passed, the interactive operation relationship between the user and the screen is established;

[0032] The operation recognition submodule is responsible for recognizing the user's gestures, mouse or touch screen mode, and obtaining the user's input information; judging the user's operation intention based on the user's gestures or input information, whether to perform zoom, translation or rotation operations; and selecting the area of ​​interest for operation based on the user's operation intention;

[0033] The operation execution submodule is responsible for the user's choice of zoom operation, adjusting the size of the third digital pathology slice image according to the user's gesture or input information, and performing corresponding operations on other related digital pathology slice images; the user chooses to perform a translation operation, moving the position of the third digital pathology slice image according to the user's gesture or input information; the user chooses to perform a rotation operation, rotating the angle of the third digital pathology slice image according to the user's gesture or input information.

[0034] The present invention provides a one-screen multi-image display method based on digital pathological sections, comprising the following steps:

[0035] Acquire a plurality of digital pathology slices, perform full-frame digital pathology slice imaging (WSI) format conversion, and perform image enhancement and noise removal preprocessing operations on the first digital pathology slice image after the format conversion to obtain a second digital pathology slice image after the preprocessing operation;

[0036] Acquire the second digital pathology slice image, automatically adjust the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtain the third digital pathology slice image after layout; establish a dual linkage function mode between the third digital pathology slice images;

[0037] Identify the user's identity information, and after verification, establish a connection between the user and the screen, identify the user's gestures, mouse or touch screen method, zoom, translate and rotate the third digital pathology slice image, and with the support of the dual linkage function mode, realize the display of the area of ​​interest.

[0038] The image preprocessing module of the present invention obtains several digital pathological slices, performs full-frame digital pathological slice imaging WSI format conversion, and performs image enhancement and noise removal preprocessing operations on the first digital pathological slice image after format conversion to obtain the second digital pathological slice image after preprocessing operation; the layout and mode setting module obtains the second digital pathological slice image, automatically adjusts the layout of the second digital pathological slice image according to the screen size and the number of digital pathological slices, and obtains the third digital pathological slice image after layout; establishes a dual linkage function mode between the third digital pathological slice images; the operation result display module identifies the user's identity information, and after verification, establishes a connection relationship between the user and the screen, identifies the user's gestures, mouse or touch screen, etc., and performs operations such as zooming, translation and rotation on the third digital pathological slice image, and with the support of the dual linkage function mode, realizes the display of the area of ​​interest; the above scheme provides a comprehensive and efficient image processing and display platform for the field of digital pathology. The specific significance includes: improving work efficiency: through automated image preprocessing and layout adjustment, the user's workload can be greatly reduced, and multiple digital pathological slice images can be displayed quickly and accurately, improving work efficiency. Provide better image quality: Through pre-processing operations such as image enhancement and noise removal, the quality of the first digital pathology slice image can be improved, allowing the observer to observe and analyze the image more clearly. Realize dual linkage function: By establishing a dual linkage function mode between the third digital pathology slice images, the user can zoom, translate and rotate the image through gestures, mouse or touch screen, so as to more conveniently observe the area of ​​interest. Provide security control: Through the identification and verification of identity information, it can be ensured that only authorized users can establish a connection with the screen and operate the image, thereby improving security.

[0039] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0040] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0042] Figure 1 This is a block diagram of a one-screen multi-image display system based on digital pathological sections in Example 1 of the present invention;

[0043] Figure 2 This is a block diagram of an image preprocessing module in Embodiment 2 of the present invention;

[0044] Figure 3 The layout and mode setting module frame in Embodiment 3 of the present invention Figure 1 ;

[0045] Figure 4 This is a block diagram of a threshold setting submodule in Embodiment 4 of the present invention;

[0046] Figure 5 The layout and mode setting module frame in Example 5 of the present invention Figure 2 ;

[0047] Figure 6 This is a block diagram of an operation-related submodule in Embodiment 6 of the present invention;

[0048] Figure 7 This is a block diagram of a function calling unit in Embodiment 7 of the present invention;

[0049] Figure 8 The operation result display module frame in Example 8 of the present invention Figure 1 ;

[0050] Fig. 9 The operation result display module frame in Example 9 of the present invention Figure 2 ;

[0051] Fig.10 This is a flow chart of a method for displaying multiple images on one screen based on digital pathological sections in Embodiment 10 of the present invention;

[0052] Fig.11 is a diagram of the process of obtaining the second digital pathological slice image in Example 11 of the present invention;

[0053] Fig.12 A process diagram of obtaining the third digital pathology slice images after layout and establishing a dual linkage function mode between the third digital pathology slice images in Embodiment 12 of the present invention;

[0054] Fig.13 This is a process diagram for realizing the display of an area of ​​interest for observation in Example 13 of the present invention. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0056] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0057] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0058] Example 1: Figure 1 As shown, an embodiment of the present invention provides a one-screen multi-image display system based on digital pathological sections, comprising:

[0059] The image preprocessing module is responsible for acquiring a number of digital pathology slices, performing full-frame digital pathology slice imaging (WSI) format conversion, and performing image enhancement and noise removal preprocessing operations on the first digital pathology slice image after the format conversion to obtain a second digital pathology slice image after the preprocessing operation;

[0060] The layout and mode setting module is responsible for acquiring the second digital pathology slice image, automatically adjusting the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtaining the third digital pathology slice image after layout; and establishing a dual linkage function mode between the third digital pathology slice images;

[0061] The operation result display module is responsible for identifying the user's identity information. After verification, it establishes a connection between the user and the screen, identifies the user's gestures, mouse or touch screen, etc., and performs operations such as zooming, translation and rotation on the third digital pathology slice image. With the support of the dual linkage function mode, the display of the area of ​​interest can be realized;

[0062] The working principle and beneficial effects of the above technical solution are as follows: the image preprocessing module of this embodiment obtains several digital pathology slices, performs full-frame digital pathology slice imaging WSI format conversion, and performs image enhancement and noise removal preprocessing operations on the first digital pathology slice image after format conversion to obtain the second digital pathology slice image after preprocessing operation; the layout and mode setting module obtains the second digital pathology slice image, automatically adjusts the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtains the third digital pathology slice image after layout; establishes a dual linkage function mode between the third digital pathology slice images; the operation result display module identifies the user's identity information, and after verification, establishes a connection between the user and the screen, identifies the user's gestures, mouse or touch screen, etc., and performs operations such as zooming, translating and rotating the third digital pathology slice image, and with the support of the dual linkage function mode, realizes the display of the area of ​​interest; the above solution provides a comprehensive and efficient image processing and display platform for the field of digital pathology. The specific significance includes: improving work efficiency: through automated image preprocessing and layout adjustment, the user's workload can be greatly reduced, and multiple digital pathology slice images can be displayed quickly and accurately, improving work efficiency. Provide better image quality: Through pre-processing operations such as image enhancement and noise removal, the quality of the first digital pathology slice image can be improved, allowing the observer to observe and analyze the image more clearly. Realize dual linkage function: By establishing a dual linkage function mode between the third digital pathology slice images, the user can zoom, translate and rotate the image through gestures, mouse or touch screen, so as to more conveniently observe the area of ​​interest. Provide security control: Through the identification and verification of identity information, it can be ensured that only authorized users can establish a connection with the screen and operate the image, thereby improving security.

[0063] In summary, this embodiment can improve the work efficiency and image quality in the field of digital pathology, and at the same time provide a more convenient and safer image operation method, which is of great significance to medical research and clinical diagnosis.

[0064] At the same time, this embodiment provides a multi-image display layout: through the layout management algorithm, multiple digital pathology slices are displayed on a single screen in an appropriate manner; the system can automatically adjust the layout according to the screen size and the number of digital pathology slices to ensure that the digital pathology slices can be displayed on the screen in a balanced manner and provide the best visibility and ease of use. Dual linkage function: The system of this embodiment supports dual linkage function, that is, when the doctor operates one of the digital pathology slices, the other synchronously displayed digital pathology slices will also be linked accordingly. This linkage can be operations such as zooming in, zooming out, moving or rotating to maintain the consistency and correlation between the digital pathology slices; in this way, doctors can more conveniently compare and analyze multiple digital pathology slices and improve work efficiency. Smooth interactive operation: The system of this embodiment provides a smooth interactive operation experience. Doctors can use gestures, mouse or touch screen to zoom, translate and rotate digital pathology slices to facilitate observation of areas of interest; the system can respond to operations in real time and provide smooth digital pathology slice display and interactive feedback. Powerful loading and processing capabilities: The system of this embodiment has powerful loading and processing capabilities, and can efficiently load and display multiple digital pathology slices; the system uses optimized image processing algorithms and parallel computing technology to provide fast loading and rendering of digital pathology slices, ensuring that doctors can observe and analyze the content of digital pathology slices in real time. Functional expansion and customization: The system of this embodiment has expandable and customizable features. Doctors can customize the system's interface layout, function buttons, and operation methods according to their own needs and preferences; the system can also support functions such as adding markers, measurement tools, and image filters to meet the personalized needs of different doctors.

[0065] The core of the present invention is a browser-based multi-image display system for multiple AI digital pathology slices on one screen. Through this system, doctors (users) can simultaneously display multiple pathological digital pathology slices on a single screen, and realize linkage operations, so as to more efficiently and intuitively analyze and diagnose pathological digital pathology slices. In the past, only one digital pathology slice could be placed within a viewport for display and movement, but now multiple pictures can be placed in the same viewport for synchronous operations. Specifically, it is manifested as: slice layout management: the system of this embodiment can display multiple digital pathology slices on a single screen at the same time through digital pathology slice layout management technology; the system can dynamically adjust the layout of digital pathology slices according to the screen size and resolution to achieve the best display effect. Doctors can customize the position and size of digital pathology slices as needed to meet different observation and analysis requirements. Linkage function: the system of this embodiment provides a linkage function, that is, the operations performed on the doctor's canvas (such as zooming in, zooming out, moving, etc.) will be synchronized to other displayed digital pathology slices accordingly, maintaining a consistent observation perspective and relevance; enabling doctors to more conveniently compare and analyze digital pathology slices, improving work efficiency and accuracy. Interactive operation: The system of this embodiment supports a variety of interactive operations, such as zooming, panning, and rotation; doctors can even operate digital pathology slices through gestures or mouse operations to obtain detailed information on the area of ​​interest. Multi-image synchronous loading: The system of this embodiment has an efficient multi-image synchronous loading function, which can quickly load and display multiple digital pathology slices. Doctors can view the changes and effects of digital pathology slices in real time without waiting for loading time, which improves work efficiency.

[0066] In summary, the one-screen multi-image display system of the present invention solves the technical problems of layout management, linkage, interactivity and loading efficiency encountered by traditional digital pathology systems when displaying multiple digital pathology slices through innovative technical solutions, providing doctors with a more convenient, efficient and accurate digital pathology analysis tool.

[0067] The one-screen multi-image display system of this embodiment can display multiple digital pathology sections on a single screen at the same time, and provide convenient switching and interactive functions. Compared with the existing technical solutions, the system of this embodiment is more flexible, efficient and intuitive, and can provide a better experience of digital pathology analysis and diagnosis. Improve work efficiency: Through the one-screen multi-image display system, doctors can view multiple digital pathology sections at the same time without frequent switching and adjustment of windows, thus saving valuable time. Doctors can quickly compare and analyze the differences between different digital pathology sections, improving the efficiency of pathological diagnosis and research. Improve observation accuracy: The multi-image display system allows doctors to observe multiple digital pathology sections on the same screen at the same time, which helps doctors to fully grasp the situation of pathological changes. Doctors can better compare digital pathology sections in different areas or at different time points, discover tiny lesion characteristics, and improve the accuracy of disease detection and diagnosis. Enhance diagnostic decision support: The one-screen multi-image display system provides a dual linkage function. When a doctor operates a digital pathology section, other digital pathology sections will also display corresponding changes synchronously, enabling doctors to more comprehensively understand the distribution and relationship of lesions; this provides doctors with better diagnostic decision support, and can more accurately assess the condition and formulate treatment plans. Improve user experience: The system has smooth interactive operations and fast loading and rendering capabilities, providing doctors with a good user experience. Doctors can freely operate digital pathology slides through gestures, mouse or touch screen, observe areas of interest, and obtain feedback in real time; making doctors more comfortable and convenient during digital pathology analysis. Promote collaboration and teaching: The one-screen multi-image display system supports multiple doctors to observe and analyze digital pathology slides at the same time. Doctors can share digital pathology slides on the same screen and conduct real-time discussions and exchanges to promote collaboration and knowledge sharing. In addition, the system can also be used for teaching and training to facilitate the guidance of doctors' learning and professional development.

[0068] In summary, this embodiment brings many beneficial effects to the field of digital pathology, improves work efficiency, observation accuracy and diagnostic decision support, while enhancing user experience, promoting collaboration and teaching, and providing doctors with a more powerful and comprehensive digital pathology digital pathology slice display system.

[0069] Example 2: Figure 2 As shown, based on Example 1, the image preprocessing module provided by the embodiment of the present invention includes:

[0070] The format conversion submodule is responsible for identifying the extensions or file header information of several digital pathology sections, and obtaining the format of the digital pathology section. If the format is TIFF, JPEG or PNG, the full-frame digital pathology section imaging WSI format conversion program is started to obtain the first digital pathology section image, and then enter the preprocessing operation; if the format is WSI format, the preprocessing operation is directly entered;

[0071] The full-frame digital pathology slice imaging WSI format conversion program includes: Image cutting: cutting digital pathology slices in TIFF, JPEG or PNG format into multiple small blocks, encoding each small block, using the encoding method required for converting JPEG2000 to WSI format, adding metadata to each encoded small block, and the metadata includes information such as the position, size and zoom level of the digital pathology slice; aligning the encoded small blocks according to their position and order in the original digital pathology slice using image registration to ensure seamless connection during splicing, and splicing the aligned digital pathology slices to form a complete digital pathology slice in WSI format;

[0072] The image enhancement submodule is responsible for converting the digital pathology slices in WSI format into grayscale images, calculating the pixel value distribution of the grayscale image, and generating the histogram of the original image; performing accumulation operations on the histogram to obtain the cumulative distribution function of the grayscale level, and calculating the pixel value mapping table corresponding to each grayscale level according to the cumulative distribution function. Using the pixel value mapping table, each pixel value in the digital pathology slice is replaced with the corresponding new pixel value to obtain the digital pathology slice in WSI format after image enhancement;

[0073] The image denoising submodule is responsible for obtaining the WSI format digital pathology slice after image enhancement, calculating and sorting the neighborhood pixel values ​​around each pixel, comparing the neighborhood blocks around each pixel with other blocks in the entire digital pathology slice, and calculating the average value based on the similarity. The average value is used as the new value of the pixel, and the filter window is used to remove noise to obtain the second digital pathology slice image after preprocessing operations;

[0074] The working principle and beneficial effects of the above technical solution are as follows: the format conversion submodule of this embodiment identifies the extensions or file header information of several digital pathology slices, and obtains the format of the digital pathology section. If the format is TIFF, JPEG or PNG format, the full-frame digital pathology slice imaging WSI format conversion program is started to obtain the first digital pathology slice image, and then enter the preprocessing operation; if the format is WSI format, the preprocessing operation is directly entered; the full-frame digital pathology slice imaging WSI format conversion program includes: image cutting: cutting the digital pathology slices in TIFF, JPEG or PNG format into multiple small blocks, encoding each small block, using the encoding method required for converting JPEG2000 to WSI format, adding metadata to each encoded small block, and the metadata includes information such as the position, size and zoom level of the digital pathology slice; aligning the encoded small blocks according to their position and order in the original digital pathology slice using image registration to ensure seamless connection during splicing, and the aligned digital The pathological slices are spliced ​​to form a complete digital pathological slice in WSI format; the image enhancement submodule converts the digital pathological slice in WSI format into a grayscale image, calculates the pixel value distribution of the grayscale image, and generates a histogram of the original image; the histogram is accumulated to obtain the cumulative distribution function of the grayscale level, and the pixel value mapping table corresponding to each grayscale level is calculated according to the cumulative distribution function. Using the pixel value mapping table, each pixel value in the digital pathological slice is replaced with the corresponding new pixel value to obtain the digital pathological slice in WSI format after image enhancement; the image denoising submodule obtains the digital pathological slice in WSI format after image enhancement, calculates the neighborhood pixel value around each pixel, and sorts them, compares the neighborhood blocks around each pixel with other blocks in the entire digital pathological slice, and calculates the average value according to the similarity, and the average value is used as the new value of the pixel. The filter window is used to remove noise to obtain the second digital pathological slice image after preprocessing operation; the above scheme is to perform preprocessing operations on digital pathological slices to improve image quality and accuracy. Specifically, it includes submodules such as format conversion, image cutting, image enhancement, and image denoising. First, the format conversion submodule can automatically identify the format of digital pathology slices and convert non-WSI format slices to WSI format, which helps to unify the format of digital pathology slices and facilitate subsequent processing. Secondly, the image cutting submodule cuts large-sized digital pathology slices into multiple small pieces, and encodes and adds metadata to each small piece for subsequent image registration and splicing, which helps to process large-sized slices and ensure seamless connection during splicing. Next, the image enhancement submodule converts digital pathology slices in WSI format into grayscale images, and enhances the images through histogram analysis and calculation of pixel value mapping tables, which can improve the contrast and clarity of the images, and enhance the image visualization and information extraction capabilities.Finally, the image denoising submodule further improves the image quality by calculating the pixel values ​​of the neighborhood around the pixel and using a filter window to remove noise, which helps to reduce noise interference in the image and make the image clearer and more accurate.

[0075] This embodiment performs comprehensive preprocessing operations on digital pathology slices to improve image quality, reduce noise interference, enhance image information, and provide a more reliable basis for subsequent pathology analysis and diagnosis.

[0076] Example 3: Figure 3 As shown, based on Example 1, the layout and mode setting module provided in this embodiment of the present invention includes:

[0077] The threshold setting submodule is responsible for obtaining the second digital pathology slice image to be laid out according to the screen size, and adopting two methods, namely, grid layout and tile layout, to lay out the image. At the same time, a threshold for switching from grid layout to tile layout is set. When the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds the threshold, the tile layout is started, otherwise the grid layout is performed.

[0078] The image arrangement submodule is responsible for arranging the second digital pathology slice images on the screen according to the grid layout and tile layout procedures; as needed, the position of each second digital pathology slice image on the screen is adjusted by dragging, adjusting the margins, etc.;

[0079] An image preview submodule, responsible for previewing the layout of the second digital pathology slice image on the entire screen after completing the layout of the second digital pathology slice image;

[0080] The working principle and beneficial effects of the above technical solution are as follows: the threshold setting submodule of the present embodiment obtains and receives the second digital pathology slice image to be laid out according to the screen size, and adopts two methods of grid layout and tile layout for layout, and sets a threshold for switching from grid layout to tile layout at the same time. When the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds the threshold, the tile layout is started, otherwise the grid layout is performed; the image arrangement submodule arranges the second digital pathology slice image on the screen according to the grid layout and tile layout procedures; the position of each second digital pathology slice image on the screen is adjusted by dragging, adjusting the margin, etc. as needed; after completing the layout of the second digital pathology slice image, the image preview submodule previews the layout of the second digital pathology slice image on the entire screen; the above scheme realizes automatic selection of grid layout or tile layout through threshold setting according to the screen size and the number of pixels of the second digital pathology slice image, so as to better adapt to the display space on the screen; it can ensure that the display effect of the second digital pathology slice image on the screen is more reasonable and beautiful. Through the image arrangement submodule, the second digital pathology slice image can be arranged on the screen according to the grid layout or tile layout program, so that each image can get the appropriate position and spacing, which is convenient for users to observe and analyze. At the same time, the user can adjust the position of each image by dragging and adjusting the margins to meet personalized needs. Finally, the image preview submodule can provide a preview of the layout of the second digital pathology slice image on the entire screen, helping users to confirm the layout effect and make necessary adjustments.

[0081] In summary, the implementation significance of this embodiment is to provide an intelligent digital pathology slice image layout solution, which can automatically select a suitable layout mode according to the screen size and the number of image pixels, thereby improving the image display effect and user experience.

[0082] Example 4: Figure 4 As shown, based on Example 3, the threshold setting submodule provided in this embodiment of the present invention includes:

[0083] The parameter acquisition unit is responsible for obtaining the width and height of the screen according to the screen size, obtaining the width and height of the preset grid unit corresponding to the current flatness; and obtaining the number of pixels, width and height of the second digital pathology slice image;

[0084] The first parameter calculation unit is responsible for calculating the number of rows and columns of the second digital pathology slice image in the current screen layout according to the number of pixels, width and height of the second digital pathology slice image and the width and height of the grid unit; when the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds a threshold, the tile layout is started; the specific calculation formulas for the number of columns and the number of rows are as follows:

[0085] Number of columns = number of pixels in a slice / (width of each grid unit on the screen * width of each slice image);

[0086] Number of rows = number of pixels in the slice / (height of each grid unit on the screen * height of each slice image)

[0087] The second parameter calculation unit is responsible for calculating the second digital pathology slice image according to the number of rows, the number of columns, the width and the height of the screen, and implementing the size of each second digital pathology slice image in the tiled layout, that is, the width and height of each second digital pathology slice image in the tiled layout; the size calculation formula is as follows:

[0088] The width of each second digital pathology slice image in the tile layout = screen width / number of columns;

[0089] The height of each second digital pathology slice image in the tile layout = screen height / number of rows;

[0090] The working principle and beneficial effects of the above technical solution are as follows: the parameter acquisition unit of this embodiment obtains the width and height of the screen according to the screen size, and obtains the width and height of the preset grid unit corresponding to the current flatness; obtains the number of pixels, width and height of the second digital pathology slice image; the first parameter calculation unit calculates the number of rows and columns of the second digital pathology slice image of the current screen layout according to the number of pixels, width and height of the second digital pathology slice image, and the width and height of the grid unit; when the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds the threshold, the tiled layout is started; the specific calculation formulas for the number of columns and the number of rows are as follows: number of columns = number of pixels of the slice / (width of each grid unit on the screen*width of each slice image); number of rows = number of pixels of the slice / (height of each grid unit on the screen*height of each slice image) The second parameter calculation unit calculates the number of rows and columns according to the number of rows, the number of columns, the width and the height of the screen The second digital pathology slice image is calculated, and the size of each second digital pathology slice image in the tiled layout is implemented, that is, the width and height of each second digital pathology slice image in the tiled layout; the size calculation formula is as follows: the width of each second digital pathology slice image in the tiled layout = screen width / number of columns; the height of each second digital pathology slice image in the tiled layout = screen height / number of rows; the above scheme calculates the appropriate size of each second digital pathology slice image in the tiled layout based on the given screen size, number of rows, number of columns and image size, in order to ensure that each second digital pathology slice image occupies an appropriate space in the layout, so that the second digital pathology slice image can be fully displayed and the layout looks neat and harmonious; by calculating the width and height of each second digital pathology slice image, it can be ensured that the size of each second digital pathology slice image is equal, thereby making the layout more beautiful and the visual analysis more convenient.

[0091] Example 5: Figure 5 As shown, based on Example 1, the layout and mode setting module of the embodiment of the present invention further includes:

[0092] The operation association submodule is responsible for adding an event listener to each digital pathology slice image to capture the user's operation actions on the digital pathology slice image, including zooming in, zooming out, and moving. When the user operates a digital pathology slice image, the operation action is published as an event using the event transmission mechanism, and other digital pathology slice images are subscribed as event listeners to receive and process the operation action, and the current operation action is transmitted to other synchronously displayed digital pathology slice images.

[0093] The operation execution submodule is responsible for applying the operation action to other digital pathology slice images accordingly after receiving the operation action;

[0094] The image update submodule is responsible for updating the display of the digital pathology slice image by re-adjusting the display parameters of the slice whenever an operation action is applied to other digital pathology slice images;

[0095] The working principle and beneficial effects of the above technical solution are as follows: the operation association submodule of this embodiment adds an event listener to each digital pathology slice image to capture the user's operation actions on the digital pathology slice image, including zooming in, zooming out and moving; when the user operates a digital pathology slice image, the event transmission mechanism is used to publish the operation action as an event, and subscribes to other digital pathology slice images as event listeners to receive and process the operation action, and transmit the current operation action to other synchronously displayed digital pathology slice images; after the operation execution submodule receives the operation action, other digital pathology slice images need to apply the operation action accordingly; whenever the image update submodule is applied to other digital pathology slice images, the display of the digital pathology slice image is updated by re-adjusting the display parameters of the slice; the above scheme establishes a dual linkage function mode, by capturing the user's operation action on a digital pathology slice image and applying it to other synchronously displayed digital pathology slice images, the consistency and correlation between digital pathology slice images are achieved. The dual linkage function mode has the following significance: Provide a better visualization experience: By applying the operation action to other digital pathology slice images, doctors can zoom in, zoom out or move on a digital pathology slice image, and other images will change accordingly, providing a more comprehensive perspective and better observation effect. Improve work efficiency: The dual linkage function mode enables doctors to quickly switch and compare between multiple digital pathology slice images, reducing the time of switching windows and adjusting parameters, and improving work efficiency. Increase diagnostic accuracy: Through the dual linkage function mode, doctors can compare and analyze between multiple digital pathology slice images, more comprehensively evaluate the characteristics of lesions, and improve the accuracy of diagnosis.

[0096] This embodiment establishes a dual linkage function mode between the third digital pathology slice images, which can provide a better visualization experience, improve work efficiency and increase diagnostic accuracy, and is of great significance to doctors' digital pathology slice image analysis and diagnosis work.

[0097] Example 6: Figure 6 As shown, based on Example 5, the operation association submodule provided in this embodiment of the present invention includes:

[0098] The mechanism creation unit is responsible for creating an event listener for each digital pathology slide image to capture the user's operation actions on the image, matching a key value for each operation action, adding the response logic of the corresponding key value using the event transfer mechanism method, and moving the control logic of each operation action of each digital pathology slide image to the event transfer mechanism method to achieve adaptive operation;

[0099] The event subscription unit is responsible for encapsulating the operation action into an event object when the user operates a digital pathology slide image, and publishing the event through the event bus or event manager; by registering the corresponding event listener on each digital pathology slide image, other digital pathology slide images that need to be displayed synchronously are subscribed as event listeners;

[0100] The function calling unit is responsible for receiving and processing operation action events. After receiving the operation action events as event listeners, other digital pathology slice images call the event processing function to process the events and extract the operation action information in the events; the current operation action is passed to other synchronously displayed digital pathology slice images by calling the response logic; the event processing function refers to a function used to process a specific event; in this embodiment, the event processing function is used to extract the operation action information in the event;

[0101] The working principle and beneficial effects of the above technical solution are as follows: the mechanism creation unit of this embodiment creates an event listener for each digital pathology slice image, which is used to capture the user's operation actions on the image, matches a key value to each operation action, and uses the event transmission mechanism method to add the response logic of the corresponding key value, and moves the control logic of each operation action of each digital pathology slice image to the event transmission mechanism method to achieve adaptive operation; when the user operates a digital pathology slice image, the event subscription unit encapsulates the operation action into an event object and publishes the event through the event bus or event manager; by registering the corresponding event listener on each digital pathology slice image, other digital pathology slices that need to be displayed synchronously are encapsulated into an event object. The image of the digital pathology slice is subscribed as an event listener; the function calling unit receives and processes the operation action event, and other digital pathology slice images receive the operation action event as event listeners, call the event processing function to process the event, and extract the operation action information in the event; pass the current operation action to other synchronously displayed digital pathology slice images by calling the response logic; the event processing function refers to a function used to process a specific event; in this embodiment, the event processing function is used to extract the operation action information in the event; the above scheme realizes the synchronous display and operation control of digital pathology slice images; by creating an event listener for each image, the user's operation action on the image can be captured, and the operation action can be published as an event through the event transmission mechanism. Other images that need to be displayed synchronously can receive and process the operation action event by subscribing to these event listeners to realize the synchronous operation of the image. The significance of the mechanism is that the user can perform operations such as zooming in, zooming out and moving on an image, and then these operations will be passed to other synchronously displayed images, so that they also perform the same operation, thereby realizing the synchronous display and control between multiple images; it is very useful for the comparison and analysis of digital pathology slice images, and it can easily compare the subtle differences between different images, and improve the accuracy and efficiency of pathology analysis.

[0102] Example 7: Figure 7 As shown, based on Example 6, the function calling unit provided in this embodiment of the present invention includes:

[0103] Combined with the element preset sub-unit, it is responsible for establishing an event processing function set. Each element in the event processing function set corresponds to the user's operation action on the image, and the corresponding event processing function set is found through the operation action;

[0104] The operation action recognition subunit is responsible for analyzing the current operation action, obtaining the current operation action parameters, obtaining the pre-classified event processing function set corresponding to the current operation action parameters according to the current operation action parameters, and directly finding the corresponding event processing function set through the current operation action;

[0105] The corresponding function calling subunit is responsible for obtaining all event processing functions in the event processing function set that process the current operation action, and each event processing function corresponds to an event subscriber, executing the event processing function in the event processing function set, and distributing the event to other synchronously displayed digital pathology slice images;

[0106] The working principle and beneficial effects of the above technical solution are as follows: the present embodiment combines the element preset subunit to establish an event processing function set, each element in the event processing function set corresponds to the user's operation action on the image, and the corresponding event processing function set is found through the operation action; the operation action recognition subunit analyzes the current operation action, obtains the current operation action parameters, obtains the pre-classified event processing function set corresponding to the current operation action parameters according to the current operation action parameters, and directly finds the corresponding event processing function set through the current operation action; the corresponding function calling subunit obtains all event processing functions in the event processing function set that process the current operation action, and each event processing function corresponds to an event subscriber, and executes the event processing function The event processing function in the event processing function set distributes the event to other synchronously displayed digital pathology slice images; the above scheme realizes the recognition of user operation actions and the automation of event processing through the event processing function set and the operation action recognition subunit; through the pre-classified event processing function set, the event processing function set corresponding to the current operation action can be quickly found, so as to extract the operation action information in the event, which can simplify the code logic, improve the development efficiency, and also facilitate the expansion and modification of the operation action; by executing the event processing function set, the event can be distributed to other synchronously displayed digital pathology slice images, so as to realize the synchronous display effect between images, improve the user experience, and provide better image browsing and operation functions.

[0107] Example 8: Figure 8 As shown, based on Example 1, the operation result display module provided by the embodiment of the present invention includes:

[0108] The identity authentication submodule is responsible for verifying the user's identity information based on the pre-entered identity information. After the verification is passed, the interactive operation relationship between the user and the screen is established;

[0109] The operation recognition submodule is responsible for recognizing the user's gestures, mouse or touch screen, etc., to obtain the user's input information; according to the user's gestures or input information, it is determined whether the user's operation intention is to perform zooming, translation or rotation operations; according to the user's operation intention, the user selects the area of ​​interest to perform the operation;

[0110] The operation execution submodule is responsible for the user's selection of zoom operation, adjusting the size of the third digital pathology slice image according to the user's gesture or input information, and performing corresponding operations on other related digital pathology slice images; the user's selection of translation operation, moving the position of the third digital pathology slice image according to the user's gesture or input information; the user's selection of rotation operation, rotating the angle of the third digital pathology slice image according to the user's gesture or input information;

[0111] The working principle and beneficial effects of the above technical solution are as follows: the identity authentication submodule of this embodiment verifies the identity information of the user according to the pre-entered identity information, and after the verification is passed, establishes an interactive operation relationship between the user and the screen; the operation recognition submodule recognizes the user's gestures, mouse or touch screen, etc., to obtain the user's input information; according to the user's gestures or input information, it is determined whether the user's operation intention is to perform zooming, translation or rotation operations. According to the user's operation intention, the user selects the area of ​​interest for operation; the user chooses to perform a zoom operation in the operation execution submodule, and adjusts the size of the third digital pathology slice image according to the user's gestures or input information, and performs corresponding operations on other related digital pathology slice images; the user chooses to perform a translation operation, and moves the position of the third digital pathology slice image according to the user's gestures or input information; the user chooses to perform a rotation operation, and rotates the angle of the third digital pathology slice image according to the user's gestures or input information; the operation result display module recognizes the user's identity information, and after verification, establishes a connection relationship between the user and the screen, recognizes the user's gestures, mouse or touch screen, etc., and performs zooming, translation and rotation operations on the third digital pathology slice image. , with the support of the dual linkage function mode, the display of the area of ​​interest can be observed; the above scheme realizes an interactive operating system for digital pathology slice images, and ensures that only verified users can perform operations through the identity authentication submodule, ensuring the security of the system; the operation recognition submodule can accurately identify the user's operation intention, provide a variety of operation methods, and enhance the user's operation experience; the operation execution submodule can perform corresponding zoom, translation and rotation operations on the image according to the user's operation intention, so that the user can freely observe the area of ​​interest, improving the display effect and usability of the image; the operation result display module presents the operation results to the user in a visual way, allowing the user to intuitively see the effect of the operation, providing good user feedback. This embodiment provides a convenient, safe and efficient digital pathology slice image interactive operating system, which provides powerful tools and support for doctors and researchers, and helps to improve the diagnosis and research efficiency of pathology slices.

[0112] Example 9: Fig. 9 As shown, based on Example 1, the operation result display module provided in this embodiment of the present invention further includes:

[0113] The interactive operation submodule is responsible for providing an interactive operation experience. The user uses gestures, a mouse or a touch screen to zoom, translate and rotate the third digital pathology slice to observe the area of ​​interest. It can respond to operations in real time and provide digital pathology slice display and interactive feedback.

[0114] The image processing submodule is responsible for reading the third digital pathology slice image data and loading it into the memory for brightness and contrast adjustment operations, decomposing the image processing task into multiple subtasks and processing them on multiple computing units at the same time; using multi-threading technology to place image loading and display operations in different threads, and perform synchronization and communication between threads to achieve simultaneous loading and display of multiple third digital pathology slice images; it also supports image marking, measurement tools, image filters and other functions;

[0115] The customization submodule is responsible for supporting the functional expansion and customization of the system. Users can customize the system's interface layout, function buttons, operation methods and other functional modules according to their own needs and preferences.

[0116] The working principle and beneficial effects of the above technical solution are as follows: the interactive operation submodule of this embodiment provides an interactive operation experience, and the user uses gestures, a mouse or a touch screen to perform operations such as zooming, translating and rotating the third digital pathology slice to observe the area of ​​interest; it can respond to operations in real time and provide digital pathology slice display and interactive feedback; the image processing submodule reads the third digital pathology slice image data and loads it into the memory for adjustment operations such as brightness and contrast, decomposes the image processing task into multiple subtasks, and processes it on multiple computing units at the same time; multi-threading technology is used to place image loading and display operations in different threads, and synchronization and communication between threads are performed to achieve simultaneous loading and display of multiple third digital pathology slice images; it also supports functions such as image marking, measurement tools and image filters; the customization submodule supports system function expansion and customization, and users customize the system interface layout, function buttons and operation methods according to their own needs and preferences; the above solution provides an efficient, flexible and highly interactive digital pathology slice loading and display module; by adopting optimized image processing algorithms and parallel computing technology, the image loading and processing speed can be accelerated and the response performance of the system can be improved. At the same time, using the interactive operation submodule, users can zoom, translate and rotate the slices through gestures, mouse or touch screen, observe the area of ​​interest, and provide a better user experience. In addition, the image processing submodule provides image processing functions such as brightness and contrast, and can adjust the image according to user needs, making the image clearer and easier to observe; the application of multi-threading technology can load and display multiple slices at the same time, improving work efficiency. At the same time, providing functions such as image marking, measurement tools and image filters can assist doctors in pathological analysis and diagnosis. Finally, with the support of the customized submodule, the system can be customized according to the needs and preferences of the user, and the interface layout, function buttons and operation methods can be customized, so that the system is more in line with individual needs and improves user satisfaction and work efficiency. This embodiment provides an efficient, flexible and interactive digital pathology slice loading and display module, which provides a better working platform for doctors and pathologists and promotes the development of pathological research and clinical diagnosis.

[0117] Example 10: Fig.10 As shown, based on Examples 1 to 9, the one-screen multi-image display method based on digital pathological sections provided by the embodiments of the present invention comprises the following steps:

[0118] S100: Acquire a plurality of digital pathology slices, perform full-frame digital pathology slice imaging (WSI) format conversion, and perform image enhancement and noise removal preprocessing operations on the first digital pathology slice image after the format conversion to obtain a second digital pathology slice image after the preprocessing operation;

[0119] S200: Acquire a second digital pathology slice image, automatically adjust the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtain a third digital pathology slice image after the layout; establish a dual linkage function mode between the third digital pathology slice images;

[0120] S300: Identify the identity information of the user, and after verification, establish a connection relationship between the user and the screen, identify the user's gestures, mouse or touch screen, etc., perform operations such as zooming, translating and rotating the third digital pathological slice image, and realize the display of the area of ​​interest under the support of the dual linkage function mode;

[0121] The working principle and beneficial effects of the above technical solution are as follows: the embodiment first obtains a number of digital pathology slices, performs full-frame digital pathology slice imaging WSI format conversion, and performs image enhancement and noise removal preprocessing operations on the first digital pathology slice image after format conversion to obtain a second digital pathology slice image after preprocessing; secondly, obtains the second digital pathology slice image, automatically adjusts the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtains the third digital pathology slice image after layout; establishes a dual linkage function mode between the third digital pathology slice images; finally, identifies the user's identity information, and after verification, establishes a connection between the user and the screen, identifies the user's gestures, mouse or touch screen, etc., and performs operations such as zooming, translating and rotating the third digital pathology slice image. With the support of the dual linkage function mode, the display of the area of ​​interest can be realized; the above solution provides a comprehensive and efficient image processing and display platform for the field of digital pathology. The specific significance includes: improving work efficiency: through automated image preprocessing and layout adjustment, the user's workload can be greatly reduced, and multiple digital pathology slice images can be displayed quickly and accurately, improving work efficiency. Provide better image quality: Through pre-processing operations such as image enhancement and noise removal, the quality of the first digital pathology slice image can be improved, allowing the observer to observe and analyze the image more clearly. Realize dual linkage function: By establishing a dual linkage function mode between the third digital pathology slice images, the user can zoom, translate and rotate the image through gestures, mouse or touch screen, so as to more conveniently observe the area of ​​interest. Provide security control: Through the identification and verification of identity information, it can be ensured that only authorized users can establish a connection with the screen and operate the image, thereby improving security.

[0122] In summary, this embodiment can improve the work efficiency and image quality in the field of digital pathology, and at the same time provide a more convenient and safer image operation method, which is of great significance to medical research and clinical diagnosis.

[0123] Example 11: Fig.11As shown, based on Example 10, the process of acquiring the second digital pathological slice image provided by the embodiment of the present invention includes the following steps:

[0124] S101: Identify the extensions or file header information of several digital pathology sections, and obtain the format of the digital pathology section. If the format is TIFF, JPEG or PNG, start the full-frame digital pathology section imaging WSI format conversion program to obtain the first digital pathology section image, and then enter the preprocessing operation; if the format is WSI format, directly enter the preprocessing operation;

[0125] The full-frame digital pathology slice imaging WSI format conversion program includes: Image cutting: cutting digital pathology slices in TIFF, JPEG or PNG format into multiple small blocks, encoding each small block, using the encoding method required for converting JPEG2000 to WSI format, adding metadata to each encoded small block, and the metadata includes information such as the position, size and zoom level of the digital pathology slice; aligning the encoded small blocks according to their position and order in the original digital pathology slice using image registration to ensure seamless connection during splicing, and splicing the aligned digital pathology slices to form a complete digital pathology slice in WSI format;

[0126] S102: converting the digital pathology slice in WSI format into a grayscale image, calculating the pixel value distribution of the grayscale image, and generating a histogram of the original image; performing a cumulative operation on the histogram to obtain a cumulative distribution function of the grayscale level, and calculating a pixel value mapping table corresponding to each grayscale level according to the cumulative distribution function, and using the pixel value mapping table, replacing each pixel value in the digital pathology slice with a corresponding new pixel value, and obtaining a digital pathology slice in WSI format after image enhancement;

[0127] S103: obtaining a digital pathology slice in WSI format after image enhancement, calculating and sorting the neighborhood pixel values ​​around each pixel, comparing the neighborhood blocks around each pixel with other blocks in the entire digital pathology slice, and calculating an average value based on similarity, using the average value as a new value of the pixel, and using a filter window to remove noise, thereby obtaining a second digital pathology slice image after preprocessing operations;

[0128] The working principle and beneficial effects of the above technical solution are as follows: This embodiment first identifies the extensions or file header information of several digital pathology slices, and obtains the format of the digital pathology section. If the format is TIFF, JPEG or PNG format, the full-frame digital pathology slice imaging WSI format conversion program is started to obtain the first digital pathology slice image, and then enter the preprocessing operation; if the format is WSI format, the preprocessing operation is directly entered; the full-frame digital pathology slice imaging WSI format conversion program includes: image cutting: cutting the digital pathology slices in TIFF, JPEG or PNG format into multiple small blocks, encoding each small block, using the encoding method required for converting JPEG2000 to WSI format, adding metadata to each encoded small block, and the metadata includes information such as the position, size and zoom level of the digital pathology slice; aligning each digital pathology slice according to its position and order in the original digital pathology slice using image registration to ensure seamless connection during splicing, and The digital pathology slices are spliced ​​to form a complete digital pathology slice in WSI format; secondly, the digital pathology slice in WSI format is converted into a grayscale image, the pixel value distribution of the grayscale image is calculated, and the histogram of the original image is generated; the histogram is accumulated to obtain the cumulative distribution function of the grayscale level, and the pixel value mapping table corresponding to each grayscale level is calculated according to the cumulative distribution function. Using the pixel value mapping table, each pixel value in the digital pathology slice is replaced with the corresponding new pixel value to obtain the digital pathology slice in WSI format after image enhancement; finally, the digital pathology slice in WSI format after image enhancement is obtained, the neighborhood pixel value around each pixel is calculated and sorted, the neighborhood block around each pixel is compared with other blocks in the entire digital pathology slice, and the average value is calculated according to the similarity. The average value is used as the new value of the pixel, and the filter window is used to remove noise to obtain the second digital pathology slice image after preprocessing operation; the above scheme is to perform preprocessing operations on digital pathology slices to improve image quality and accuracy. Specifically, it includes submodules such as format conversion, image cutting, image enhancement, and image denoising. First, the format conversion submodule can automatically identify the format of digital pathology slices and convert non-WSI format slices to WSI format, which helps to unify the format of digital pathology slices and facilitate subsequent processing. Secondly, the image cutting submodule cuts large-sized digital pathology slices into multiple small pieces, and encodes and adds metadata to each small piece for subsequent image registration and splicing, which helps to process large-sized slices and ensure seamless connection during splicing. Next, the image enhancement submodule converts digital pathology slices in WSI format into grayscale images, and enhances the images through histogram analysis and calculation of pixel value mapping tables, which can improve the contrast and clarity of the images, and enhance the image visualization and information extraction capabilities.Finally, the image denoising submodule further improves the image quality by calculating the pixel values ​​of the neighborhood around the pixel and using a filter window to remove noise, which helps to reduce noise interference in the image and make the image clearer and more accurate.

[0129] This embodiment performs comprehensive preprocessing operations on digital pathology slices to improve image quality, reduce noise interference, enhance image information, and provide a more reliable basis for subsequent pathology analysis and diagnosis.

[0130] Example 12: Fig.12 As shown, based on Example 10, the process of obtaining the third digital pathology slice images after layout and establishing the dual linkage function mode between the third digital pathology slice images provided in the embodiment of the present invention includes the following steps:

[0131] S201: receiving a second digital pathology slice image to be laid out according to the screen size, and performing layout in two ways: a grid layout and a tile layout, and setting a threshold for switching from the grid layout to the tile layout. When the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds the threshold, the tile layout is started, otherwise the grid layout is performed;

[0132] S202: Arranging the second digital pathology slice images on the screen according to the grid layout and tile layout procedures; adjusting the position of each second digital pathology slice image on the screen by dragging, adjusting the margin, etc. as needed;

[0133] S203: After completing the layout of the second digital pathology slice image, previewing the layout of the second digital pathology slice image on the entire screen;

[0134] S204: adding an event listener to each digital pathology slice image to capture the user's operation actions on the digital pathology slice image, including zooming in, zooming out, and moving; when the user operates a digital pathology slice image, the operation action is published as an event using the event delivery mechanism, and other digital pathology slice images are subscribed as event listeners to receive and process the operation action, and the current operation action is delivered to other synchronously displayed digital pathology slice images;

[0135] S205: After receiving the operation action, other digital pathology slice images need to apply the operation action accordingly;

[0136] S206: Whenever the operation action is applied to other digital pathology slice images, the display of the digital pathology slice images is updated by re-adjusting the display parameters of the slices;

[0137] The working principle and beneficial effects of the above technical solution are as follows: the present embodiment first obtains and receives the second digital pathology slice image to be laid out according to the screen size, and adopts two methods, namely, grid layout and tile layout, for layout, and sets a threshold for switching from grid layout to tile layout. When the ratio of the number of columns to the number of rows of the second digital pathology slice image in the grid layout exceeds the threshold, the tile layout is started, otherwise the grid layout is performed; secondly, the second digital pathology slice images are arranged on the screen according to the procedures of the grid layout and the tile layout; as needed, the position of each second digital pathology slice image on the screen is adjusted by dragging, adjusting the margins, etc.; then, after completing the layout of the second digital pathology slice image, the layout of the second digital pathology slice image on the entire screen is previewed; an event listener is added to each digital pathology slice image to capture the user's operation actions on the digital pathology slice image, including Including zooming in, zooming out and moving, etc.; when a user operates a digital pathology slice image, the operation action is published as an event using the event delivery mechanism, and other digital pathology slice images are subscribed as event listeners to receive and process the operation action, and the current operation action is passed to other synchronously displayed digital pathology slice images; then after receiving the operation action, other digital pathology slice images need to apply the operation action accordingly; finally, whenever the operation action is applied to other digital pathology slice images, the display of the digital pathology slice image is updated by re-adjusting the display parameters of the slice; the above scheme automatically selects the grid layout or the tile layout according to the screen size and the number of pixels of the second digital pathology slice image through the threshold setting, so as to better adapt to the display space on the screen; it can ensure that the display effect of the second digital pathology slice image on the screen is more reasonable and beautiful. Through the image arrangement submodule, the second digital pathology slice image can be arranged on the screen according to the grid layout or tile layout program, so that each image can get the appropriate position and spacing, which is convenient for users to observe and analyze. At the same time, the user can adjust the position of each image by dragging and adjusting the margins to meet personalized needs. Finally, the image preview submodule can provide a preview of the layout of the second digital pathology slice image on the entire screen, helping users to confirm the layout effect and make necessary adjustments; a dual linkage function mode is established, which captures the user's operation actions on a digital pathology slice image and applies it to other synchronously displayed digital pathology slice images, achieving consistency and correlation between digital pathology slice images. The dual linkage function mode has the following significance: Provide a better visualization experience: By applying the operation actions to other digital pathology slice images, doctors can zoom in, zoom out or move on a digital pathology slice image, and other images will change accordingly, providing a more comprehensive perspective and better observation effect.Improve work efficiency: The dual linkage function mode allows doctors to quickly switch and compare multiple digital pathology slice images, reducing the time for switching windows and adjusting parameters, and improving work efficiency. Increase diagnostic accuracy: Through the dual linkage function mode, doctors can compare and analyze multiple digital pathology slice images, more comprehensively evaluate lesion characteristics, and improve diagnostic accuracy.

[0138] In summary, the implementation significance of this embodiment is to provide an intelligent digital pathology slice image layout solution, which can automatically select a suitable layout mode according to the screen size and the number of image pixels, and improve the image display effect and user experience. This embodiment establishes a dual linkage function mode between the third digital pathology slice images, which can provide a better visualization experience, improve work efficiency and increase diagnostic accuracy, and is of great significance to the doctor's digital pathology slice image analysis and diagnosis work.

[0139] Example 13: Fig.13 As shown, based on Example 10, the process of realizing the display of the area of ​​interest in the embodiment of the present invention includes the following steps:

[0140] S301: verifying the identity information of the user according to the pre-entered identity information, and establishing an interactive operation relationship between the user and the screen after the verification is passed;

[0141] S302: Recognize the user's gesture, mouse or touch screen, etc., to obtain the user's input information; determine the user's operation intention is to perform a zoom, translation or rotation operation according to the user's gesture or input information; according to the user's operation intention, the user selects an area of ​​interest to perform an operation;

[0142] S303: The user selects to perform a zoom operation, and the size of the third digital pathology slice image is adjusted according to the user's gesture or input information, and corresponding operations are performed on other related digital pathology slice images; the user selects to perform a translation operation, and the position of the third digital pathology slice image is moved according to the user's gesture or input information; the user selects to perform a rotation operation, and the angle of the third digital pathology slice image is rotated according to the user's gesture or input information;

[0143] The working principle and beneficial effects of the above technical solution are as follows: This embodiment first verifies the user's identity information according to the pre-entered identity information, and after the verification is passed, establishes an interactive operation relationship between the user and the screen; secondly, identifies the user's gestures, mouse or touch screen, etc. to obtain the user's input information; based on the user's gestures or input information, determines whether the user's operation intention is to perform zooming, translation or rotation operations. According to the user's operation intention, the user selects the area of ​​interest for operation; finally, the user chooses to perform a zoom operation, and according to the user's gestures or input information, adjusts the size of the third digital pathology slice image, and performs corresponding operations on other related digital pathology slice images; the user chooses to perform a translation operation, and according to the user's gestures or input information, moves the position of the third digital pathology slice image; the user chooses to perform a rotation operation, and according to the user's gestures or input information, rotates the angle of the third digital pathology slice image; the operation result display module identifies the user's identity information, and after verification, establishes a connection relationship between the user and the screen, identifies the user's gestures, mouse or touch screen, etc., and performs zooming, translation and rotation operations on the third digital pathology slice image, in the double With the support of the linkage function mode, the display of the area of ​​interest can be observed; the above scheme realizes an interactive operating system for digital pathology slice images, and ensures that only verified users can perform operations through the identity authentication submodule, thereby ensuring the security of the system; the operation recognition submodule can accurately identify the user's operation intention, provide multiple operation modes, and enhance the user's operation experience; the operation execution submodule can perform corresponding zoom, translation and rotation operations on the image according to the user's operation intention, so that the user can freely observe the area of ​​interest, and improve the display effect and usability of the image; the operation result display module presents the operation results to the user in a visual way, so that the user can intuitively see the effect of the operation, and provides good user feedback. This embodiment provides a convenient, safe and efficient digital pathology slice image interactive operating system, which provides powerful tools and support for doctors and researchers, and helps to improve the diagnosis and research efficiency of pathology slices.

[0144] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A one-screen multi-image display system based on digital pathology sections, characterized in that: Include: The image preprocessing module is responsible for acquiring a number of digital pathology slices, performing full-frame digital pathology slice imaging (WSI) format conversion, and performing image enhancement and noise removal preprocessing operations on the first digital pathology slice image after the format conversion to obtain a second digital pathology slice image after the preprocessing operation; The layout and mode setting module is responsible for acquiring the second digital pathology slice image, automatically adjusting the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtaining the third digital pathology slice image after layout; and establishing a dual linkage function mode between the third digital pathology slice images; The operation result display module is responsible for identifying the user's identity information. After verification, it establishes a connection between the user and the screen, identifies the user's gestures, mouse or touch screen method, and performs zooming, translation and rotation operations on the third digital pathology slice image. With the support of the dual linkage function mode, it realizes the display of the area of ​​interest for observation.

2. The one-screen multi-image display system based on digital pathological sections according to claim 1, characterized in that: Layout and mode setting module, including: The threshold setting submodule is responsible for obtaining the second digital pathology slice image to be laid out according to the screen size, and adopting two methods, namely, grid layout and tile layout, and setting the threshold for switching from grid layout to tile layout. The image arrangement submodule is responsible for arranging the second digital pathology slice image on the screen according to the grid layout and tile layout procedures; The image preview submodule is responsible for previewing the layout of the second digital pathology slice image on the entire screen after completing the layout of the second digital pathology slice image.

3. The one-screen multi-image display system based on digital pathological sections according to claim 2, characterized in that: In the threshold setting submodule, when the ratio of the number of columns to the number of rows of the second digital pathological slice image in the grid layout exceeds the threshold, the tile layout is started, otherwise the grid layout is performed.

4. The one-screen multi-image display system based on digital pathological sections as claimed in claim 2, characterized in that: In the image arrangement submodule, the position of each second digital pathology slice image on the screen is adjusted by dragging and adjusting the margin as needed.

5. The one-screen multi-image display system based on digital pathological sections according to claim 1, characterized in that: The layout and mode setting module also includes: The operation association submodule is responsible for adding an event listener to each digital pathology slice image to capture the user's operation actions on the digital pathology slice image, including zooming in, zooming out, and moving; When a user operates a digital pathology slide image, the operation action is published as an event using the event delivery mechanism, and other digital pathology slide images are subscribed as event listeners to receive and process the operation action, and the current operation action is delivered to other synchronously displayed digital pathology slide images; The operation execution submodule is responsible for applying the operation action to other digital pathology slice images accordingly after receiving the operation action; The image update submodule is responsible for updating the display of the digital pathology slice image by re-adjusting the display parameters of the slice whenever an operation action is applied to other digital pathology slice images.

6. The one-screen multi-image display system based on digital pathological sections according to claim 5, characterized in that: Operation-related submodules, including: The mechanism creation unit is responsible for creating an event listener for each digital pathology slide image to capture the user's operation actions on the image, matching a key value for each operation action, adding the response logic of the corresponding key value using the event transfer mechanism method, and moving the control logic of each operation action of each digital pathology slide image to the event transfer mechanism method to achieve adaptive operation; The event subscription unit is responsible for encapsulating the operation action into an event object when the user operates a digital pathology slide image, and publishing the event through the event bus or event manager; by registering the corresponding event listener on each digital pathology slide image, other digital pathology slide images that need to be displayed synchronously are subscribed as event listeners; The function calling unit is responsible for receiving and processing operation action events. After receiving the operation action events as event listeners, other digital pathology slice images call the event processing function to process the events and extract the operation action information in the events. The action of the current operation is passed to other digital pathology slide images displayed synchronously by calling the response logic.

7. The one-screen multi-image display system based on digital pathological sections according to claim 5, characterized in that: The event processing function of the event subscription unit refers to a function used to process a specific event, and the specific event refers to operation action information.

8. The one-screen multi-image display system based on digital pathological sections according to claim 6, characterized in that: Function call unit, including: Combined with the element preset sub-unit, it is responsible for establishing an event processing function set. Each element in the event processing function set corresponds to the user's operation action on the image, and the corresponding event processing function set is found through the operation action; The operation action recognition subunit is responsible for analyzing the current operation action, obtaining the current operation action parameters, obtaining the pre-classified event processing function set corresponding to the current operation action parameters according to the current operation action parameters, and directly finding the corresponding event processing function set through the current operation action; The corresponding function calling sub-unit is responsible for obtaining all event processing functions in the event processing function set that process the current operation action, and each event processing function corresponds to an event subscriber, executes the event processing functions in the event processing function set, and distributes the events to other digital pathology slice images displayed synchronously.

9. The one-screen multi-image display system based on digital pathological sections according to claim 1, characterized in that: Operation result display module, including: The identity authentication submodule is responsible for verifying the user's identity information based on the pre-entered identity information. After the verification is passed, the interactive operation relationship between the user and the screen is established; The operation recognition submodule is responsible for recognizing the user's gestures, mouse or touch screen mode, and obtaining the user's input information; judging the user's operation intention based on the user's gestures or input information, whether to perform zoom, translation or rotation operations; and selecting the area of ​​interest for operation based on the user's operation intention; The operation execution submodule is responsible for the user's selection of zoom operation, adjusting the size of the third digital pathology slice image according to the user's gesture or input information, and performing corresponding operations on other related digital pathology slice images; the user selects the translation operation, and moves the position of the third digital pathology slice image according to the user's gesture or input information; The user selects to perform a rotation operation, and the angle of the third digital pathology slice image is rotated according to the user's gesture or input information.

10. A method for displaying multiple images on one screen based on digital pathological sections, characterized in that: The following steps are involved: Acquire a plurality of digital pathology slices, perform full-frame digital pathology slice imaging (WSI) format conversion, and perform image enhancement and noise removal preprocessing operations on the first digital pathology slice image after the format conversion to obtain a second digital pathology slice image after the preprocessing operation; Acquire the second digital pathology slice image, automatically adjust the layout of the second digital pathology slice image according to the screen size and the number of digital pathology slices, and obtain the third digital pathology slice image after layout; establish a dual linkage function mode between the third digital pathology slice images; Identify the user's identity information, and after verification, establish a connection between the user and the screen, identify the user's gestures, mouse or touch screen method, zoom, translate and rotate the third digital pathology slice image, and with the support of the dual linkage function mode, realize the display of the area of ​​interest.

Images