Organ slice hierarchical tissue image processing method, device and equipment and storage medium

Abstract

The application discloses an organ slice hierarchical organization image processing method and device, equipment and a storage medium. The scheme provided by the application can intelligently process organ slice microscopic images, extract image features and segment regions, and then construct a region spatial relationship graph. The spatial relationship graph is matched with an organ hierarchical structure knowledge graph, and teaching knowledge data is fused for visualization. The organ organization hierarchical structure and spatial relationship can be intuitively and dynamically displayed, the shortcomings of static images in traditional teaching can be effectively made up, the students can be helped to establish the correlation between organization positions and physiological functions, the understanding of abstract knowledge and the learning efficiency can be improved, and the scheme is suitable for ordinary teaching scenes.

Description

Technical Field

[0001] This application relates to the fields of teaching informatization and image processing technology, and in particular to a method, apparatus, device and storage medium for processing organ slice layered tissue images. Background Technology

[0002] In junior high school biology teaching, the knowledge of the tissue composition of organs (such as the stomach and small intestine) is one of the key teaching points. Students need to understand the spatial hierarchical structure of different tissues (such as smooth muscle and epithelial tissue) within organs and establish the relationship between tissue location and its physiological function. However, in actual teaching, traditional teaching methods mainly rely on teachers using static diagrams or photomicrographs from textbooks for explanation. These two-dimensional images are difficult to intuitively and dynamically demonstrate the spatial relationships such as inclusion, adjacency, and hierarchy between tissues, leading students to often fall into the dilemma of "seeing the tissues but not the relationships." In addition, textbook diagrams are usually idealized and differ significantly from actual tissue slice images in color, shape, and boundary clarity, making it difficult for students to accurately match abstract diagrams with real images, resulting in a disconnect between knowledge points.

[0003] Currently, while existing image processing technologies can segment biological tissue slices, such as the SAM (Segment Anything Model)-based segmentation method, they can only segment different tissue regions in the image as isolated instances, and their effect on improving teaching quality is not significant. In addition, although there are some specialized tools in the field of biomedical image processing, such as medical image annotation software and organ 3D reconstruction software, these software are usually geared towards scientific research and clinical diagnosis, are expensive, complex to operate, require professional training, and have high requirements for the operating environment, making it difficult to promote their use in teaching environments such as middle school campuses and ordinary classrooms. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for processing organ slice hierarchical tissue images, which is used to realize the purpose of the invention in ordinary teaching scenarios, automatically identify the spatial hierarchical relationship of tissues in organ slices, integrate teaching knowledge, achieve dynamic visualization, and improve teaching effectiveness.

[0005] To achieve the aforementioned objectives, the first aspect of this application provides a method for processing organ slice layered tissue images, comprising:

[0006] Obtain microscopic images of organ sections;

[0007] The organ slice microscopic image is processed by a pre-trained slice image processing model to extract image features and segment image regions. Then, the type is identified based on the image features corresponding to each image region. The slice image processing model includes a shared encoder and multiple parallel decoders, which are used to simultaneously output the region object type, layer region type, region boundary information and layer direction information of each image region. The output results of each decoder are mutually verified to determine the final region object type.

[0008] Based on the relative positional relationships of each image region in the image, and combined with the region object type corresponding to each image region, a region spatial relationship diagram is generated;

[0009] Based on the regional spatial relationship diagram, it is matched with a preset organ hierarchical structure knowledge graph. The teaching knowledge data corresponding to each regional entity contained in the regional spatial relationship diagram is matched and extracted. The teaching knowledge data is then integrated and visualized with the regional spatial relationship diagram to obtain slice tissue hierarchical knowledge visualization data.

[0010] Preferably, the shared encoder is configured to extract image features from the organ slice microscopic image;

[0011] Each parallel decoder is configured to output the corresponding feature determination result based on the image features obtained from the shared encoder, according to the feature determination logic trained in the parallel decoder.

[0012] Preferably, the parallel decoder includes: a region boundary detection decoder, a layer type identification decoder, a tissue type identification decoder, and a direction field prediction decoder; the direction field prediction decoder is used to output a normalized direction vector representing the direction of each pixel from the inner tissue to the outer tissue.

[0013] Preferably, the region object type includes: layer region type, tissue type, or cell type.

[0014] Preferably, the organ hierarchical knowledge graph is constructed in the following ways:

[0015] Acquire pre-defined anatomical knowledge data;

[0016] Anatomical knowledge data is classified according to different organ types;

[0017] Based on the anatomical knowledge corresponding to each organ type after classification, a hierarchical knowledge graph of organ hierarchy is constructed according to the hierarchical relationship of organ-layer-tissue-cell.

[0018] Preferably, it further includes:

[0019] When an interactive operation command is received, the system response action corresponding to the interactive operation command is executed on the sliced ​​organizational hierarchy knowledge visualization data according to the preset correspondence between interactive operation types and system response actions.

[0020] Preferably, according to the preset correspondence between interactive operation types and system response actions, the system response actions corresponding to the interactive operation instructions executed on the sliced ​​organizational hierarchy knowledge visualization data include:

[0021] When the interactive operation command is an instruction to adjust the display depth, based on the received real-time display depth value, the displayable organizational level in the sliced ​​organizational hierarchy knowledge visualization data is determined according to the correspondence between the hierarchical slice image and the display depth value. The correspondence between the hierarchical slice image and the display depth value is a proportional correspondence between the outermost organizational image to the innermost organizational image and between the minimum display depth value and the maximum display depth value.

[0022] When the interactive operation instruction is a selection operation on a target node in the hierarchical tree, the region entity in the image corresponding to the target node is highlighted, and the textbook description information of the region entity is displayed;

[0023] When the interactive operation instruction is to select an entity in a target region of an image, the node corresponding to the entity in the target region is highlighted in the hierarchy tree, and the teaching material description information of the node is displayed.

[0024] A second aspect of this application provides an organ slice layered tissue image processing apparatus, comprising:

[0025] Image acquisition unit, used to acquire microscopic images of organ slices;

[0026] The model processing unit is used to extract image features and segment image regions from the organ slice microscopic image using a pre-trained slice image processing model, and then perform type recognition based on the image features corresponding to each image region. The slice image processing model includes a shared encoder and multiple parallel decoders, which are used to simultaneously output the region object type, layer region type, region boundary information and layer direction information of each image region, and to perform mutual verification based on the output results of each decoder to determine the final region object type.

[0027] The spatial relationship determination unit is used to generate a regional spatial relationship map based on the relative positional relationship of each image region in the image and the corresponding region object type of each image region.

[0028] The knowledge data processing unit is used to match the regional spatial relationship diagram with a preset organ hierarchical structure knowledge graph, match and extract the teaching knowledge data corresponding to each regional entity contained in the regional spatial relationship diagram, so as to fuse and visualize the teaching knowledge data with the regional spatial relationship diagram to obtain slice tissue hierarchical knowledge visualization data.

[0029] A third aspect of this application provides an intelligent microscope device, including: a memory and a processor;

[0030] The memory stores program code that corresponds to an organ slice layered tissue image processing method provided in the first aspect of this application.

[0031] The processor is used to read and execute the program code to implement the method for processing organ slice hierarchical tissue images.

[0032] The fourth aspect of this application provides a computer-readable storage medium storing program code, which is read and executed by a processor to implement an organ slice layered tissue image processing method as provided in the first aspect of this application.

[0033] As can be seen from the above technical solutions, this application has the following advantages:

[0034] The solution provided in this application intelligently processes organ slice microscopic images, extracts image features, segments regions, and then constructs a regional spatial relationship map. By matching this spatial relationship map with an organ hierarchical structure knowledge graph and integrating teaching knowledge data for visualization, the hierarchical structure and spatial relationships of organ tissues can be displayed intuitively and dynamically. This effectively compensates for the shortcomings of static images in traditional teaching, helps students establish the connection between tissue location and physiological function, and improves their understanding of abstract knowledge and learning efficiency. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating an embodiment of an organ slice layered tissue image processing method provided in this application.

[0037] Figure 2 This is a schematic diagram of the architecture of an embodiment of an organ slice hierarchical tissue image processing device provided in this application.

[0038] Figure 3 This is a schematic diagram of the control unit architecture of an embodiment of an intelligent microscope device provided in this application. Detailed Implementation

[0039] This application provides a method, apparatus, device, and storage medium for processing hierarchical tissue images of organ slices, which is used to realize the purpose of the invention in ordinary teaching scenarios, automatically identify the spatial hierarchical relationship of tissues in organ slices, integrate teaching knowledge, achieve dynamic visualization, and improve teaching effectiveness.

[0040] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] First, a detailed description of an embodiment of an organ slice layered tissue image processing method provided in this application is as follows:

[0042] Please see Figure 1 This application provides a method for processing organ slice layered tissue images, the main steps of which include:

[0043] Step 101: Obtain microscopic images of organ sections;

[0044] Step 102: Using a pre-trained slice image processing model, extract image features and segment image regions from organ slice microscopic images. Then, perform type recognition based on the image features corresponding to each image region. The slice image processing model includes a shared encoder and multiple parallel decoders, which are used to simultaneously output the region object type, layer region type, region boundary information, and layer direction information of each image region. The output results of each decoder are mutually verified to determine the final region object type.

[0045] Step 103: Based on the relative positional relationship of each image region in the image, and combined with the region object type corresponding to each image region, generate a region spatial relationship diagram;

[0046] Step 104: Match the regional spatial relationship diagram with the preset organ hierarchical structure knowledge graph, and extract the teaching knowledge data corresponding to each regional entity contained in the regional spatial relationship diagram. In order to integrate and visualize the teaching knowledge data with the regional spatial relationship diagram, the slice tissue hierarchy knowledge visualization data is obtained.

[0047] It should be noted that, for ease of understanding, some key terms in this embodiment are explained below:

[0048] Organ section microscopic images refer to digital images obtained by observing and photographing sections of biological organs and tissues using a microscope. These images typically contain microscopic structural information about different tissues and cells within the organ.

[0049] A slice image processing model refers to a deep learning model or other machine learning model trained on a large amount of biological slice image data. This model is configured to recognize and process specific patterns and structures in organ slice microscopic images, such as performing feature extraction and region segmentation.

[0050] Image feature extraction refers to the process of identifying and quantifying representative information from raw image data. This information can describe key attributes of the image content, such as texture, color, shape, or edges.

[0051] Image region segmentation refers to the process of dividing an image into multiple independent regions with specific semantic or visual features. Each region typically corresponds to a specific object or organizational structure in the image.

[0052] Region object type refers to the classification label assigned to each individual region after image region segmentation. This label is used to identify the type of biological tissue or hierarchical structure that the region represents.

[0053] A regional spatial relationship graph refers to a data model represented by a graph structure, where nodes represent various regional entities identified in an image, and edges represent the spatial relationships between these regional entities, such as containment, adjacency, and vertical relationships.

[0054] Organ hierarchical knowledge graphs refer to a structured form of knowledge representation that stores hierarchical information of a specific organ from macroscopic to microscopic levels in the form of a graph. This includes the inclusion relationships between organs, layers, tissues, and cells, as well as the anatomical and physiological knowledge between each level.

[0055] Teaching knowledge data refers to multimedia information such as text, images, and videos related to organ and tissue structure and suitable for teaching purposes. This data typically includes descriptions of tissue function, location, and pathological features.

[0056] Visualized tissue layer knowledge data refers to data that integrates organ slice image processing results, regional spatial relationships, organ hierarchical structure knowledge, and teaching knowledge data, and presents them in a graphical and interactive manner. This data aims to intuitively display the hierarchical structure of organs and tissues and related knowledge.

[0057] More specifically, the organ slice layer tissue image processing method provided in this application has the following specific steps:

[0058] The first step is to acquire microscopic images of organ sections. As one implementation method, these images can be acquired through various means. For example, one can manually operate a microscope and connect a digital camera to take pictures, then manually import the captured image files into a processing system. Alternatively, existing physical sections can be digitized using a scanner to generate high-resolution digital image files. These image files are typically stored in common image formats such as JPEG, PNG, or TIFF.

[0059] Next, upon receiving the microscopic image of the organ slice, the image is input into a pre-trained slice image processing model. This model first performs image feature extraction on the input image, for example, extracting deep semantic features through the encoder part of a convolutional neural network. Subsequently, based on the extracted image features, the model performs image region segmentation, identifying and separating different tissue or cellular regions within the image. Specifically, traditional image processing algorithms based on thresholding, edge detection, or region growing can be used, or pixel-level segmentation can be performed using the decoder part based on deep learning architectures such as U-Net or Mask R-CNN. After region segmentation, for each segmented image region, type identification is performed based on its internal image features. This type identification process can employ methods such as support vector machines, random forests, or neural network classifiers to determine the type of object represented by each region, such as identifying it as epithelial tissue, connective tissue, or muscle tissue.

[0060] Building upon this foundation, after determining the region object type for each image region, the system analyzes the relative positional relationships of these regions within the original image. For example, it can calculate the centroid coordinates, bounding box overlap, or distance information of each region to determine if spatial relationships such as adjacency, containment, or overlap exist between them. Specifically, if one region is completely inside another, it can be considered an containment relationship, such as the mucosa and epithelial tissue; if the boundaries of two regions are in contact, they are considered adjacency. Combining the determined region object type for each region, these spatial relationships are encoded into a graph structure. In this graph structure, each image region is represented as a node, and the spatial relationships between regions are represented as edges connecting the nodes. For example, one node can be labeled as "inner layer of gastric wall epithelium" and another node can be labeled as "middle layer of gastric wall smooth muscle". They are connected by an "adjacent" edge. In addition to the inclusion and adjacency relationships mentioned above, there is also a covering relationship, that is, one region completely covers the upper layer of another region, such as epithelial tissue and columnar cells. There is also a composition relationship, that is, one region or multiple regions together form another large region, such as epithelial tissue + lamina propria + muscularis mucosae = mucosa.

[0061] Ultimately, the generated regional spatial relationship graph is used to match a pre-defined organ hierarchy knowledge graph. This matching process aims to associate specific tissue regions identified in the image with the abstract anatomical hierarchy in the knowledge graph. For example, structured matching can be performed by comparing the type of region object with the name of the tissue node in the knowledge graph, and by comparing the spatial relationships between regions with the hierarchical relationships in the knowledge graph. Once a match is successful, the system extracts pedagogical knowledge data corresponding to each region entity in the regional spatial relationship graph from the organ hierarchy knowledge graph. This pedagogical knowledge data can include the physiological functions, pathological characteristics, and developmental processes of the tissue. Finally, the extracted pedagogical knowledge data is fused with the regional spatial relationship graph and presented using visualization techniques. This visualization process can include adding text annotations, color coding, and highlighting to the nodes or edges of the regional spatial graph to visually display the hierarchical structure of the organ and related pedagogical knowledge. Thus, the final output of the sliced ​​tissue hierarchy knowledge visualization data not only includes the number of organ levels but also presents a regional spatial relationship graph with rich knowledge annotations.

[0062] This application utilizes intelligent processing of organ slice microscopic images to extract image features and segment regions, thereby constructing a regional spatial relationship map. By matching this spatial relationship map with an organ hierarchical structure knowledge graph and integrating teaching knowledge data for visualization, the hierarchical structure and spatial relationships of organ tissues can be displayed intuitively and dynamically. This effectively compensates for the shortcomings of static images in traditional teaching, helps students establish the connection between tissue location and physiological function, and improves their understanding of abstract knowledge and learning efficiency.

[0063] Based on the above basic embodiments, this application further proposes an implementation of a specific network architecture for a slice image processing model, wherein the slice image processing model includes a shared encoder and multiple parallel decoders. The shared encoder is configured to extract image features from the organ slice microscopic image; the parallel decoders are configured to output corresponding feature determination results based on the image features obtained from the shared encoder, according to the feature determination logic trained in the parallel decoders.

[0064] Specifically, the slice image processing model employs an encoder-decoder architecture. The shared encoder is the core component of this architecture, its main function being to extract deep image features from the input organ slice microscopic images. This shared encoder typically consists of a series of convolutional layers, activation functions, and pooling layers (or stride convolutional layers). Through layer-by-layer processing, it can progressively extract semantic features from the original image, ranging from low-level (e.g., edges, texture) to high-level (e.g., tissue structure, cell morphology). These features are highly abstract and information-rich, effectively capturing key visual information in the image while filtering out noise and redundant data. For example, the shared encoder can employ mature convolutional neural network structures such as the shrinking path in ResNet, VGG, or U-Net++ architectures to leverage their powerful feature learning capabilities.

[0065] Building upon this, the slice image processing model further incorporates multiple parallel decoders. Each parallel decoder is configured to receive and utilize image features obtained from a shared encoder. These parallel decoders each carry pre-trained feature determination logic, meaning each decoder is specifically trained to output corresponding feature determination results for a particular task. For example, one parallel decoder might be trained to identify region boundaries in an image, another to identify different hierarchical types, and a third to identify specific tissue types. Each parallel decoder typically consists of upsampling layers (such as transposed convolutions or bilinear interpolation), convolutional layers, and activation functions, designed to "decode" the abstract features extracted by the encoder back into spatial resolution and semantic information relevant to a specific task, ultimately outputting determination results in the form of pixel-level classification, region-level labels, or probability maps. This modular design allows different image processing tasks to be performed in parallel or independently, improving the model's flexibility and processing efficiency.

[0066] By introducing an architecture of a shared encoder and multiple parallel decoders, this application effectively decouples feature extraction from organ slice microscopic images from subsequent tasks such as image region segmentation and type recognition. The shared encoder focuses on extracting high-level, semantically rich general features from the original image, while multiple parallel decoders can perform specialized feature determination and output results for different specific tasks, either in parallel or independently, based on these general features. This modular design allows each decoder to be optimized and trained according to its specific task requirements, avoiding mutual interference and performance bottlenecks that may occur when a single model handles multiple tasks. Therefore, this scheme significantly improves the accuracy, robustness, and scalability of the slice image processing model in the analysis of complex organ slice images, thus providing more accurate and reliable input data for subsequent regional spatial relationship graph generation and knowledge visualization processing.

[0067] Based on the above embodiments, this application further proposes that the parallel decoder includes: a region boundary detection module, a layer type identification module, an organization type identification module, and a hierarchical direction determination module.

[0068] The region boundary detection module is primarily responsible for identifying and accurately delineating the geometric boundaries of various image regions in organ slice microscopic images. Its implementation can include, but is not limited to, using deep learning-based semantic segmentation networks, such as U-Net and Mask R-CNN, to train on a large amount of image data with region boundary annotations. This allows the network to learn the edge features of different tissue regions, thereby accurately outputting pixel-level region boundary masks on new input images. Alternatively, traditional image processing algorithms, such as edge detection operators (Canny, Sobel, etc.) or morphological operations, can be combined to post-process the output of the deep learning model, further optimizing the smoothness and continuity of the boundaries.

[0069] The layer type recognition module aims to identify different layer types in organ slice microscopic images. For example, for a gastric wall slice, it may be necessary to identify the mucosa, submucosa, muscularis propria, and serosa. Its implementation typically involves image classification or semantic segmentation techniques. For instance, a convolutional neural network (CNN) classifier can be trained to classify each image region after region segmentation and output its corresponding layer type. Alternatively, based on semantic segmentation, different category labels can be assigned to different layer types, thereby directly identifying each layer region at the pixel level. Training this module requires a large number of image samples with layer type annotations.

[0070] The tissue type identification module is used to identify the specific tissue type of each region in a microscopic image of an organ section, such as epithelial tissue, connective tissue, muscle tissue, and nerve tissue. Similar to the layer type identification module, its implementation can be based on deep learning-based image classification or semantic segmentation models. By training on image data containing annotations of different tissue types, the model can learn and distinguish various microscopic tissue structure features, thereby accurately determining the tissue type of image regions or pixels.

[0071] The layer orientation determination module is responsible for determining the relative spatial orientation relationships between different layers or tissue regions in organ slice micrographs. For example, it determines whether a layer is located inside or outside another layer, or whether a tissue faces the luminal plane or the basal plane. This is crucial for constructing accurate regional spatial relationship maps. Its implementation can include training a specialized classifier that takes the features and relative position information of two adjacent regions as input and outputs their directional relationship (e.g., "inside," "outside," "above," "below," etc.). Alternatively, it can be aided by analyzing pre-defined anatomical landmarks or gradient information in the image. For example, in tubular organs, layer orientation can be inferred based on the distribution characteristics of intraluminal and extraluminal tissues.

[0072] For example, assuming this scheme uses U-Net++ as the basic architecture, ResNet34 as the encoder, and four independent decoders outputting tissue type segmentation maps, layer region segmentation maps, boundary detection maps, and orientation field prediction maps, respectively. Skip connections between the encoder and decoders preserve multi-scale features, enabling the network to simultaneously utilize high-level semantic information (determining "what kind of tissue") and low-level detailed information (locating "where the boundary is").

[0073] More specifically, taking a gastric wall slice as an example, the key computational logic of the slice image processing model provided in this embodiment from input to output can be seen in the following step example:

[0074] 1) Input layer:

[0075] The network receives a slice image of the stomach wall, with dimensions of 512×512×3 (RGB three channels).

[0076] 2) Encoder feature extraction:

[0077] The encoder consists of 5 residual blocks, extracting features layer by layer. The computation process is as follows:

[0078] Layers 1-2: Extract low-level features such as edges and textures, such as the arrangement direction of epithelial cells in the mucosal layer and the parallel texture of muscle fibers.

[0079] Layers 3-4: Extract mid-layer semantic features, such as "This is the epithelial tissue region" or "This is the muscle layer region".

[0080] Layer 5: Extract high-level semantic features (16×16×512), such as "the mucosa layer is above and the muscle layer is below in the entire field of vision", to form a global spatial understanding.

[0081] 3) Decoder output:

[0082] Taking the organization type segmentation task (decoder 1) as an example, the process of transforming features into classification results is as follows:

[0083] 3.1) The decoder receives features from each layer of the encoder (through skip connections) and gradually upsamples them to restore the resolution;

[0084] 3.2) The last layer uses a 1×1 convolution to map the feature map to 5 channels, corresponding to epithelial tissue, connective tissue, muscle tissue, neural tissue, and background, respectively;

[0085] 3.3) For each pixel position (x, y), calculate the probability of it belonging to each category using the Softmax function;

[0086] 3.4) Take the category with the highest probability as the final organization type of the pixel.

[0087] The output operations of the other three decoders are similar:

[0088] Decoder 2 (Layer Region Segmentation): Outputs several channels (mucosal layer, submucosal layer, muscle layer, serous layer, background), and determines the layer to which the pixel belongs after Softmax.

[0089] Decoder 3 (Boundary Detection): Outputs 1 channel. After processing by the Sigmoid function, values ​​> 0.5 are determined to be boundary pixels.

[0090] Decoder 4 (Direction Prediction): Outputs 2 channels. After processing by the Tanh function, it represents the normalized direction vector (dx, dy) from the inner layer to the outer layer.

[0091] Through the aforementioned technical solution, the parallel decoder is refined into a region boundary detection module, a layer type recognition module, a tissue type recognition module, and a hierarchical direction determination module. This enables the slice image processing model to extract and identify the key information needed to construct a regional spatial relationship map from organ slice microscopic images more precisely and comprehensively. Specifically, the region boundary detection module ensures the accuracy of image region segmentation, providing an accurate regional range for subsequent type recognition; the layer type recognition module and the tissue type recognition module accurately classify the biological attributes of image regions from macroscopic and microscopic levels, respectively, greatly enriching the information dimension of regional object types; and the hierarchical direction determination module compensates for the spatial topological relationships between levels that cannot be obtained by type recognition alone, ensuring that the regional spatial relationship map accurately reflects the true hierarchical structure of the organ. This modular professional division of labor significantly improves the accuracy and efficiency of image feature extraction and region segmentation, resulting in a more accurate and complete regional spatial relationship map. This lays a solid foundation for subsequent matching with organ hierarchical structure knowledge graphs and the integration and visualization of teaching knowledge data. The final output of slice tissue hierarchical knowledge visualization data can more accurately and intuitively display the hierarchical structure of organs and related knowledge.

[0092] Furthermore, since each decoder shares the features extracted by the same encoder, they can also complement each other's information: for example:

[0093] Tissue type information (decoder 1) helps with layer region segmentation (decoder 2): knowing "this is epithelial tissue" allows us to infer "it should belong to the mucosal layer";

[0094] Layer region information (decoder 2) helps verify tissue type (decoder 1): knowing "this is the mucosa" confirms "the interior should be epithelial tissue rather than muscle";

[0095] Boundary information (decoder 3) locates the boundaries between different levels / organizations;

[0096] Directional information (decoder 4) ensures the correct hierarchical order (from inner to outer layer).

[0097] The four decoders are trained end-to-end using a joint loss function: the tissue type segmentation decoder and the layer region segmentation decoder use the cross-entropy loss function, the boundary detection decoder uses the Dice loss function, and the orientation prediction decoder uses the L2 loss function. Multi-task collaborative optimization is achieved through a shared encoder, which enables the output results of each decoder to verify each other and jointly improve the recognition accuracy.

[0098] For example, when a pixel is identified as "epithelial tissue" by decoder 1 (probability 0.92) and "mucosal layer" by decoder 2 (probability 0.93), the two pixels are in the same spatial position and can verify each other, thus improving the overall recognition accuracy.

[0099] Based on the above basic embodiments, this application further proposes that the above-mentioned regional object types include organization types or layer regional types.

[0100] Specifically, region object type refers to the biological or anatomical classification of each independent region obtained after image region segmentation in a microscopic image of an organ slice. It is a semantic annotation of the content of a specific region in the image. Accurate region object type is the foundation for constructing a region spatial relationship map and is also key information for subsequent matching with the organ hierarchical structure knowledge graph. Among them, tissue type refers to classifying an image region into a specific histological type after identification, such as epithelial tissue, connective tissue, muscle tissue, and nervous tissue. This is usually achieved by training a model to identify tissue-specific morphological features, cell arrangement, cell nucleus size and shape, cytoplasmic staining characteristics, and other microstructural information. For example, in a gastric wall slice, different tissue types such as gastric mucosa, submucosa, muscularis propria, and serosa can be identified. Layer region type refers to classifying an image region into a specific anatomical hierarchical structure of an organ after identification, such as the inner, middle, and outer layers of the organ wall, or the cortex and medulla of a specific gland. This type of recognition focuses more on macroscopic hierarchical division, typically based on a comprehensive judgment of the relative position and shape of a region, as well as the types of tissues it contains. For example, when identifying the stomach wall, in addition to identifying specific tissue types, it can also identify hierarchical regions such as "mucosal layer" and "muscular layer." The tissue type recognition module and layer type recognition module in the aforementioned parallel decoder can directly output or collaboratively output the types of these region objects.

[0101] By explicitly defining the region object type as tissue type or layer region type, this application can provide more biologically and anatomically meaningful semantic annotations for each image region. Specifically, when the slice image processing model identifies a specific tissue type or layer region type, the subsequently generated regional spatial relationship map will contain more accurate and richer semantic information. This enables the regional spatial relationship map to achieve higher accuracy and more reliable matching results when matched with a pre-defined organ hierarchical structure knowledge graph, thereby effectively avoiding matching ambiguity or errors caused by region type generalization. Ultimately, the obtained slice tissue hierarchical knowledge visualization data can more accurately reflect the true hierarchical structure and tissue composition of organs, providing more valuable and guiding knowledge annotations for teaching and research, and improving the accuracy and practicality of slice tissue hierarchical knowledge visualization data.

[0102] In some embodiments, this application further proposes a method for constructing the organ hierarchical knowledge graph, which includes:

[0103] Acquire pre-defined anatomical knowledge data; classify the anatomical knowledge data according to different organ types; based on the anatomical knowledge corresponding to each organ type after classification, construct an organ hierarchical structure knowledge graph corresponding to each organ type according to the hierarchical relationship of organ-layer-tissue-cell.

[0104] Specifically, acquiring pre-defined anatomical knowledge data refers to collecting and organizing a collection of professional knowledge describing the structures of organs, tissues, cells, and their interrelationships within an organism. This data can come from various authoritative sources, such as medical textbooks, anatomical atlases, professional medical databases, clinical case reports, research literature, and experiential knowledge provided by medical experts. The acquisition process typically involves steps such as data collection, digitization, and structured processing, aiming to transform scattered, unstructured anatomical information into a format that can be processed by computer systems.

[0105] Subsequently, the anatomical knowledge data is categorized according to different organ types. The purpose of this step is to organize the vast amount of anatomical information by its corresponding organ; for example, all knowledge related to the heart is grouped into one category, and knowledge related to the lungs into another. This categorization can be automated through manual annotation, keyword matching, and natural language processing (NLP) techniques, or semi-automated by combining expert systems. Categorizing the data provides a clear and focused dataset for subsequent knowledge graph construction targeting specific organs, improving the efficiency and accuracy of the construction process.

[0106] Based on this, and using the anatomical knowledge corresponding to each organ type after classification, a hierarchical knowledge graph of the organ hierarchy is constructed according to the organ-layer-tissue-cell relationship. This means that the entities (nodes) in the knowledge graph will include the organ itself, its various layers (e.g., the serosa, muscle layer, submucosa, and mucosa of the stomach wall), the specific tissue types that constitute these layers (e.g., smooth muscle tissue, connective tissue, and epithelial tissue), and the typical cell types in these tissues (e.g., gastric gland cells, parietal cells, and chief cells). The relationships (edges) between entities represent hierarchical or spatial associations such as inclusion, composition, and location. The construction process can employ graph database technology, using entities as nodes and relationships as edges to store and manage this hierarchical anatomical knowledge. This refined hierarchical structure ensures that the knowledge graph can comprehensively and accurately reflect the microstructure of organs, providing a solid foundation for subsequent image region recognition and knowledge annotation.

[0107] Through the above technical solution, this application provides a systematic and standardized method for constructing an organ hierarchical structure knowledge graph. First, by acquiring and classifying anatomical knowledge data, a dedicated knowledge foundation is established for different organs, avoiding knowledge confusion and redundancy. Second, a knowledge graph is constructed based on a clear hierarchical relationship of "organ-layer-tissue-cell," resulting in a clear and logically rigorous knowledge structure that accurately describes the macroscopic to microscopic structural details of organs. This refined knowledge graph significantly improves the accuracy of matching and the completeness of knowledge extraction when matched with regional spatial relationship diagrams, ensuring the precise association between teaching knowledge data and image regions. Finally, the resulting slice tissue hierarchical knowledge visualization data not only includes the number of organ layers but also features highly accurate and detailed knowledge annotations, greatly enhancing the teaching value and practicality of the visualization data and providing reliable knowledge support for medical teaching and research.

[0108] In some embodiments, this application further proposes that when an interactive operation instruction is received, the system will execute the system response action corresponding to the interactive operation instruction on the sliced ​​organizational hierarchy knowledge visualization data according to the preset correspondence between interactive operation types and system response actions.

[0109] Specifically, "receiving an interactive operation instruction" means that the system detects and acquires input from the user, which expresses the user's intention to operate or query the currently displayed slice-based hierarchical knowledge visualization data. This instruction can be implemented through various human-computer interaction methods, such as clicking a specific area on the visualization interface with a mouse, entering keyboard shortcuts, performing gestures on a touchscreen, issuing voice commands, or indicating points of interest through eye-tracking devices. The system's internal input processing module is configured to capture and recognize these different forms of user interaction signals.

[0110] The "preset correspondence between interactive operation types and system response actions" refers to a set of mapping rules or configuration tables established within the system. This table associates specific user interaction behaviors (i.e., interactive operation types) with the corresponding functions or display changes the system should perform (i.e., system response actions). For example, this correspondence could be defined as follows: when a user clicks on a tissue area, the system response is to display detailed anatomical information of that area; when a user drags a slider, the system response is to adjust the display depth of the visualization data; or when a user performs a zoom gesture, the system response is to zoom in or out of the current view. This preset correspondence ensures that the system's response to user commands is predictable and meaningful.

[0111] "The system response action corresponding to the interactive operation command on the sliced ​​tissue hierarchy knowledge visualization data" refers to the system dynamically processing and updating the generated sliced ​​tissue hierarchy knowledge visualization data after receiving the interactive operation command, based on the aforementioned preset correspondence. For example, if the interactive operation command is "display a specific tissue type," the system response action might be to highlight all areas belonging to that tissue type in the visualization data and pop up relevant knowledge annotations. If the command is "adjust display depth," the system will re-render the visualization data according to the new depth value, displaying only the tissue hierarchy within that depth range. Executing these response actions involves real-time modification, re-rendering, or display of additional information in the visualization data to meet the user's personalized exploration needs.

[0112] More specifically, the interactive function designs provided in this embodiment include, but are not limited to, the types shown in Table 1 below:

[0113] Table 1 Examples of Interactive Function Design

[0114]

[0115] By introducing interactive operation commands and their corresponding system response actions, this application can transform static visualized tissue layer knowledge data into a dynamic and interactive presentation. When a user issues a specific interactive operation command, the system can quickly execute corresponding response actions on the visualized data according to preset correspondences, such as highlighting specific areas, adjusting display depth, or displaying detailed knowledge annotations. This allows users to flexibly explore and focus on organ layer structures and tissue regions of interest according to their own learning or analysis needs, thereby greatly improving the efficiency of users' understanding of complex anatomical knowledge and their learning experience. This interactive mechanism effectively solves the limitations of traditional static visualized data in personalized exploration and deep learning, making the teaching and learning of organ slice layer knowledge more intuitive and efficient.

[0116] Based on this, this application further proposes a processing method. When the interactive operation instruction is an instruction to adjust the display depth, the system response action corresponding to the interactive operation instruction for the sliced ​​tissue hierarchy knowledge visualization data includes: based on the received real-time display depth value, determining the displayable tissue level in the sliced ​​tissue hierarchy knowledge visualization data according to the correspondence between the hierarchical slice image and the display depth value, wherein the correspondence between the hierarchical slice image and the display depth value is: a proportional correspondence between the outermost tissue image and the innermost tissue image and between the minimum display depth value and the maximum display depth value.

[0117] Specifically, the command to adjust the display depth refers to an instruction issued by the user through a human-computer interaction interface (e.g., a slider, scroll wheel, input box, or preset button) to adjust the layer depth of the currently displayed organ slice tissue knowledge visualization data. This command carries or implies the user's desired display depth information. The received real-time display depth value is a specific numerical value parsed by the system from the command to adjust the display depth, reflecting the user's current desired display depth. For example, if the user operates through a slider, the value can be a percentage corresponding to the current position of the slider or a normalized value; if input through an input box, it is the specific layer value entered by the user.

[0118] The correspondence between the hierarchical slice images and the display depth values ​​is a pre-established mapping rule that defines the association between different display depth values ​​and the displayable tissue levels in the organ slice hierarchy. This correspondence can be a lookup table stored in the system configuration table or a function mapping implemented through a specific algorithm. Determining the displayable tissue levels in the slice tissue hierarchy knowledge visualization data means that the system calculates or finds the specific levels, such as organs, layers, tissues, or cells, that should be presented in the visualization data based on the received real-time display depth values ​​and referring to the correspondence. For example, when the display depth value is small, only the organ level may be displayed; as the display depth value gradually increases, more detailed hierarchical information such as layers, tissues, and even cells within the organ will be gradually displayed.

[0119] Specifically, the proportional correspondence between the outermost and innermost tissue images and between the minimum and maximum display depth values ​​defines the specific form of this correspondence. This means that there is a linear or near-linear proportional relationship between changes in display depth values ​​and changes in tissue levels. For example, when the display depth value changes from 0% to 100%, the corresponding displayable tissue level smoothly transitions from the most macroscopic overall organ view to the most microscopic cellular detail view, and this transition is uniform and predictable. This proportional relationship ensures that users receive smooth, intuitive, and expected visual feedback when adjusting the display depth, thereby enhancing the ease of use and accuracy of the interaction.

[0120] For example, when the interactive operation instruction is a selection operation on a target node in the hierarchical tree, the region entity in the image corresponding to the target node is highlighted, and the textbook description information of the region entity is displayed;

[0121] Specifically, the selection operation of a target node in the hierarchical tree refers to an instruction issued by the user through a human-computer interaction interface (e.g., mouse click, touch click, keyboard arrow key selection, or search box positioning) to specify a knowledge node representing an organ, layer, tissue, or cell within the hierarchical tree component. This instruction carries a unique identifier for the selected node (such as node ID, node path, or node name). For example, if the user clicks the "Gastric Mucosal Layer" node in the hierarchical tree panel on the left, the system receives a selection operation for that node.

[0122] The highlighted region in the image corresponding to the target node refers to the image region (such as a polygon mask, outline, or pixel set) associated with the target node, which the system searches for in a pre-established knowledge graph or mapping table based on the target node's identifier, and then performs visual enhancement rendering on that region in the currently displayed visualization image. Highlighting methods may include, but are not limited to: changing the region boundary color (e.g., changing it to red or yellow), increasing the region fill transparency, adding a blinking animation, displaying an arrow indicator, or enlarging the region. For example, when the user selects the "Gastric Mucosal Layer" node, the system outlines the area covered by the mucosal layer in the gastric slice image with a bright green outline and makes it slightly glow.

[0123] The textbook description information displayed for the entity in the region refers to the standardized teaching text content corresponding to the target node extracted from the knowledge base by the system and displayed in a designated information panel (such as the details box on the right, a floating prompt box, or the bottom status bar). This textbook description information typically includes: the definition, location, morphological characteristics, physiological functions, common lesions, and related clinical significance of the tissue structure. The source of the description information can be authoritative anatomy textbooks, histological atlases, or teaching documents reviewed by experts. For example, when the "Gastric Mucosal Layer" node is selected, the information panel displays: "The gastric mucosal layer is the innermost layer of the stomach wall, composed of epithelium, lamina propria, and muscularis mucosae. Its surface has numerous gastric pits containing gastric glands that secrete gastric juice and participate in digestion." In particular, the textbook description information can support multimedia expansion, such as simultaneously displaying related microscopic images, 3D models, or links to teaching videos.

[0124] When the interactive operation instruction is to select an entity in a target region of an image, the node corresponding to the entity in the target region is highlighted in the hierarchy tree, and the teaching material description information of the node is displayed.

[0125] Specifically, the selection operation of a target region entity in an image refers to a user's instruction, issued through a human-computer interaction interface (e.g., mouse click, touchscreen tap, box selection, or gesture selection), to directly select a visible anatomical structure (such as organ outline, tissue layer, cell cluster, etc.) within an image area displaying visualized data of organ slice hierarchical tissue knowledge. This instruction carries the screen coordinates or pixel coordinates of the selected location, and the system determines the corresponding image region entity through coordinate mapping. For example, if a user directly clicks on the area containing the "submucosal layer" in a stomach slice image, the system will parse the click coordinates and identify that the location belongs to the "submucosal layer" entity.

[0126] Highlighting the node corresponding to the target region entity in the hierarchical tree refers to the system searching for a matching knowledge node in the hierarchical tree component based on the identified region entity identifier, and then visually highlighting that node. Highlighting methods may include, but are not limited to: changing the node's background color (e.g., light blue), bolding the node text, displaying an indicator icon (e.g., an arrow or asterisk) before the node, automatically expanding the parent node path to make the node visible, or scrolling the hierarchical tree view to center the node in the field of view. For example, when a user clicks on the "submucosal layer" region in an image, the node "Gastric Wall → Submucosal Layer" in the left-hand hierarchical tree will become bold orange text, and the "Gastric Wall" parent node will automatically expand, ensuring the user can clearly see the node's position in the hierarchical structure.

[0127] The description information displayed for this node follows the same mechanism as in Example 1: the system extracts the standardized teaching text content corresponding to the node from the knowledge base and displays it in the information panel. Since users select entities directly from images, the system also needs to provide an additional "contextual confirmation" mechanism: if the user clicks on a location that simultaneously belongs to multiple overlapping or adjacent areas (e.g., the click point is at the boundary between the mucosa and submucosa), the system can provide a choice based on a preset priority (e.g., more refined hierarchical priority, or more prominent area priority) or by displaying a candidate list. For example, when a user clicks on a blurred area in a stomach wall image, the system can display a small window: "You may want to select: A. Gastric mucosa, B. Gastric submucosa," and after user confirmation, the corresponding node will be highlighted and its description displayed.

[0128] The above is a description of the method logic of an embodiment of the organ slice layered tissue image processing method provided in this application. The following is a more specific example to illustrate the above technical solution in more detail:

[0129] In a biology teaching scenario, User A needs to explain the tissue composition and hierarchical structure of the stomach to students. User A first obtains a microscopic image of a stomach slice using a digital microscope. This image clearly shows multiple tissue regions of the stomach wall.

[0130] Subsequently, the microscopic image of the gastric slice is input into a pre-trained slice image processing model. This model comprises a shared encoder and multiple parallel decoders. The shared encoder extracts image features from the input microscopic image of the gastric slice, obtaining deep features of the image. These image features are then passed to the parallel decoders. The parallel decoders include a region boundary detection module, a layer type identification module, a tissue type identification module, and a hierarchical orientation determination module. The region boundary detection module first identifies the boundaries of each independent tissue region in the image. The layer type identification module and the tissue type identification module, based on the image features obtained from the shared encoder, identify the type of these regions according to pre-trained feature determination logic; for example, identifying one region as "mucosal layer" and another as "smooth muscle tissue." The hierarchical orientation determination module determines the relative positional relationship of these regions within the gastric wall; for example, "mucosal layer" is located inside "submucosal layer," and "muscular layer" is located outside "submucosal layer." Through this series of processes, the model determines the region object type of each region in the image; for example, "mucosal layer" belongs to the layer region type, and "smooth muscle tissue" belongs to the tissue type. Unlike existing methods that can only segment different tissue regions in an image as isolated instances, this scheme not only segments regions but also identifies their types and determines their hierarchical orientation, laying the foundation for subsequent knowledge association.

[0131] After determining the region object type for each image region, the system generates a region spatial relationship map based on the relative positional relationships of these regions in the image (e.g., containment, adjacency, inner-outer relationship) and the determined region object types. This map represents the spatial connections between different levels of tissue in the stomach wall in a structured manner. For example, the "stomach wall" contains the "serosa," the "serosa" is adjacent to the "muscular layer," and the "muscular layer" contains "smooth muscle tissue," etc.

[0132] Next, the system matches the generated regional spatial relationship map with a pre-built organ hierarchical structure knowledge graph. This knowledge graph is pre-constructed by first acquiring pre-defined anatomical knowledge data, and then classifying the anatomical knowledge data according to different organ types (e.g., "stomach"). Based on the classified gastric anatomy knowledge, an organ hierarchical structure knowledge graph corresponding to the stomach is constructed according to the organ-layer-tissue-cell hierarchy. By matching the image-generated regional spatial relationship map with this knowledge graph, the system can identify the corresponding nodes of each regional entity (such as "mucosal layer" and "smooth muscle tissue") contained in the regional spatial relationship map in the knowledge graph, and extract teaching knowledge data related to these entities, such as "the main function of the mucosal layer is to secrete gastric juice and protect the stomach wall" and "smooth muscle tissue is responsible for gastric peristalsis."

[0133] Finally, the system integrates and visualizes the extracted teaching knowledge data with the regional spatial relationship diagram to obtain visualized data of tissue layer knowledge. This visualized data includes the number of organ layers (e.g., the stomach wall mainly contains four layers) and a regional spatial relationship diagram with knowledge annotations. Compared with traditional teaching methods that rely on static diagrams or photomicrographs, this visualized data can intuitively show the spatial relationships between tissues, such as inclusion, adjacency, and hierarchy, and directly relate them to their physiological functions. This solves the problem that students "see only tissues, not relationships" and have difficulty accurately matching abstract diagrams with real images.

[0134] During the teaching process, when user A receives a command to adjust the display depth, the system executes the corresponding system response action on the visualized tissue hierarchy data according to the preset correspondence between interactive operation types and system response actions. For example, based on the received real-time display depth value, the system determines the displayable tissue layers in the visualized tissue hierarchy data according to the proportional correspondence between the layered slice images and the display depth values ​​(from the outermost tissue image to the innermost tissue image and from the minimum display depth value to the maximum display depth value). When user A sets the display depth value to a small value, the system only displays the outermost layer of the stomach wall (such as the serosa) and its related information; as the display depth value increases, the system gradually displays the inner structures (such as the muscularis propria, submucosa, and mucosa) and their corresponding tissue types and functional information. This interactive method allows user A and students to dynamically explore the hierarchical structure of the stomach wall, gain a deeper understanding of the spatial relationships and functions of different tissue layers, and improve teaching effectiveness.

[0135] The following is a detailed description of embodiments of an organ slice layered tissue image processing device, an intelligent microscope device, and a storage medium provided in this application.

[0136] Please see Figure 2 This application proposes an organ slice layered tissue image processing device, comprising:

[0137] Image acquisition unit 201 is used to acquire microscopic images of organ slices;

[0138] The model processing unit 202 is used to extract image features and segment image regions of the organ slice microscopic image through a pre-trained slice image processing model, and then perform type recognition based on the image features corresponding to each image region to determine the region object type of each image region.

[0139] The spatial relationship determination unit 203 is used to generate a regional spatial relationship diagram based on the relative positional relationship of each image region in the image and the corresponding region object type of each image region.

[0140] The knowledge data processing unit 204 is used to match the spatial relationship diagram of the region with a preset organ hierarchical structure knowledge graph, match and extract the teaching knowledge data corresponding to each regional entity contained in the spatial relationship diagram of the region, so as to integrate and visualize the teaching knowledge data with the spatial relationship diagram of the region to obtain slice tissue hierarchical knowledge visualization data.

[0141] This device dynamically links image segmentation results with anatomical hierarchical knowledge through structured matching of regional spatial relationship maps with organ hierarchical structure knowledge graphs, thus intuitively displaying the hierarchical structure and spatial relationships of organ tissues. Specifically, the model processing unit outputs regional object types combined with spatial location information to generate a regional spatial relationship map reflecting tissue inclusion, adjacency, and other relationships; the knowledge data processing unit then matches this map with a preset knowledge graph, extracts and integrates teaching knowledge data, and finally generates knowledge-annotated visual data. This design not only compensates for the shortcomings of traditional static images, enabling students to dynamically understand tissue hierarchy, but also achieves lightweight operation, allowing for use in ordinary teaching environments without professional training, significantly improving teaching efficiency and learning experience.

[0142] Further, please refer to Figure 3 This application proposes an intelligent microscope device, including a memory 33 and a processor 31, which are connected via a communication bus 34. The memory 33 stores program code corresponding to the above-described organ slice layered tissue image processing method; the processor 31 is used to read and execute the program code to implement the above-described organ slice layered tissue image processing method.

[0143] Specifically, an intelligent microscope is a device that integrates optical imaging, digital image processing, and computational analysis functions. It not only provides traditional microscopic observation functions but also enables real-time or near-real-time processing of acquired images through a built-in computing unit, thereby achieving more advanced functions, such as the organ slice layered tissue image processing method described in this application. Such devices typically have a user-friendly interface, allowing complex image analysis tasks to be completed directly on-site, improving work efficiency and accuracy.

[0144] A memory is a hardware component in an intelligent microscope device used to store data and instructions. In this application, the memory is mainly used to store the program code that implements the above-mentioned organ slice hierarchical tissue image processing method, as well as intermediate data and final results generated during the execution of the method, such as organ slice microscopic images, image features, regional spatial relationship diagrams, organ hierarchical structure knowledge graphs, and slice tissue hierarchical knowledge visualization data. The memory can be a combination of various forms such as random access memory (RAM), read-only memory (ROM), flash memory, or hard disk drive (HDD) to meet the storage needs of different types of data.

[0145] The processor is the core computing unit of the intelligent microscope device, responsible for interpreting and executing the program code stored in the memory, thereby driving the entire device to complete various functions. In this application, the processor reads and executes the program code corresponding to the above-mentioned organ slice hierarchical tissue image processing method, coordinating the work of various hardware modules to perform complex computational tasks such as image feature extraction, region segmentation, type recognition, spatial relationship graph generation, knowledge graph matching, data fusion, and visualization processing. The processor can be a central processing unit (CPU), graphics processing unit (GPU), or application-specific integrated circuit (ASIC), etc., and its performance directly affects the speed and efficiency of image processing.

[0146] The program code is a set of instructions for implementing the above-mentioned organ slice hierarchical tissue image processing method, stored in memory in a computer-executable form. This code defines the specific steps and logic of the method, including how to acquire organ slice microscopic images, how to extract image features and segment regions using a slice image processing model, how to perform type recognition based on image region features, how to generate a regional spatial relationship map, how to match it with an organ hierarchical structure knowledge graph, and how to perform data fusion and visualization processing. When the processor executes this program code, the abstract method steps are transformed into concrete computational operations, thereby realizing the image processing function on the intelligent microscope device.

[0147] By integrating the aforementioned organ section layer tissue image processing method into an intelligent microscope device, with the program code stored in the memory and executed by the processor, this application provides an efficient, convenient, and integrated solution. This integration enables complex image processing and knowledge visualization functions to be completed directly on-site during microscopic observation, without the need for additional general-purpose computer equipment, greatly simplifying the operation process and improving the efficiency of diagnosis and teaching. The intelligent microscope device can perform in-depth analysis of microscopic images in real-time or near real-time and present the analysis results to the user in the form of intuitive visualization data of section tissue layer knowledge, thereby enhancing the user's understanding of organ section tissue structure and related knowledge, and improving the quality of teaching and research. In addition, the optimized configuration of the dedicated equipment also ensures the stability and reliability of method execution, providing users with a smoother and more accurate user experience.

[0148] Furthermore, this application also proposes a computer-readable storage medium storing program code, which is used to be read and executed by a processor to implement the above-mentioned organ slice layered tissue image processing method.

[0149] Specifically, a computer-readable storage medium refers to any physical device or medium capable of storing digital data and computer program instructions. It provides a computer system with persistent or non-persistent data storage capabilities. This medium can take many forms, such as non-volatile storage media like hard disk drives, solid-state drives, optical discs, USB flash drives, and memory cards; or volatile storage media like random access memory (RAM), although program code is typically loaded from non-volatile media into RAM for execution. Its core function is to act as a carrier of program code, ensuring that the program code can be stably stored and accessed and read by the processor when needed.

[0150] The program code refers to a collection of computer instructions written to achieve a specific function. These instructions can be source code written in a high-level language, or machine-executable binary code or intermediate code formed after compilation, assembly, or interpretation. The program code can be stored on a storage medium in the form of files, such as executable files, dynamic link libraries, and script files. The program code is the specific implementation of the organ slice hierarchical tissue image processing method; it contains all the logic, algorithms, and steps of the method from acquiring organ slice microscopic images to finally generating visualized data of slice tissue hierarchy knowledge.

[0151] The processor reading and execution refers to the processor (such as a central processing unit or graphics processing unit) acting as the core computing unit of a computer system, responsible for interpreting and executing instructions in the program code. The processor loads the program code from a computer-readable storage medium into memory through a memory management unit, and then executes these instructions one by one or in parallel through components such as an instruction decoder, arithmetic logic unit, and control unit. The execution process includes data access, logical judgment, and arithmetic operations, thereby driving the operation of the entire organ slice hierarchical tissue image processing method. The processor reads and executes the program code, transforming the static instruction set into a dynamic calculation process, enabling the aforementioned organ slice hierarchical tissue image processing method to actually run and complete its predetermined functions.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the terminals, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0154] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0155] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0159] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing organ slice layered tissue images, characterized in that, include: Obtain microscopic images of organ sections; The organ slice microscopic image is processed by a pre-trained slice image processing model to extract image features and segment image regions. Then, the type is identified based on the image features corresponding to each image region. The slice image processing model includes a shared encoder and multiple parallel decoders, which are used to simultaneously output the region object type, layer region type, region boundary information and layer direction information of each image region. The output results of each parallel decoder are mutually verified to determine the final region object type. Based on the relative positional relationships of each image region in the image, and combined with the region object type corresponding to each image region, a region spatial relationship diagram is generated; Based on the regional spatial relationship diagram, it is matched with a preset organ hierarchical structure knowledge graph. The teaching knowledge data corresponding to each regional entity contained in the regional spatial relationship diagram is matched and extracted. The teaching knowledge data is then integrated and visualized with the regional spatial relationship diagram to obtain slice tissue hierarchical knowledge visualization data.

2. The method for processing organ slice layered tissue images according to claim 1, characterized in that, The shared encoder is configured to extract image features from the organ slice microscopic images; Each parallel decoder is configured to output the corresponding feature determination result based on the image features obtained from the shared encoder, according to the feature determination logic trained in the parallel decoder.

3. The method for processing organ slice layered tissue images according to claim 2, characterized in that, The parallel decoder includes: a region boundary detection decoder, a layer type recognition decoder, a tissue type recognition decoder, and a direction field prediction decoder; the direction field prediction decoder is used to output a normalized direction vector representing the direction of each pixel from the inner tissue to the outer tissue.

4. The method for processing organ slice layered tissue images according to claim 1, characterized in that, Region object types include: layer region type, tissue type, or cell type.

5. The method for processing organ slice layered tissue images according to claim 1, characterized in that, The methods for constructing the organ hierarchical knowledge graph include: Acquire pre-defined anatomical knowledge data; Anatomical knowledge data is classified according to different organ types; Based on the anatomical knowledge corresponding to each organ type after classification, a hierarchical knowledge graph of organ hierarchy is constructed according to the hierarchical relationship of organ-layer-tissue-cell.

6. The method for processing organ slice layered tissue images according to claim 1, characterized in that, Also includes: When an interactive operation command is received, the system response action corresponding to the interactive operation command is executed on the sliced ​​organizational hierarchy knowledge visualization data according to the preset correspondence between interactive operation types and system response actions.

7. The method for processing organ slice layered tissue images according to claim 6, characterized in that, According to the preset correspondence between interactive operation types and system response actions, the system response actions corresponding to the interactive operation instructions executed on the sliced ​​organizational hierarchy knowledge visualization data include: When the interactive operation command is an instruction to adjust the display depth, based on the received real-time display depth value, the displayable organizational level in the sliced ​​organizational hierarchy knowledge visualization data is determined according to the correspondence between the hierarchical slice image and the display depth value. The correspondence between the hierarchical slice image and the display depth value is a proportional correspondence between the outermost organizational image to the innermost organizational image and between the minimum display depth value and the maximum display depth value. When the interactive operation instruction is a selection operation on a target node in the hierarchical tree, the region entity in the image corresponding to the target node is highlighted, and the textbook description information of the region entity is displayed; When the interactive operation instruction is to select an entity in a target region of an image, the node corresponding to the entity in the target region is highlighted in the hierarchy tree, and the teaching material description information of the node is displayed.

8. An organ slice layered tissue image processing device, characterized in that, include: Image acquisition unit, used to acquire microscopic images of organ slices; The model processing unit is used to extract image features and segment image regions from the organ slice microscopic image using a pre-trained slice image processing model, and then perform type recognition based on the image features corresponding to each image region. The slice image processing model includes a shared encoder and multiple parallel decoders, which are used to simultaneously output the region object type, layer region type, region boundary information and layer direction information of each image region, and to perform mutual verification based on the output results of each decoder to determine the final region object type. The spatial relationship determination unit is used to generate a regional spatial relationship map based on the relative positional relationship of each image region in the image and the corresponding region object type of each image region. The knowledge data processing unit is used to match the regional spatial relationship diagram with a preset organ hierarchical structure knowledge graph, match and extract the teaching knowledge data corresponding to each regional entity contained in the regional spatial relationship diagram, so as to fuse and visualize the teaching knowledge data with the regional spatial relationship diagram to obtain slice tissue hierarchical knowledge visualization data.

9. An intelligent microscope device, characterized in that, include: Memory and processor; The memory stores program code, which corresponds to an organ slice layered tissue image processing method as described in any one of claims 1 to 7; The processor is used to read and execute the program code to implement the method for processing organ slice hierarchical tissue images.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that is read and executed by a processor to implement an organ slice layered tissue image processing method as described in any one of claims 1 to 7.

Images