Gastrointestinal surgery auxiliary system and method

Abstract

The present application relates to the field of intelligent assistance, and specifically discloses a gastrointestinal surgical assistance system and method, which obtains the gastrointestinal CT scan image of the target patient object, uses the image recognition and processing algorithm based on computer vision and AI to perform image preprocessing on the gastrointestinal CT scan image, and then extracts the main semantic features and boundary semantic features from the preprocessed image, and intelligently generates the image semantic segmentation result containing the annotated tumor boundary based on the fine-grained enhanced fusion representation between the two features. In this way, it is possible to capture subtle changes that are difficult for the human eye to discern, thereby more accurately identifying and annotating tumor boundary information, and providing doctors with intuitive visual assistance. In addition, the automated processing flow significantly reduces the time for doctors to perform manual analysis, and improves the speed and efficiency of diagnosis.

Description

Technical Field

[0001] The present application relates to the field of intelligent assistance, and more specifically, to a gastrointestinal surgical assistance system and method. Background Art

[0002] With the development of medical imaging technology, especially the advancement of computed tomography (CT) technology, doctors are able to obtain high-quality images of in vivo structures, which is of great significance for the diagnosis and treatment of diseases. In digestive system diseases, especially early detection and precise positioning of gastrointestinal tumors, are crucial to improving patients' survival rate and quality of life.

[0003] However, traditional CT image analysis mainly relies on the experience and subjective judgment of radiologists, which is time-consuming and laborious. Especially when faced with a large amount of image data, the workload of doctors increases dramatically, which not only increases their burden, but also may affect the accuracy and consistency of diagnosis. In addition, the human eye has limited sensitivity to subtle changes, especially when faced with low contrast or tiny lesions, some critical but subtle information may be ignored, thereby increasing the risk of misdiagnosis and missed diagnosis.

[0004] Therefore, a gastrointestinal surgical auxiliary solution is desired to assist doctors in more effective tumor detection, thereby improving diagnostic accuracy and work efficiency. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed.

[0006] According to one aspect of the present application, a gastrointestinal surgical assistance system is provided, comprising:

[0007] A gastrointestinal CT scan image acquisition module, used to acquire a gastrointestinal CT scan image of a target patient;

[0008] A CT scan image preprocessing module, used for performing image noise reduction and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image;

[0009] A CT scan image feature extraction module, used for performing semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain gastrointestinal CT scan trunk semantic coding features and gastrointestinal CT scan boundary feature semantic coding features;

[0010] A CT scan image multi-scale feature fusion module is used to perform trunk-boundary fine-grained enhancement interaction on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain gastrointestinal CT scan multi-scale semantic enhancement fusion coding features; wherein the CT scan image multi-scale feature fusion module includes: a CT feature dissociation unit, used to perform feature dissociation on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain a set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local features; a CT feature compensation aggregation unit, used to perform fine-grained semantic feature compensation aggregation on the set of gastrointestinal CT scan trunk semantic local features and the set of gastrointestinal CT scan boundary semantic local features to obtain the gastrointestinal CT scan multi-scale semantic enhancement fusion coding features;

[0011] The semantic segmentation module is used to obtain an image semantic segmentation result based on the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan, and the image semantic segmentation result includes a marked tumor boundary.

[0012] Furthermore, the CT scan image feature extraction module is used to: input the gastrointestinal CT scan enhanced image into a dual-branch boundary information loss compensation network of a trunk network and a boundary feature extraction branch to obtain a gastrointestinal CT scan trunk semantic coding feature map as the gastrointestinal CT scan trunk semantic coding feature and a gastrointestinal CT scan boundary feature semantic coding feature map as the gastrointestinal CT scan boundary feature semantic coding feature.

[0013] Furthermore, the CT feature dissociation unit is used to perform feature dissociation on the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map along the channel dimension of the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map to obtain a set of gastrointestinal CT scan trunk semantic local feature matrices as the set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local feature matrices as the set of gastrointestinal CT scan boundary semantic local features.

[0014] Furthermore, the CT feature compensation aggregation unit comprises:

[0015] A trunk-boundary fine-grained extraction subunit is used to input each group of gastrointestinal CT scan trunk semantic local feature matrices and gastrointestinal CT scan boundary semantic local feature matrices corresponding to channel dimensions in the set of gastrointestinal CT scan trunk semantic local feature matrices and the set of gastrointestinal CT scan boundary semantic local feature matrices into a shared semantic information extraction module based on a twin network structure to obtain a set of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices;

[0016] A trunk-boundary semantic compensation feature calculation subunit, used for performing semantic feature compensation on the set of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices;

[0017] The trunk-boundary fine-grained feature semantic enhancement subunit is used to semantically enhance the set of trunk-boundary fine-grained local shared semantic feature matrices of the gastrointestinal CT scan based on the set of trunk-boundary semantic compensation text semantic encoding feature matrices of the gastrointestinal CT scan to obtain a multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan as the multi-scale semantic enhancement fusion encoding feature of the gastrointestinal CT scan.

[0018] Furthermore, the backbone-boundary semantic compensation feature calculation subunit is used to:

[0019] Inputting each gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix into a semantic compensation decoding module based on a large language model to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text descriptions;

[0020] Each gastrointestinal CT scan trunk-boundary semantic compensation text description in the set of the gastrointestinal CT scan trunk-boundary semantic compensation text description is respectively input into a semantic encoder based on a text convolutional neural network model to obtain a set of semantic encoding feature matrices of the gastrointestinal CT scan trunk-boundary semantic compensation text.

[0021] Furthermore, the backbone-boundary fine-grained feature semantic enhancement subunit is used to:

[0022] Inputting each corresponding group of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices and gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices and the set of the gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices into a fine-grained semantic interaction compensation module to obtain a set of gastrointestinal CT scan trunk-boundary fine-grained local interactive semantic reinforcement feature matrices;

[0023] The set of the gastrointestinal CT scan trunk-boundary fine-grained local interaction semantic enhancement feature matrices is aggregated along the channel dimension to obtain the gastrointestinal CT scan multi-scale semantic enhancement fusion encoding feature map.

[0024] Furthermore, the semantic segmentation module is used to perform image semantic segmentation on the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain the image semantic segmentation result, and the image semantic segmentation result includes a marked tumor boundary.

[0025] Furthermore, the semantic segmentation module is used to: use an image semantic segmenter based on a Softmax function to process the multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan to obtain the image semantic segmentation result, and the image semantic segmentation result includes a marked tumor boundary.

[0026] According to another aspect of the present application, a gastrointestinal surgical assistance method is provided, comprising:

[0027] Acquire a gastrointestinal CT scan image of a target patient;

[0028] Performing image noise reduction and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image;

[0029] Performing semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain gastrointestinal CT scan trunk semantic coding features and gastrointestinal CT scan boundary feature semantic coding features;

[0030] The main-boundary fine-grained enhancement interaction is performed on the main semantic coding features of the gastrointestinal CT scan and the semantic coding features of the boundary features of the gastrointestinal CT scan to obtain the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan, including: feature dissociation is performed on the main semantic coding features of the gastrointestinal CT scan and the semantic coding features of the boundary features of the gastrointestinal CT scan to obtain a set of main semantic local features of the gastrointestinal CT scan and a set of boundary semantic local features of the gastrointestinal CT scan; fine-grained semantic feature compensation aggregation is performed on the set of main semantic local features of the gastrointestinal CT scan and the set of boundary semantic local features of the gastrointestinal CT scan to obtain the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan;

[0031] Based on the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan, an image semantic segmentation result is obtained, and the image semantic segmentation result includes a marked tumor boundary.

[0032] Compared with the prior art, the present application provides a gastrointestinal surgical assistance system and method, which obtains the gastrointestinal CT scan image of the target patient, uses the image recognition and processing algorithm based on computer vision and AI to preprocess the gastrointestinal CT scan image, and then extracts the main semantic features and boundary semantic features from the preprocessed image, and intelligently generates the image semantic segmentation result containing the annotated tumor boundary based on the fine-grained enhanced fusion representation between the two features. In this way, it is possible to capture subtle changes that are difficult for the human eye to discern, especially in low-contrast and small lesions, reducing misdiagnosis and missed diagnosis caused by the limitations of human eye observation, thereby more accurately identifying and annotating tumor boundary information, and providing doctors with intuitive visual assistance. In addition, the automated processing flow significantly reduces the time for doctors to perform manual analysis and improves the speed and efficiency of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0034] Figure 1 4 is a block diagram of a gastrointestinal surgery assistance system according to an embodiment of the present application.

[0035] Figure 2 Schematic diagram of data flow of a gastrointestinal surgical assistance system according to an embodiment of the present application.

[0036] Figure 3 It is a block diagram of a multi-scale feature fusion module of a CT scan image in a gastrointestinal surgery auxiliary system according to an embodiment of the present application.

[0037] Figure 4 The figure is a flow chart of a gastrointestinal surgery assisting method according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0039] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0040] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0041] It is worth noting that in this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0042] With the development of medical imaging technology, especially the advancement of computed tomography (CT) technology, doctors are able to obtain high-quality images of in vivo structures, which is of great significance for the diagnosis and treatment of diseases. In digestive system diseases, especially early detection and precise positioning of gastrointestinal tumors, are crucial to improving patients' survival rate and quality of life.

[0043] However, traditional CT image analysis mainly relies on the experience and subjective judgment of radiologists, which is time-consuming and laborious. Especially when faced with a large amount of image data, the workload of doctors increases dramatically, which not only increases their burden, but also may affect the accuracy and consistency of diagnosis. In addition, the human eye has limited sensitivity to subtle changes, especially when faced with low contrast or tiny lesions, some critical but subtle information may be ignored, thereby increasing the risk of misdiagnosis and missed diagnosis.

[0044] Therefore, in response to the above problems, the technical concept of the present application is to obtain a gastrointestinal CT scan image of the target patient, use an image recognition and processing algorithm based on computer vision and AI to preprocess the gastrointestinal CT scan image, and then extract the main semantic features and boundary semantic features from the preprocessed image, and intelligently generate an image semantic segmentation result containing the annotated tumor boundary based on the fine-grained enhanced fusion representation between the two features. In this way, it is possible to capture subtle changes that are difficult for the human eye to discern, especially in low-contrast and micro-lesions, reducing misdiagnosis and missed diagnosis caused by the limitations of human eye observation, thereby more accurately identifying and annotating tumor boundary information, and providing doctors with intuitive visual assistance. In addition, the automated processing flow significantly reduces the time for doctors to perform manual analysis and improves the speed and efficiency of diagnosis.

[0045] Figure 1 4 is a block diagram of a gastrointestinal surgery assistance system according to an embodiment of the present application. Figure 2 FIG. 2 is a data flow diagram of a gastrointestinal surgery auxiliary system according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, the gastrointestinal surgical auxiliary system 100 includes: a gastrointestinal CT scan image acquisition module 110, which is used to acquire a gastrointestinal CT scan image of a target patient object; a CT scan image preprocessing module 120, which is used to perform image denoising and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image; a CT scan image feature extraction module 130, which is used to perform semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain a gastrointestinal CT scan trunk semantic coding feature and a gastrointestinal CT scan boundary feature semantic coding feature; a CT scan image multi-scale feature fusion module 140, which is used to perform trunk-boundary fine-grained enhancement interaction on the gastrointestinal CT scan trunk semantic coding feature and the gastrointestinal CT scan boundary feature semantic coding feature to obtain a gastrointestinal CT scan multi-scale semantic enhancement fusion coding feature; a semantic segmentation module 150, which is used to obtain an image semantic segmentation result based on the gastrointestinal CT scan multi-scale semantic enhancement fusion coding feature, and the image semantic segmentation result includes a labeled tumor boundary.

[0046] In the embodiment of the present application, the gastrointestinal CT scan image acquisition module 110 is used to acquire the gastrointestinal CT scan image of the target patient object. It should be understood that the gastrointestinal CT scan image refers to the image obtained by imaging the gastrointestinal tract of the patient through the computed tomography (CT) technology. The gastrointestinal CT scan image can provide a high-resolution image of the gastrointestinal tract and its surrounding tissues, helping doctors to accurately diagnose gastrointestinal diseases, including but not limited to inflammation, ulcers, tumors, etc. Based on this, in order to more accurately understand and analyze the internal gastrointestinal conditions of the patient object, so as to accurately determine the boundary distribution of the tumor, so as to better assist the doctor in diagnosis, in the technical solution of the present application, the gastrointestinal CT scan image of the target patient object is acquired. In particular, CT scanning is a non-invasive medical imaging technology that scans the human body at multiple angles through X-rays, and uses a computer to process these data to generate detailed cross-sectional images of the internal structure of the human body.

[0047] Before a CT scan, you need to make sure that the patient is properly prepared. This may include fasting for a period of time to ensure that there is no food residue in the gastrointestinal tract, which may interfere with the quality of the image. In addition, the doctor needs to explain the entire process to the patient and make sure that they understand and agree to undergo the examination. In some cases, in order to improve the contrast, the patient may need to be given a contrast agent orally or intravenously.

[0048] The patient is then placed in the CT scanner, usually on a movable bed. The bed slowly passes through a circular scanning device, while the X-ray source and detector rotate around the patient. During this process, X-rays pass through the patient's body from different angles, and the detector on the opposite side receives the attenuated X-ray intensity information. These data are processed by a computer to generate a series of cross-sectional images, which are called CT scan images. For the gastrointestinal area, multi-slice spiral CT technology is usually used, which can quickly obtain a large number of high-quality tomographic images in a short time, thereby providing more comprehensive and detailed anatomical structure information.

[0049] When the CT scan is completed, the raw data will be converted into a digital image format, such as the DICOM standard format. These images not only contain rich visual information, but also come with a series of metadata, such as patient information, scanning parameters, etc. These image data will then be stored in the hospital's information system, such as the PACS system, for radiologists and other medical professionals to access and analyze.

[0050] Once the image data is ready, the next step is how to effectively import these images into the system. Typically, the system will have a specially designed interface that allows the user to select a specific patient case and automatically load the corresponding CT scan images. This interface may be part of a graphical user interface or the result of integration with other medical information systems. Once the case is selected, the system reads the relevant image files and converts them into an internal representation suitable for further processing.

[0051] It is worth noting that although the above description outlines the general process of obtaining gastrointestinal CT scan images, the specific implementation details may vary according to different medical institutions and their equipment configurations. For example, some advanced CT equipment may have automated patient positioning functions, reducing the need for human intervention; while some hospitals may use more advanced image post-processing software that can preview and adjust image quality in real time. In any case, the ultimate goal is to ensure that the clearest and most accurate images are obtained to facilitate subsequent diagnosis and treatment decisions. In addition, with the development of cloud computing and big data technologies, such systems may also support remote access and cross-institutional collaboration in the future, allowing experts to view and analyze these important medical imaging materials anywhere, thereby further improving the quality and accessibility of medical services. In short, obtaining high-quality gastrointestinal CT scan images is a prerequisite for the operation of the entire gastrointestinal surgical auxiliary system, which lays a solid foundation for accurate tumor detection and boundary annotation.

[0052] In the embodiment of the present application, the CT scan image preprocessing module 120 is used to perform image noise reduction and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image. Accordingly, considering that there is often noise in the gastrointestinal CT scan image, such as random noise, streak artifacts, etc., these noises may be caused by factors such as the limitation of the equipment itself, improper setting of scanning parameters or patient movement, and the noise will reduce the clarity and visual effect of the image, affecting the doctor's observation and judgment of the lesion area. In addition, the brightness difference between different tissues will affect the contrast of the image, further affecting the identification of the lesion area. Therefore, in order to significantly improve the image quality and provide a better basis for subsequent feature extraction and image segmentation, in the technical solution of the present application, the gastrointestinal CT scan image is subjected to image noise reduction and contrast enhancement to obtain a gastrointestinal CT scan enhanced image, so that these noises can be effectively reduced, making the image clearer and cleaner while making the boundaries of different tissues more obvious to distinguish normal tissue from abnormal tissue (such as tumor).

[0053] After obtaining the original gastrointestinal CT scan images, there are often varying degrees of noise in the images due to various factors such as equipment limitations, improper scanning parameter settings, or patient body movement. These noises may appear in the form of random noise or streak artifacts, which will reduce the clarity and visual effects of the image, thereby affecting the doctor's observation and judgment of the lesion area. Therefore, the first step is to denoise the image. There are various choices of denoising techniques, ranging from traditional filtering methods (such as median filtering, Gaussian filtering) to more advanced deep learning-based methods. For CT images, a common method is to use adaptive filters, which can dynamically adjust their parameters according to the local image content, thereby removing noise while retaining important details such as edges as much as possible. In addition, the use of convolutional neural networks (CNNs) for denoising has become increasingly popular because CNNs can learn different types of noise patterns in images and eliminate them in a targeted manner while maintaining the structural integrity of the image.

[0054] Next is contrast enhancement, which is to further improve the visual expression of the image. Contrast refers to the degree of difference between the brightest and darkest parts of an image, which directly affects the distinguishability of different tissues in the image. In the field of medical imaging, effective contrast enhancement is particularly important, especially when facing low-contrast lesions. Common contrast enhancement techniques include histogram equalization, adaptive histogram equalization, and transform domain-based methods (such as wavelet transform). Among them, histogram equalization is a simple and effective method that expands the dynamic range of the image by redistributing pixel values, making the overall brightness distribution of the image more uniform, thereby improving the contrast. However, this method may cause the image to be oversaturated, that is, some areas are too bright or too dark. In contrast, adaptive histogram equalization performs equalization operations in a local range, which can better preserve the details of the image. In addition, transform domain-based methods, especially wavelet transform combined with multi-resolution analysis, can independently adjust the contrast of the image at different scales, which helps to highlight subtle structural features.

[0055] In practical applications, noise reduction and contrast enhancement are often used in combination, and the two complement each other. For example, while using a convolutional neural network for noise reduction, the contrast of the model output can be optimized through a specific loss function design, or the contrast enhancement step can be performed separately after noise reduction. This combined method can not only effectively remove noise, but also significantly improve the contrast of the image, making the boundaries between different tissues more obvious, thereby providing a high-quality foundation for subsequent feature extraction.

[0056] It is worth noting that although the above methods provide multiple ways to improve image quality, the specific implementation needs to be considered in specific application scenarios. For example, for certain special types of tumors, it may be necessary to pay special attention to certain subtle changes in details. At this time, it is necessary to choose a technical solution that can effectively reduce noise and finely enhance contrast. In addition, with the development of technology, more and more research is devoted to developing more intelligent preprocessing algorithms that can automatically identify the type and degree of noise in the image and adjust the noise reduction intensity accordingly; at the same time, they can also adaptively adjust the contrast enhancement effect according to the image content to achieve the best visual presentation.

[0057] Therefore, by performing image denoising and contrast enhancement on gastrointestinal CT scan images, the image quality can be significantly improved, laying a solid foundation for subsequent steps such as semantic encoding, boundary feature extraction, and multi-scale feature fusion, thereby ensuring that the final image semantic segmentation results are more accurate and reliable.

[0058] In the embodiment of the present application, the CT scan image feature extraction module 130 is used to perform semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features. Specifically, in the embodiment of the present application, the CT scan image feature extraction module 130 is used to: input the gastrointestinal CT scan enhanced image into the dual-branch boundary information loss compensation network of the trunk network and the boundary feature extraction branch to obtain the gastrointestinal CT scan trunk semantic coding feature map as the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding feature map as the gastrointestinal CT scan boundary feature semantic coding features. It should be understood that considering that the gastrointestinal CT scan enhanced image contains the main macroscopic structural information, such as the position and morphology of the organs, but also implies the boundary detail information between different tissues or organs. In the traditional single-channel network processing process, as the network depth increases, the underlying detail information (especially the boundary information) is easily ignored or lost, thereby reducing the effect and accuracy of feature extraction, resulting in the final segmentation result being not accurate enough. Based on this, in the technical solution of the present application, the gastrointestinal CT scan enhanced image is input into a dual-branch boundary information loss compensation network of a backbone network and a boundary feature extraction branch to obtain a gastrointestinal CT scan backbone semantic coding feature map and a gastrointestinal CT scan boundary feature semantic coding feature map. It can be understood that the backbone network is responsible for extracting the backbone semantic features of the gastrointestinal CT scan enhanced image, which can reflect the overall structure and content of the image, including large-scale information such as organs and tissues. The boundary feature extraction branch is specifically used to extract boundary information in the gastrointestinal CT scan enhanced image, that is, the edges between different tissues or structures. These boundary information is crucial for accurately locating the lesion area (such as the tumor boundary). Therefore, the dual-branch network can retain and enhance boundary information at different levels through the collaborative work of the backbone network and the boundary feature extraction branch, thereby effectively compensating for information loss and improving the accuracy of segmentation.

[0059] Specifically, the gastrointestinal CT scan enhanced image is input into the dual-branch boundary information loss compensation network of the backbone network and the boundary feature extraction branch to obtain the gastrointestinal CT scan backbone semantic coding feature map as the gastrointestinal CT scan backbone semantic coding feature and the gastrointestinal CT scan boundary feature semantic coding feature map. The specific processing process is as follows:

[0060] First, at the input, the preprocessed enhanced gastrointestinal CT scan images are fed into the network. This initial stage may include several convolutional layers and pooling operations to perform preliminary feature extraction and reduce the spatial resolution of the image. These primary feature maps are then split into two paths, one entering the backbone network and the other entering the boundary feature extraction branch.

[0061] The backbone network is usually based on popular convolutional neural network (CNN) architectures, such as ResNet, VGG, or DenseNet, which are stacked with multiple convolutional layers and pooling layers. These networks are good at extracting high-level abstract features in images, which can reflect the overall structure and content of the image. In each layer, convolution kernels of different sizes are used to detect feature patterns of multiple scales. As the depth of the network increases, these features gradually become more abstract, and eventually one or more backbone semantic encoding feature maps are formed. These feature maps contain large-scale information of the gastrointestinal tract and its surrounding tissues, such as the location and morphology of organs.

[0062] At the same time, the boundary feature extraction branch focuses on capturing edge information in the image. This branch can adopt a design concept similar to U-Net, which consists of a downsampling path and an upsampling path. The downsampling path is similar to the first few layers of the backbone network, using convolution and pooling operations to reduce spatial dimensions and extract coarse features. However, unlike the backbone network, the boundary feature extraction branch pays special attention to information that helps to depict boundaries during the downsampling process. To achieve this, this branch may use special convolution kernel configurations, such as dilated convolutions, to expand the receptive field without losing resolution. In addition, an attention mechanism can be introduced to highlight important boundary areas.

[0063] Once the downsampling process is completed, the boundary feature extraction branch will start upsampling to gradually restore the spatial size of the image. In this process, the jump connection technology is usually combined to combine the low-level features from the downsampling path with the high-level features to better preserve the boundary information. Upsampling can be achieved through methods such as transposed convolution or nearest neighbor interpolation. In this way, the boundary feature extraction branch finally generates boundary semantic encoding feature maps, which clearly show the edges between different tissues or structures.

[0064] Therefore, the use of the entire dual-branch network structure can not only fully capture the macroscopic structural information in the image, but also finely depict the subtle boundary details, so that even in the face of complex medical imaging data, it can effectively improve the accuracy and reliability of tumor detection.

[0065] In an embodiment of the present application, the CT scan image multi-scale feature fusion module 140 is used to perform trunk-boundary fine-grained enhancement interaction on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain gastrointestinal CT scan multi-scale semantic enhancement fusion coding features. Furthermore, it is considered that the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features respectively contain feature information of different scales. Specifically, the trunk semantic coding features focus on large-scale global structural information, while the boundary feature semantic coding features focus on small-scale local detail information. These boundary information and global structural information are very important for the segmentation of the lesion area. Therefore, in the technical solution of the present application, the trunk semantic coding features of the gastrointestinal CT scan and the semantic coding features of the gastrointestinal CT scan boundary features are subjected to trunk-boundary fine-grained enhancement interaction to obtain multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan. In this way, information can be transmitted and supplemented between features of different scales. For example, the fine-grained information in the boundary features can enhance the local details in the trunk features, thereby providing a richer and more comprehensive representation of the gastrointestinal CT image, and further more accurately identifying and segmenting the lesion area, reducing the occurrence of mis-segmentation and missed segmentation, and improving the accuracy and reliability of segmentation.

[0066] Figure 3 FIG. 1 is a block diagram of a multi-scale feature fusion module of a CT scan image in a gastrointestinal surgery auxiliary system according to an embodiment of the present application. Specifically, Figure 3 As shown, the CT scan image multi-scale feature fusion module 140 includes: a CT feature dissociation unit 141, which is used to perform feature dissociation on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain a set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local features; a CT feature compensation aggregation unit 142, which is used to perform fine-grained semantic feature compensation aggregation on the set of gastrointestinal CT scan trunk semantic local features and the set of gastrointestinal CT scan boundary semantic local features to obtain the gastrointestinal CT scan multi-scale semantic enhancement fusion coding features.

[0067] Specifically, the CT feature dissociation unit 141 is used to: perform feature dissociation on the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map along the channel dimension of the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map to obtain a set of gastrointestinal CT scan trunk semantic local feature matrices as the set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local feature matrices as the set of gastrointestinal CT scan boundary semantic local features.

[0068] The processing process of the CT feature separation unit 141 can be expressed as follows:

[0069] ;

[0070] ;

[0071] in, and are respectively the gastrointestinal CT scan trunk semantic coding feature map and the gastrointestinal CT scan boundary feature semantic coding feature map, To perform feature dissociation on the feature map, , , and They are the first, second, and third in the set of gastrointestinal CT scan backbone semantic local feature matrices. and A gastrointestinal CT scan backbone semantic local feature matrix, , , and They are the first, second, and third in the set of semantic local feature matrices of gastrointestinal CT scan boundary. and A semantic local feature matrix of the gastrointestinal CT scan boundary. In other words, each feature channel contains a specific type of feature information (such as texture, shape, color, etc.), and feature dissociation can capture the local refined feature information in the image more finely.

[0072] Specifically, the CT feature compensation aggregation unit 142 includes: a trunk-boundary fine-grained extraction subunit, which is used to input the set of the gastrointestinal CT scan trunk semantic local feature matrix and the set of the gastrointestinal CT scan boundary semantic local feature matrix of each group of corresponding channel dimensions into the shared semantic information extraction module based on the twin network structure to obtain a set of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices; a trunk-boundary semantic compensation feature calculation subunit, which is used to extract the gastrointestinal CT scan trunk semantic local feature matrix and the gastrointestinal CT scan boundary semantic local feature matrix of each group of corresponding channel dimensions; The set of intestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices is subjected to semantic feature compensation to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text semantic coding feature matrices; the trunk-boundary fine-grained feature semantic reinforcement subunit is used to semantically reinforce the set of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices based on the set of gastrointestinal CT scan trunk-boundary semantic compensation text semantic coding feature matrices to obtain a gastrointestinal CT scan multi-scale semantic reinforcement fusion coding feature map as the gastrointestinal CT scan multi-scale semantic reinforcement fusion coding feature.

[0073] The processing process of the above-mentioned backbone-edge fine-grained extraction subunit can be expressed as follows:

[0074] ;

[0075] in, It is the first in the set of gastrointestinal CT scan backbone semantic local feature matrices. A gastrointestinal CT scan backbone semantic local feature matrix, It is the first in the set of semantic local feature matrices of gastrointestinal CT scan boundary. A gastrointestinal CT scan boundary semantic local feature matrix, , and They are point-by-point addition, point-by-point multiplication, and point-by-point subtraction. For cascade processing, is point convolutional coding, yes function, yes and The gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix between. That is, through the twin network structure, the trunk semantic local feature matrix and the boundary semantic local feature matrix can be processed simultaneously, and the shared semantic information between them can be extracted and mined. This shared information reflects the common characteristics of the trunk features and boundary features in different channels, which helps to understand the image content more comprehensively. It is worth mentioning that the twin network architecture enables the model to grasp the correlation between different features and promote cross-domain or cross-modal knowledge transfer and learning.

[0076] More specifically, in an embodiment of the present application, the trunk-boundary semantic compensation feature calculation subunit is used to: input each gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix into a semantic compensation decoding module based on a large language model to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text descriptions; input each gastrointestinal CT scan trunk-boundary semantic compensation text description in the set of the gastrointestinal CT scan trunk-boundary semantic compensation text description into a semantic encoder based on a text convolutional neural network model to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices. It should be understood that if only shared semantic features are focused on during the feature interaction process, other important features, such as complementary features, may be lost. Complementary features contain some unique information that may not be obvious in a single trunk or boundary feature, but can provide additional context and details during the fusion process, which helps to improve the completeness and accuracy of the final feature representation. Therefore, each of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices is input into the semantic compensation decoding module based on the large language model to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text descriptions. In particular, the semantic compensation decoding module based on the large language model is similar to an accurate detector, which can convert complex fine-grained local shared semantic features into natural language descriptions, and use the prior knowledge within the model to supplement and improve these features at the semantic level, and generate a detailed text report. These text descriptions can include key information such as the location, size, morphology, and boundary clarity of the lesion, thereby improving the semantic depth and breadth of the feature representation. Subsequently, each gastrointestinal CT scan trunk-boundary semantic compensation text description in the set of the gastrointestinal CT scan trunk-boundary semantic compensation text description is respectively input into a semantic encoder based on a text convolutional neural network model to obtain a set of semantic encoding feature matrices of the gastrointestinal CT scan trunk-boundary semantic compensation text. In this way, each compensated text description can be semantically embedded and encoded to achieve feature alignment, so as to facilitate subsequent semantic compensation fusion processing.

[0077] The processing process of the above-mentioned backbone-boundary semantic compensation feature calculation subunit can be expressed as follows:

[0078] ;

[0079] ;

[0080] in, yes and The gastrointestinal CT scan backbone-boundary fine-grained local shared semantic feature matrix between For semantic compensation decoding operation, for Corresponding gastrointestinal CT scan backbone-boundary semantically compensated text description, is the text convolutional encoding, for Corresponding gastrointestinal CT scan backbone-boundary semantic compensation text semantic encoding feature matrix.

[0081] More specifically, in an embodiment of the present application, the trunk-boundary fine-grained feature semantic reinforcement subunit is used to: input each corresponding group of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices and gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices and the set of the gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices into a fine-grained semantic interaction compensation module to obtain a set of gastrointestinal CT scan trunk-boundary fine-grained local interaction semantic reinforcement feature matrices; aggregate the set of the gastrointestinal CT scan trunk-boundary fine-grained local interaction semantic reinforcement feature matrices along the channel dimension to obtain the gastrointestinal CT scan multi-scale semantic reinforcement fusion encoding feature map.

[0082] The processing process of the above-mentioned backbone-boundary fine-grained feature semantic enhancement sub-unit can be expressed as follows:

[0083] ;

[0084] ;

[0085] in, yes and The gastrointestinal CT scan backbone-boundary fine-grained local shared semantic feature matrix between for The corresponding gastrointestinal CT scan backbone-boundary semantic compensation text semantic encoding feature matrix, for The transposed matrix of is matrix multiplication, is the The scale is the width of the matrix multiplied by the height of the matrix, yes function, and are weighting coefficients, , , and They are the first, second, and third in the set of gastrointestinal CT scan trunk-boundary fine-grained local interaction semantic enhancement feature matrices. and A gastrointestinal CT scan backbone-boundary fine-grained local interaction semantic enhancement feature matrix, is the number of feature matrices in the set of the gastrointestinal CT scan trunk-boundary fine-grained local interactive semantic enhancement feature matrices, To aggregate a collection of feature matrices along the channel dimension, It is the multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan.

[0086] In this way, through fine-grained semantic interaction compensation, image features (gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix) and text features (gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrix) can be deeply fused in fine-grained manner, so as to make full use of the complementary information in the image and text to generate a more comprehensive and accurate feature representation. Finally, the set of the gastrointestinal CT scan trunk-boundary fine-grained local interactive semantic reinforcement feature matrix is ​​aggregated along the channel dimension to obtain the gastrointestinal CT scan multi-scale semantic reinforcement fusion encoding feature map, so as to integrate different local information into a unified feature map, which helps to capture the global and local features in the image and improve the comprehensiveness and richness of the feature representation.

[0087] In an embodiment of the present application, the semantic segmentation module 150 is used to obtain an image semantic segmentation result based on the multi-scale semantic enhancement fusion coding feature of the gastrointestinal CT scan, and the image semantic segmentation result includes a marked tumor boundary. Specifically, in an embodiment of the present application, the semantic segmentation module 150 is used to: perform image semantic segmentation on the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain the image semantic segmentation result, and the image semantic segmentation result includes a marked tumor boundary.

[0088] More specifically, in the embodiment of the present application, the semantic segmentation module 150 is used to: use the image semantic segmenter based on the Softmax function to process the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain the image semantic segmentation result, and the image semantic segmentation result includes the marked tumor boundary. That is, the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan obtained by fine-grained enhancement of the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map are used for image segmentation processing, so as to intelligently generate an image semantic segmentation result including the marked tumor boundary.

[0089] In particular, in a specific embodiment of the present application, the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan can be processed by an image semantic segmenter based on a Softmax function, and the specific process is as follows:

[0090] First, the multi-scale semantically enhanced fusion encoding feature map of the gastrointestinal CT scan is taken as input.

[0091] Next, the convolutional layer of the convolutional neural network (CNN) is used to extract features from the multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan. This step can help the model capture the local patterns and structural information in the feature map, and increase the nonlinear ability of the model through activation functions (such as ReLU), so that the model can better learn complex features.

[0092] Then, upsampling technology or transposed convolution operation is used to restore the feature map to a size close to the original image to facilitate pixel-level classification. In order to retain more detailed information, jump connections can be combined to combine early low-level features with later high-level features to enhance the model's ability to capture details.

[0093] After the feature map is restored to its original size, a convolutional layer is applied with the number of output channels equal to the number of categories to be identified to generate the classification score for each pixel. The Softmax conversion mechanism is used to convert the classification score of each pixel into a probability value, indicating the probability that the pixel belongs to each category.

[0094] Next, the difference between the predicted probability distribution and the actual label is calculated, and this difference is used to guide the model training process, so that the model can gradually reduce errors and improve classification accuracy.

[0095] Finally, the probability map output by the model is thresholded to determine the final segmentation result of each pixel. For example, if the threshold is 0.5, then pixels with a probability greater than 0.5 are marked as tumors, otherwise they are marked as background. At the same time, in order to remove noise and small areas in the segmentation results, morphological operations such as corrosion and expansion can be used to ensure that the segmentation boundaries are smoother and more natural, and obtain image semantic segmentation results containing annotated tumor boundaries. Through this series of steps, the image semantic segmenter based on the Softmax function can effectively process the multi-scale semantic enhancement fusion encoding feature map of gastrointestinal CT scans, generate high-quality segmentation results, and provide strong support for clinical applications.

[0096] In one example, considering that the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map respectively represent the trunk image semantic features and boundary image semantic features of the gastrointestinal CT scan enhanced image, when performing feature enhancement interaction based on cross-domain fine-grained semantic compensation, due to the cross-domain fine-grained semantic compensation differences caused by the image semantic feature differences, the gastrointestinal CT scan multi-scale semantic enhancement fusion encoding feature map obtained by feature enhancement interaction will have semantic aggregation distribution richness, resulting in repetition of unstructured probability mapping, affecting the accuracy of the image semantic segmentation results obtained by performing image semantic segmentation.

[0097] Based on this, when performing image semantic segmentation on the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain an image semantic segmentation result, the present application optimizes the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan, including the steps of:

[0098] Determining the number of zero eigenvalues ​​of the multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan , and the number of zero eigenvalues Subtract one to get the zero differentiable value of multi-scale semantic enhancement fusion encoding of gastrointestinal CT scans ;

[0099] The multi-scale semantic enhancement fusion encoding of the gastrointestinal CT scan is zero-differentiable Multiply and divide by the number of zero eigenvalues ​​respectively Get the first gastrointestinal CT scan multi-scale semantic enhancement fusion coding field fitting value and the second gastrointestinal CT scan multi-scale semantic enhancement fusion coding field fitting value ;

[0100] Calculate the square root of the sum of squares of all eigenvalues ​​of the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain the multi-scale semantic enhancement fusion coding mode representation value of the gastrointestinal CT scan:

[0101] ;

[0102] in, The first one represents the multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan eigenvalues, It represents the representation value of multi-scale semantic enhancement fusion coding mode of gastrointestinal CT scan;

[0103] The multi-scale semantic enhancement fusion encoding of gastrointestinal CT scans is used to encode zero differentiable values. The power function of each eigenvalue of the multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan is calculated as an exponent , and multiply the first gastrointestinal CT scan multi-scale semantic enhancement fusion encoding field fitting value To obtain the first gastrointestinal CT scan multi-scale semantic enhancement fusion coding intermediate feature map ;

[0104] The multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan is combined with the multi-scale semantic enhancement fusion coding field fitting value of the second gastrointestinal CT scan and the representation value of the multi-scale semantic enhancement fusion coding mode of gastrointestinal CT scan Perform point multiplication to obtain the second gastrointestinal CT scan multi-scale semantic enhancement fusion encoding intermediate feature map ;

[0105] Calculate the first gastrointestinal CT scan multi-scale semantic enhancement fusion coding intermediate feature map Multi-scale semantic enhancement fusion encoding of the intermediate feature map with the second gastrointestinal CT scan The weighted points minus To obtain the optimized gastrointestinal CT scan multi-scale semantic enhancement fusion encoding feature map, where and represents the weighted hyperparameter, It means point multiplication by position. Finally, the optimized gastrointestinal CT scan multi-scale semantic enhancement fusion coding feature map is subjected to image semantic segmentation to obtain the image semantic segmentation result.

[0106] That is, based on the vector field fitting law of high-dimensional manifolds, the isolated zeros in the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan are used as the differentiable dimension of the high-dimensional manifold, so as to fix the normal direction pointing to the distribution center near the local coordinates represented by the eigenvalues ​​of the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan. This avoids invalid repetition in the probability mapping process caused by the unstructured characteristics of the rich distribution of the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan by effectively aligning the feature distribution pattern of the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to the probability density differentiable field, thereby improving the accuracy of the image semantic segmentation results obtained by performing image semantic segmentation.

[0107] In summary, the gastrointestinal surgical assistance system 100 based on the embodiment of the present application is explained, which obtains the gastrointestinal CT scan image of the target patient object, uses the image recognition and processing algorithm based on computer vision and AI to perform image preprocessing on the gastrointestinal CT scan image, and then extracts the main semantic features and boundary semantic features from the preprocessed image, and intelligently generates the image semantic segmentation result containing the marked tumor boundary based on the fine-grained enhanced fusion representation between the two features. In this way, it is possible to capture subtle changes that are difficult for the human eye to discern, especially in low-contrast and small lesions, reducing misdiagnosis and missed diagnosis caused by the limitations of human eye observation, thereby more accurately identifying and annotating tumor boundary information, and providing doctors with intuitive visual assistance. In addition, the automated processing flow significantly reduces the time for doctors to perform manual analysis and improves the speed and efficiency of diagnosis.

[0108] As described above, the gastrointestinal surgery assistance system 100 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a gastrointestinal surgery assistance algorithm. In a possible implementation, the gastrointestinal surgery assistance system 100 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the gastrointestinal surgery assistance system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the gastrointestinal surgery assistance system 100 can also be one of the many hardware modules of the wireless terminal.

[0109] Alternatively, in another example, the gastrointestinal surgery assisting system 100 and the wireless terminal may be separate devices, and the gastrointestinal surgery assisting system 100 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0110] Figure 4 FIG. 1 is a flow chart of a gastrointestinal surgery assisting method according to an embodiment of the present application. Figure 4 As shown, the gastrointestinal surgical assistance method according to the embodiment of the present application includes: S110, acquiring a gastrointestinal CT scan image of a target patient object; S120, performing image denoising and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image; S130, performing semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain a gastrointestinal CT scan trunk semantic coding feature and a gastrointestinal CT scan boundary feature semantic coding feature; S140, performing trunk-boundary fine-grained enhancement interaction on the gastrointestinal CT scan trunk semantic coding feature and the gastrointestinal CT scan boundary feature semantic coding feature to obtain a gastrointestinal CT scan multi-scale semantic enhancement fusion. The combined coding features include: performing feature dissociation on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain a set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local features; performing fine-grained semantic feature compensation aggregation on the set of gastrointestinal CT scan trunk semantic local features and the set of gastrointestinal CT scan boundary semantic local features to obtain the gastrointestinal CT scan multi-scale semantic enhancement fusion coding features; S150, based on the gastrointestinal CT scan multi-scale semantic enhancement fusion coding features, obtaining an image semantic segmentation result, wherein the image semantic segmentation result includes a marked tumor boundary.

[0111] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned gastrointestinal surgical auxiliary method have been described in detail above. Figures 1 to 3 The invention has been described in detail in the description of the gastrointestinal surgical assisting system, and therefore, its repeated description will be omitted.

[0112] The above has described various implementations of the present disclosure, and the above description is exemplary and not exhaustive. It is not limited to the disclosed implementations, and many modifications and changes are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The choice of terms used in this article is intended to best explain the principles of each implementation, practical application or improvement of technology in the market, or to enable other ordinary technicians in this technical field to understand the various embodiments disclosed herein.

Claims

1. A gastrointestinal surgical auxiliary system, characterized in that: include: A gastrointestinal CT scan image acquisition module, used to acquire a gastrointestinal CT scan image of a target patient; A CT scan image preprocessing module, used for performing image noise reduction and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image; A CT scan image feature extraction module, used for performing semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain gastrointestinal CT scan trunk semantic coding features and gastrointestinal CT scan boundary feature semantic coding features; A CT scan image multi-scale feature fusion module is used to perform trunk-boundary fine-grained enhancement interaction on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain gastrointestinal CT scan multi-scale semantic enhancement fusion coding features; wherein the CT scan image multi-scale feature fusion module includes: a CT feature dissociation unit, used to perform feature dissociation on the gastrointestinal CT scan trunk semantic coding features and the gastrointestinal CT scan boundary feature semantic coding features to obtain a set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local features; a CT feature compensation aggregation unit, used to perform fine-grained semantic feature compensation aggregation on the set of gastrointestinal CT scan trunk semantic local features and the set of gastrointestinal CT scan boundary semantic local features to obtain the gastrointestinal CT scan multi-scale semantic enhancement fusion coding features; A semantic segmentation module, used to obtain an image semantic segmentation result based on the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan, wherein the image semantic segmentation result includes a marked tumor boundary; Wherein, the CT feature compensation aggregation unit includes: A trunk-boundary fine-grained extraction subunit is used to input each group of corresponding channel dimensions of the gastrointestinal CT scan trunk semantic local feature matrix and the gastrointestinal CT scan boundary semantic local feature matrix in the set of gastrointestinal CT scan trunk semantic local feature matrix and the gastrointestinal CT scan boundary semantic local feature matrix into a shared semantic information extraction module based on a twin network structure to obtain a set of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices; A trunk-boundary semantic compensation feature calculation subunit, used for performing semantic feature compensation on the set of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices; The trunk-boundary fine-grained feature semantic enhancement subunit is used to semantically enhance the set of trunk-boundary fine-grained local shared semantic feature matrices of the gastrointestinal CT scan based on the set of trunk-boundary semantic compensation text semantic encoding feature matrices of the gastrointestinal CT scan to obtain a multi-scale semantic enhancement fusion encoding feature map of the gastrointestinal CT scan as the multi-scale semantic enhancement fusion encoding feature of the gastrointestinal CT scan.

2. The gastrointestinal surgery assisting system according to claim 1, characterized in that: The CT scan image feature extraction module is used to: input the gastrointestinal CT scan enhanced image into a dual-branch boundary information loss compensation network of a trunk network and a boundary feature extraction branch to obtain a gastrointestinal CT scan trunk semantic coding feature map as the gastrointestinal CT scan trunk semantic coding feature and a gastrointestinal CT scan boundary feature semantic coding feature map as the gastrointestinal CT scan boundary feature semantic coding feature.

3. The gastrointestinal surgery auxiliary system according to claim 2, characterized in that: The CT feature dissociation unit is used to perform feature dissociation on the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map along the channel dimension of the gastrointestinal CT scan trunk semantic encoding feature map and the gastrointestinal CT scan boundary feature semantic encoding feature map to obtain a set of gastrointestinal CT scan trunk semantic local feature matrices as the set of gastrointestinal CT scan trunk semantic local features and a set of gastrointestinal CT scan boundary semantic local feature matrices as the set of gastrointestinal CT scan boundary semantic local features.

4. The gastrointestinal surgery auxiliary system according to claim 3, characterized in that: The backbone-boundary semantic compensation feature calculation subunit is used to: Inputting each gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrix into a semantic compensation decoding module based on a large language model to obtain a set of gastrointestinal CT scan trunk-boundary semantic compensation text descriptions; Each gastrointestinal CT scan trunk-boundary semantic compensation text description in the set of the gastrointestinal CT scan trunk-boundary semantic compensation text description is respectively input into a semantic encoder based on a text convolutional neural network model to obtain a set of semantic encoding feature matrices of the gastrointestinal CT scan trunk-boundary semantic compensation text.

5. The gastrointestinal surgery auxiliary system according to claim 4, characterized in that: The backbone-boundary fine-grained feature semantic reinforcement subunit is used to: Inputting each corresponding group of gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices and gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices in the set of the gastrointestinal CT scan trunk-boundary fine-grained local shared semantic feature matrices and the set of the gastrointestinal CT scan trunk-boundary semantic compensation text semantic encoding feature matrices into a fine-grained semantic interaction compensation module to obtain a set of gastrointestinal CT scan trunk-boundary fine-grained local interactive semantic reinforcement feature matrices; The set of the gastrointestinal CT scan trunk-boundary fine-grained local interaction semantic enhancement feature matrices is aggregated along the channel dimension to obtain the gastrointestinal CT scan multi-scale semantic enhancement fusion encoding feature map.

6. The gastrointestinal surgery auxiliary system according to claim 5, characterized in that: The semantic segmentation module is used to perform image semantic segmentation on the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain the image semantic segmentation result, and the image semantic segmentation result includes the marked tumor boundary.

7. The gastrointestinal surgery auxiliary system according to claim 6, characterized in that: The semantic segmentation module is used to: use an image semantic segmenter based on a Softmax function to process the multi-scale semantic enhancement fusion coding feature map of the gastrointestinal CT scan to obtain the image semantic segmentation result, wherein the image semantic segmentation result includes a marked tumor boundary.

8. A gastrointestinal surgery assisting method, using the gastrointestinal surgery assisting system according to claim 1, characterized in that: include: Acquire a gastrointestinal CT scan image of a target patient; Performing image noise reduction and contrast enhancement on the gastrointestinal CT scan image to obtain a gastrointestinal CT scan enhanced image; Performing semantic coding and boundary feature extraction on the gastrointestinal CT scan enhanced image to obtain gastrointestinal CT scan trunk semantic coding features and gastrointestinal CT scan boundary feature semantic coding features; The main-boundary fine-grained enhancement interaction is performed on the main semantic coding features of the gastrointestinal CT scan and the semantic coding features of the boundary features of the gastrointestinal CT scan to obtain the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan, including: feature dissociation is performed on the main semantic coding features of the gastrointestinal CT scan and the semantic coding features of the boundary features of the gastrointestinal CT scan to obtain a set of main semantic local features of the gastrointestinal CT scan and a set of boundary semantic local features of the gastrointestinal CT scan; fine-grained semantic feature compensation aggregation is performed on the set of main semantic local features of the gastrointestinal CT scan and the set of boundary semantic local features of the gastrointestinal CT scan to obtain the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan; Based on the multi-scale semantic enhancement fusion coding features of the gastrointestinal CT scan, an image semantic segmentation result is obtained, and the image semantic segmentation result includes a marked tumor boundary.

Images