Lightweight multi-scale biomedical image segmentation method

Abstract

The invention discloses a lightweight multi-scale biomedical image segmentation method, and relates to the technical field of medical image processing, and the method comprises the steps: collecting and obtaining a reference position medical image for a target operation task; performing feature importance and feature dominance analysis in combination with key texture feature information of a target operation, and calibrating a datum positioning reference point; real-time image tracking segmentation is carried out, rapid positioning alignment and multi-scale feature extraction are carried out on continuous medical images, and a multi-scale feature map is generated; introducing a feature fusion mechanism, and performing layer-by-layer fusion on the multi-scale feature map to obtain a fused feature map; and obtaining an image segmentation result. The technical problems that existing biomedical image segmentation is difficult to adapt to complex and diversified image segmentation tasks, so that the image segmentation precision is low, and the reliability and safety of surgical navigation are reduced are solved, high-precision image segmentation is realized, and the technical effect of improving the reliability and safety of surgery is achieved.

Description

Technical Field

[0001] The present application relates to the technical field related to medical image processing, and in particular to a lightweight multi-scale biomedical image segmentation method. Background Art

[0002] In the biomedical field, accurate image segmentation is a key link in surgical navigation, disease diagnosis and treatment planning. Real-time image segmentation technology can be used to assist surgical navigation. For example, in minimally invasive surgery, image segmentation can help surgeons identify and avoid important anatomical structures (such as blood vessels and nerves). However, with the rapid development of medical imaging technology, from X-rays, CT, MRI to ultrasound imaging, the types and number of medical images are increasing. These biomedical images are usually highly complex and diverse, containing complex anatomical structures and pathological features, such as different imaging modalities, resolutions, contrasts, and noises, which makes traditional image segmentation methods face huge challenges. Existing image segmentation methods need to process complex biomedical images in real time when facing dynamically changing surgeries, such as organ movement, insertion or removal of instruments, etc. It is often difficult to achieve accurate and efficient image segmentation processing while ensuring segmentation accuracy, thereby affecting the accuracy of locating lesions and anatomical structures during surgery.

[0003] At present, there are technical problems in biomedical image segmentation related technologies that are difficult to adapt to complex and diverse image segmentation tasks, which in turn leads to low image segmentation accuracy and reduced reliability and safety of surgical navigation. Summary of the invention

[0004] The present application provides a lightweight multi-scale biomedical image segmentation method to solve the technical problem that the existing biomedical image segmentation is difficult to adapt to complex and diverse image segmentation tasks, which in turn leads to low image segmentation accuracy and reduced reliability and safety of surgical navigation. The application realizes high-precision image segmentation and achieves the technical effect of improving the reliability and safety of surgery.

[0005] The present application provides a lightweight multi-scale biomedical image segmentation method, comprising: acquiring a reference medical image for a target surgical task, wherein the reference medical image is a reference positioning image before the start of the surgery; performing feature importance and feature dominance analysis based on the reference medical image in combination with key texture feature information of the target surgery, and calibrating reference positioning reference points; based on a real-time surgical process, performing continuous image acquisition, and performing real-time image tracking and segmentation with reference to the reference positioning reference points, including rapid positioning alignment and multi-scale feature extraction of continuous medical images, to generate a multi-scale feature map; introducing a feature fusion mechanism, fusing the multi-scale feature map layer by layer, to obtain a fused feature map; performing lightweight segmentation and decoding on the fused feature map, to obtain an image segmentation result.

[0006] In a possible implementation, importance analysis and feature dominance analysis are performed based on the reference position medical image in combination with key texture feature information of the target surgery, and reference positioning reference points are calibrated. The following processing is also performed: the anatomical key structures in the reference position medical image are analyzed, and key texture feature information is extracted based on the anatomical key structures, wherein the key texture feature information includes but is not limited to texture features, shape features, and edge features; the key texture feature information is sorted by significance through feature dominance analysis, and based on the sorting results, key texture feature points are selected and calibrated as reference positioning reference points.

[0007] In a possible implementation, based on the real-time surgical process, continuous image acquisition is performed, and real-time image tracking and segmentation is performed with reference to the benchmark positioning reference points, and the following processing is also performed: based on the real-time surgical process, continuous image acquisition is performed to obtain a continuous image sequence, and the continuous image sequence includes multiple continuous medical images; based on the continuous image sequence, positioning identification and motion trajectory analysis of the benchmark positioning reference points are performed to generate a benchmark point motion trajectory; with reference to the benchmark point motion trajectory, the continuous medical images are quickly positioned and aligned, and multi-scale feature extraction is performed on the aligned images to generate a multi-scale feature map.

[0008] In a possible implementation, multi-scale feature extraction is performed on the aligned images to capture local details and global information, generate a multi-scale feature map, and perform the following processing: extract an initial medical image based on the continuous image sequence; perform the first multi-scale feature extraction on the initial medical image after identifying and calibrating the benchmark positioning reference points to obtain initial image features; refer to the image acquisition order, traverse the aligned continuous image sequence in turn, combine the initial image features, perform multi-scale feature extraction, and generate a multi-scale feature map.

[0009] In a possible implementation, multi-scale feature extraction is performed, and the following processing is also performed: a multi-scale processing layer is constructed, the multi-scale processing layer includes multiple convolution layers and multi-scale pooling layers, and convolution processing is performed on different areas of the continuous medical image according to the multiple convolution layers to obtain feature maps of different scales; the feature maps of different scales are aggregated through the multi-scale pooling layer, and the multi-scale feature map is output.

[0010] In a possible implementation, a feature fusion mechanism is introduced to fuse the multi-scale feature map layer by layer to obtain a fused feature map, and the following processing is performed: based on the multi-scale feature map, high-level semantic features and low-level spatial detail features are extracted; a feature pyramid network is introduced to fuse the high-level semantic features with the low-level spatial detail features layer by layer, and in each fusion layer, a jump connection is used to combine the high-resolution features with the low-resolution features to generate a fused feature map.

[0011] In a possible implementation, the fused feature map is lightweight segmented and decoded to obtain an image segmentation result, and the following processing is also performed: a lightweight decoding network structure is constructed using deep separable convolution technology; based on the lightweight decoding network structure, the fused feature map is segmented and decoded to output a segmentation result map.

[0012] In a possible implementation, the lightweight multi-scale biomedical image segmentation method also performs the following processing: the real-time image tracking and lightweight segmentation decoding are implemented through a hardware acceleration module, and parallel computing is performed on a GPU or TPU to accelerate multi-scale feature extraction and segmentation decoding, and output the segmented image in real time.

[0013] The lightweight multi-scale biomedical image segmentation method proposed in this application is intended to collect and acquire reference medical images for target surgical tasks; perform feature importance and feature dominance analysis in combination with key texture feature information of the target surgery, and calibrate reference positioning reference points; perform real-time image tracking and segmentation, including rapid positioning alignment and multi-scale feature extraction of continuous medical images, and generate multi-scale feature maps; introduce a feature fusion mechanism, fuse multi-scale feature maps layer by layer, and obtain fused feature maps; and obtain image segmentation results. This solves the technical problem that existing biomedical image segmentation is difficult to adapt to complex and diverse image segmentation tasks, which leads to low image segmentation accuracy and reduced reliability and safety of surgical navigation, and achieves high-precision image segmentation, achieving the technical effect of improving the reliability and safety of surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0015] Figure 1 A schematic flow chart of a lightweight multi-scale biomedical image segmentation method provided in an embodiment of the present application; Figure 2 A schematic diagram of the process of performing real-time image tracking and segmentation in a lightweight multi-scale biomedical image segmentation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0018] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0019] The present application embodiment provides a lightweight multi-scale biomedical image segmentation method, such as Figure 1 As shown, the method includes: Step S100 , for a target surgical task, a reference position medical image is acquired, where the reference position medical image is a reference positioning image before the start of the surgery.

[0020] Preferably, a reference medical image of the target surgical task is acquired as a reference positioning image before the operation begins. Specifically, before the target surgical task is performed, a reference medical image is acquired through medical imaging technology (such as X-ray, CT, MRI, etc.) to determine the surgical site, plan the surgical path, and serve as a reference positioning image for image tracking and segmentation reference during subsequent surgical procedures. This reference medical image plays a vital role in the surgical process. It is not only a map for surgical navigation, helping doctors to accurately locate the surgical site and identify key anatomical structures, but also a benchmark for subsequent real-time image tracking and segmentation. By deeply analyzing and processing this image, doctors can extract key information required for the operation, such as the direction of blood vessels, organ morphology, etc., so as to formulate a more accurate and safe surgical plan.

[0021] Step S200: performing feature importance and feature dominance analysis based on the reference position medical image and combining key texture feature information of the target surgery to calibrate the reference positioning reference points.

[0022] Preferably, feature importance and feature dominance analysis is performed based on the reference medical image in combination with key texture feature information of the target surgery. Specifically, the reference medical image is deeply analyzed to identify key anatomical structures and pathological features related to the surgery to form key texture feature information, including the contours and boundaries of the target organ and its surrounding organs, anatomical structures such as blood vessels and nerve distribution, bone structure, and pathological features such as tumor features, inflammation and infection areas, and diseased tissues; then, feature importance analysis is performed in combination with the key texture feature information of the target surgery, that is, the importance of the features to the surgery is analyzed. At the same time, the dominance of these features is analyzed and evaluated, that is, their clarity and recognizability in the medical image are evaluated. Based on the feature importance analysis and feature dominance analysis, those feature points that are both important and significant are selected and calibrated as reference positioning reference points to provide accurate references for subsequent image tracking segmentation and surgical navigation. These reference points are usually key points in the anatomical structure or obvious signs of pathological features, such as vascular bifurcation points, organ edges, tumor boundaries, etc., which help doctors to more accurately locate the surgical site, plan the surgical path, and monitor the progress of the surgery, thereby improving the accuracy and safety of the surgery.

[0023] In a possible implementation, step S200 further includes step S210, analyzing the key anatomical structures in the reference position medical image, and extracting key texture feature information based on the key anatomical structures, wherein the key texture feature information includes but is not limited to texture features, shape features, and edge features; step S220, ranking the key texture feature information by significance through feature dominance analysis, and selecting key texture feature points to be calibrated as reference points for benchmark positioning based on the ranking results.

[0024] Preferably, the anatomical key structures in the reference medical image are analyzed, and key texture feature information is extracted based on these structures. Specifically, rich anatomical information is identified and annotated from the reference medical image, such as key structures such as organs, blood vessels, bones, etc. as anatomical key structures, and then key texture feature information is extracted based on the anatomical key structures, including but not limited to texture features, shape features, and edge features. Texture features reflect the grayscale and color changes between pixels in the image. In medical images, texture feature extraction is used for lesion detection and analysis. Common texture feature extraction methods include gray level co-occurrence matrix (GLCM), gray level difference matrix (GLDM), gray level size co-occurrence matrix (GLRLM), etc.; shape features are features that describe the geometric shape of an object (such as shape features including area, perimeter, diameter, etc.), which play an important role in abnormal area detection and segmentation in medical images; edge features are features of the outline or boundary of an object in an image, which are crucial for tasks such as image segmentation and target detection. Common edge detection methods include Canny edge detection, Sobel edge detection, etc.

[0025] Preferably, after extracting the key texture feature information, it is necessary to perform feature dominance analysis, that is, to sort the feature information by significance, that is, to sort the features according to their significance and importance, and finally calibrate the benchmark positioning reference points. Specifically, in medical image processing, principal component analysis (PCA), linear discriminant analysis (LDA), etc. are used for feature dominance analysis. According to the results of the feature dominance analysis, key texture feature points are selected as benchmark positioning reference points, which are usually points with significant features or important significance in key anatomical structures, such as vascular bifurcation points, organ boundary points, etc. By calibrating these reference points, accurate references can be provided for subsequent image tracking, segmentation or surgical navigation.

[0026] Step S300, based on the real-time surgical process, continuous image acquisition is performed, and real-time image tracking and segmentation is performed with reference to the benchmark positioning reference points, including rapid positioning alignment and multi-scale feature extraction of continuous medical images to generate a multi-scale feature map.

[0027] Preferably, during the operation, images of the surgical site are continuously collected by medical imaging equipment (such as endoscopes, CT, MRI, etc.), and then real-time image tracking and segmentation are performed with reference to the reference positioning reference points, that is, these reference positioning reference points are matched with the real-time collected surgical images during the real-time operation to achieve rapid positioning and alignment of continuous medical images. Specifically, the real-time collected images are spatially aligned with the reference position medical images using image registration technology, which usually includes image transformation (such as translation, rotation, scaling, etc.) to minimize the difference between the two images; multi-scale feature extraction refers to extracting feature information at different scales from continuous medical images, To have a more comprehensive understanding of the anatomical structure and pathological characteristics of the surgical site, specifically, objects in the surgical site (such as organs, blood vessels, tumors, etc.) may have different sizes and shapes. Through multi-scale analysis, feature information at different scales can be captured, so as to more accurately identify the surgical site. Then, feature extraction methods (including edge detection, corner detection, feature point extraction, and local area feature extraction, etc.) are used to extract precise features in medical images. The extracted feature information can be used for tasks such as image segmentation, target detection and recognition in surgical navigation. Finally, the feature information extracted from different scales is integrated to generate a multi-scale feature map containing the corresponding feature information. It ensures that the images during the operation are consistent with the reference images, and provides comprehensive anatomical structure and pathological feature information, providing doctors with accurate surgical navigation support.

[0028] In one possible implementation, Figure 2 As shown, step S300 further includes step S310, based on the real-time surgical process, performing continuous image acquisition to obtain a continuous image sequence, wherein the continuous image sequence includes multiple continuous medical images; step S320, based on the continuous image sequence, performing positioning identification and motion trajectory analysis of the benchmark positioning reference point to generate a benchmark point motion trajectory; step S330, referring to the benchmark point motion trajectory, quickly positioning and aligning the continuous medical images, and performing multi-scale feature extraction on the aligned images to generate a multi-scale feature map.

[0029] Preferably, during real-time surgery, medical image acquisition equipment is used to perform continuous image acquisition of the surgical site. These devices can capture dynamic changes of the surgical site at a high frame rate and generate an image sequence containing multiple continuous medical images, which provides basic data for subsequent image analysis and processing. Then, based on the continuous image sequence, the positioning identification and motion trajectory analysis of the benchmark positioning reference points are performed. Specifically, in the continuous image sequence, the benchmark positioning reference points are first positioned and identified, that is, the positions of these reference points can be automatically identified in the continuous image sequence through image processing technology (such as feature matching, template matching, etc.). These benchmark positioning reference points are usually anatomical structure points or marker points with significant characteristics or important significance determined before or during surgery; after the benchmark positioning reference points are identified, their motion trajectories need to be analyzed, including tracking the positions of the benchmark positioning reference points in different image frames, and calculating their motion paths, speeds, accelerations and other parameters, to generate benchmark point motion trajectories, which provide important information for subsequent tasks such as image alignment and surgical navigation.

[0030] Preferably, the continuous medical images are quickly positioned and aligned with reference to the motion trajectory of the reference points to eliminate motion artifacts in the image sequence and enable the anatomical structures in different image frames to accurately correspond. Specifically, image registration is a technology for aligning anatomical structures in different image frames. During real-time surgery, a feature-based registration method or an iterative registration method can be used to achieve rapid positioning and alignment of images. The motion trajectory of the reference points can provide important reference information for image registration, thereby improving the accuracy and efficiency of registration. After rapid positioning and alignment, the anatomical structures in different image frames will be able to accurately correspond, which helps doctors observe changes in the surgical site in real time during surgery. Finally, multi-scale feature extraction is performed on the aligned images to help Capturing anatomical structures and pathological features at different scales provides rich information for subsequent image analysis and diagnosis. Specifically, multi-scale feature extraction involves extracting features from images at different scales, such as using filters of different sizes, pyramid structures, or convolutional neural networks. In medical image processing, commonly used multi-scale features include texture features, shape features, edge features, etc., which can capture subtle changes in anatomical structures and pathological features at different scales. After multi-scale feature extraction, a multi-scale feature map can be generated, which contains feature information at different scales and can be used for a variety of medical image processing tasks, such as lesion detection, segmentation, classification, etc., providing doctors with more comprehensive and accurate image information, which helps to improve the accuracy and efficiency of diagnosis.

[0031] In a possible implementation, step S330 further includes step S331, extracting an initial medical image based on the continuous image sequence; step S332, performing a first multi-scale feature extraction on the initial medical image after identifying and calibrating the benchmark positioning reference points to obtain initial image features; step S333, referring to the image acquisition order, traversing the aligned continuous image sequence in sequence, combining the initial image features, performing multi-scale feature extraction, and generating a multi-scale feature map.

[0032] Preferably, a representative medical image is extracted from the continuous image sequence as the initial medical image to reflect the initial state of the surgical site or lesion area. In the initial medical image, the benchmark positioning reference points are automatically identified and calibrated through image processing technology (such as feature point detection, template matching, etc.), and then the initial medical image is subjected to the first multi-scale feature extraction. The multi-scale feature extraction involves extracting features from images at different scales to capture anatomical structures and pathological features at different scales, such as pyramid decomposition, wavelet transform, convolutional neural network, etc. The initial image features can be obtained through the first multi-scale feature extraction; referring to the image acquisition order, the aligned continuous image sequence is traversed in turn. These images have been aligned through image registration technology to ensure that the anatomical structures in different image frames can accurately correspond. During the traversal process, the initial image features are combined to perform multi-scale feature extraction on each aligned medical image. By comparing the feature differences between the current image and the initial image, the dynamic changes of the anatomical structure and pathological features during the operation are captured. After multi-scale feature extraction, a multi-scale feature map can be generated for a variety of medical image processing tasks, such as lesion detection, segmentation, classification, etc., thereby improving the accuracy of medical image segmentation.

[0033] In a possible implementation, step S332 further includes step S3321, constructing a multi-scale processing layer, the multi-scale processing layer includes multiple convolution layers and multi-scale pooling layers, and performing convolution processing on different areas of the continuous medical image according to the multiple convolution layers to obtain feature maps of different scales; step S3322, aggregating the feature maps of different scales through the multi-scale pooling layer, and outputting the multi-scale feature map.

[0034] Preferably, a multi-scale processing layer is constructed to process the input image and extract features of different scales therefrom, which is usually composed of multiple convolutional layers and multi-scale pooling layers. Specifically, the convolutional layer is the basic unit for feature extraction in deep learning, and extracts local features in the image through convolution operations (i.e., using convolution kernels to slide on the input image and perform dot product operations). In the multi-scale processing layer, multiple convolutional layers can be designed, each of which uses convolution kernels of different sizes or different convolution parameters to extract features of different scales. Each convolutional layer can capture feature information of different regions and scales in the input image; the pooling layer is usually used to reduce the dimension of the feature map and reduce the amount of calculation while retaining important feature information. In the multi-scale processing layer, the multi-scale pooling layer is designed to aggregate feature maps of different scales, which can be achieved by using pooling windows of different sizes or different pooling strategies (such as maximum pooling, average pooling, etc.). The function of the multi-scale pooling layer is to fuse and aggregate feature maps from different convolutional layers to generate an output feature map containing multi-scale feature information.

[0035] Preferably, the continuous medical image is input into the multi-scale processing layer, and convolution processing is performed through multiple convolution layers. Each convolution layer uses its specific convolution kernel and parameters to extract feature information of different regions and scales in the input image. After processing by multiple convolution layers, feature maps of multiple scales can be obtained, and then these feature maps of different scales are input into the multi-scale pooling layer. The multi-scale pooling layer aggregates these feature maps according to its design strategy (such as using pooling windows of different sizes or different pooling strategies). After processing by the multi-scale pooling layer, a multi-scale feature map containing multi-scale feature information is finally output, which contains feature information of different regions and scales in the input image and can be used for subsequent image analysis, classification, detection and other tasks.

[0036] Step S400, introducing a feature fusion mechanism, fusing the multi-scale feature maps layer by layer to obtain a fused feature map.

[0037] Preferably, a feature fusion mechanism is introduced to fuse multi-scale feature maps layer by layer to obtain a fused feature map, wherein the feature fusion mechanism refers to integrating feature information from different scales, different levels or different modalities to extract a more comprehensive and richer feature representation. In medical image processing, especially in surgical navigation and image segmentation tasks, the feature fusion mechanism can significantly improve the accuracy of surgical image segmentation. Specifically, the multi-scale feature map contains structural feature information of different sizes and shapes. Layer-by-layer fusion refers to fusing feature maps of different scales according to a certain strategy (such as element-by-element addition, feature map splicing, channel weighted fusion, etc.) to generate a new and more representative feature map. In the process of layer-by-layer fusion, it is ensured that feature maps of different scales remain consistent in spatial size and number of channels, or are adjusted by upsampling, downsampling, 1x1 convolution and other methods. The final fused feature map is obtained by layer-by-layer fusion, which contains feature information from different scales (anatomical structure and pathological feature information). The fused feature map has higher information density and stronger representation ability. By fusing feature information from different scales, more comprehensive anatomical structures and pathological features can be captured, thereby improving the accuracy of surgical navigation and image segmentation, and being able to cope with noise, occlusion, and lighting changes in the image.

[0038] In a possible implementation, step S400 further includes step S410, extracting high-level semantic features and low-level spatial detail features based on the multi-scale feature map; step S420, introducing a feature pyramid network, fusing the high-level semantic features with the low-level spatial detail features layer by layer, and in each fusion layer, using jump connections to combine high-resolution features with low-resolution features to generate a fused feature map.

[0039] Preferably, high-level semantic features and low-level spatial detail features are extracted based on the multi-scale feature map. Specifically, high-level semantic features usually refer to abstract, global features in the image, which can reflect the objects, scenes or category information in the image. High-level semantic features are usually extracted through deep convolutional layers of convolutional neural networks (CNN); low-level spatial detail features refer to local, detailed features in the image, such as edges, textures, shapes, etc. These features are crucial for tasks such as image segmentation and positioning. In convolutional neural networks (CNN), low-level spatial detail features are usually extracted through shallow convolutional layers.

[0040] Preferably, a feature pyramid network (FPN) is introduced for feature fusion. The feature pyramid network (FPN) is a network structure for multi-scale feature fusion, which can effectively combine high-level semantic features with low-level spatial detail features. Specifically, FPN first constructs a feature pyramid, which is composed of multiple feature maps of different scales, each of which corresponds to a specific scale. In FPN, high-level semantic features are combined with low-level spatial detail features by layer-by-layer fusion, that is, starting from the top-level feature map, its resolution is increased by upsampling (such as bilinear interpolation), and then fused with the feature map of the next layer, such as by addition, splicing, etc. In order to better combine high-resolution features with low-resolution features, FPN introduces jump connections, which directly connect or fuse shallow feature maps with deep feature maps, thereby retaining more detail information. Through jump connections, FPN can effectively combine high-resolution and low-resolution features in each fusion layer; finally, a fused feature map is generated, which contains both the global information of high-level semantic features and the local information of low-level spatial detail features, which can improve the accuracy of image analysis and diagnosis.

[0041] Step S500, performing lightweight segmentation decoding on the fused feature map to obtain an image segmentation result.

[0042] Preferably, the fused feature map is subjected to lightweight segmentation decoding to obtain the image segmentation result. It is a key technology in the field of computer vision, especially in medical image processing, which aims to extract useful information from the complex fused feature map and generate accurate image segmentation results in an efficient manner. Specifically, lightweight segmentation decoding refers to taking the fused feature map as input, and generating the final image segmentation result in an efficient manner through a series of lightweight convolutional layers, upsampling layers, feature fusion modules, etc. Lightweight segmentation decoding refers to taking the fused feature map as input, and generating the final image segmentation result in an efficient manner through a series of lightweight convolutional layers, upsampling layers, feature fusion modules, etc., using lightweight convolutional layers (such as depthwise separable convolution, grouped convolution, etc.) to extract features. These convolutional layers have relatively high It has few parameters and computational complexity, but still maintains good feature extraction capabilities; restores the feature map to the same size as the original image through upsampling layers (such as bilinear interpolation, transposed convolution, etc.) to facilitate pixel-level classification; in the decoding process, introduces a feature fusion module to further integrate feature information at different levels or scales to improve segmentation accuracy; in order to further reduce the amount of computation and model parameters, lightweight strategies such as pruning, quantization, knowledge distillation, etc. can be adopted. These strategies can greatly reduce the size and computational complexity of the model without significantly reducing the segmentation accuracy; after lightweight segmentation decoding, the final image segmentation result can be obtained, usually in the form of a labeled image, in which each pixel is assigned a category label, indicating the anatomical structure or pathological feature to which the pixel belongs.

[0043] In a possible implementation, step S500 further includes step S510, using deep separable convolution technology to construct a lightweight decoding network structure; step S520, based on the lightweight decoding network structure, segmenting and decoding the fused feature map, and outputting a segmentation result map.

[0044] Preferably, depthwise separable convolution is an effective convolution operation optimization method, which decomposes the traditional standard convolution into two parts: depthwise convolution and pointwise convolution. The depthwise convolution performs convolution operations on each input channel separately, while the pointwise convolution uses a 1x1 convolution kernel to combine the outputs of the depthwise convolution. This decomposition method significantly reduces the amount of computation and model parameters, thereby improving the operation efficiency and storage efficiency of the network. Then, the depthwise separable convolution technology is used to construct a decoding network structure with fewer parameters and lower computational complexity. The decoding network is usually used to restore the encoded feature map to the resolution of the original image and generate a segmentation result map. When constructing a lightweight decoding network, multiple decoding layers can be designed, and each decoding layer uses depthwise separable convolution to replace the traditional standard convolution, thereby reducing the complexity and computation of the model while ensuring segmentation performance.

[0045] Preferably, the fused feature map obtained by the feature pyramid network (FPN) is used as the input of the lightweight decoding network for segmentation and decoding. In the lightweight decoding network, the fused feature map will be processed by multiple decoding layers, each decoding layer will use depth-separable convolution to extract features, and gradually restore the resolution of the feature map to the resolution of the original image. In the decoding process, upsampling operations (such as bilinear interpolation, transposed convolution, etc.) can also be introduced to increase the resolution of the feature map. After processing by multiple decoding layers, a segmentation result map with the same resolution as the original image is finally output. Each pixel on the segmentation result map corresponds to a category label, indicating the category to which the pixel belongs. By visualizing the segmentation result map, the segmentation effect of different areas in the image can be intuitively seen, which can reduce the complexity and computational complexity of the model while ensuring the segmentation performance.

[0046] In a possible implementation, step S500 further includes: the real-time image tracking and lightweight segmentation decoding are implemented by a hardware acceleration module, and parallel computing is performed on a GPU or TPU to accelerate multi-scale feature extraction and segmentation decoding, and output the segmented image in real time.

[0047] Preferably, real-time image tracking and lightweight segmentation decoding are realized, and parallel computing is performed with the help of hardware acceleration modules (such as GPU or TPU) to improve processing speed and efficiency, and output segmented images in real time. Specifically, real-time image tracking refers to real-time tracking and positioning of target objects in video streams or continuous image sequences, and continuous tracking of target objects is achieved by extracting, matching and updating key features in the image. Lightweight segmentation decoding refers to segmentation and decoding of images using lightweight network structures to generate segmentation result maps. Lightweight network structures usually reduce the complexity and computational complexity of the model by reducing the number of network layers, using small convolution kernels, introducing depth-separable convolution and other technologies, while ensuring segmentation performance. It improves processing speed and efficiency and is applicable For real-time surgical tasks; the hardware acceleration module refers to the hardware equipment used to accelerate image processing tasks, including the graphics processing unit (GPU) and the tensor processing unit (TPU). The GPU is good at processing large-scale parallel computing tasks, while the TPU is optimized for model reasoning; parallel computing refers to dividing the computing task into multiple subtasks and executing these subtasks simultaneously on multiple processors or cores, that is, through the parallel processing of real-time image tracking and lightweight segmentation decoding, the processing speed and efficiency of medical images are significantly improved. Finally, the trained lightweight segmentation decoding model is deployed on the GPU or TPU, and the parallel computing capability of the GPU or TPU is used to perform multi-scale feature extraction and segmentation decoding on the input image to generate a segmentation result map, that is, output the segmented image in real time.

[0048] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A lightweight multi-scale biomedical image segmentation method, characterized in that: The method comprises: For the target surgical task, a reference position medical image is acquired, where the reference position medical image is a reference positioning image before the start of the surgery; According to the reference position medical image, combining the key texture feature information of the target surgery, feature importance and feature dominance analysis are performed to calibrate the reference positioning reference point; Based on the real-time surgical process, continuous image acquisition is performed, and real-time image tracking and segmentation is performed with reference to the benchmark positioning reference points, including rapid positioning alignment and multi-scale feature extraction of continuous medical images to generate a multi-scale feature map; Introducing a feature fusion mechanism to fuse the multi-scale feature maps layer by layer to obtain a fused feature map; The fused feature map is lightly segmented and decoded to obtain an image segmentation result.

2. A lightweight multi-scale biomedical image segmentation method as claimed in claim 1, characterized in that: According to the reference position medical image, combined with the key texture feature information of the target surgery, importance analysis and feature dominance analysis are performed to calibrate the reference positioning reference points, including: Analyzing the key anatomical structures in the reference position medical image, and extracting key texture feature information based on the key anatomical structures, wherein the key texture feature information includes but is not limited to texture features, shape features, and edge features; The key texture feature information is sorted by significance through feature dominance analysis, and based on the sorting results, key texture feature points are selected and calibrated as benchmark positioning reference points.

3. A lightweight multi-scale biomedical image segmentation method as claimed in claim 1, characterized in that: Based on the real-time surgical process, continuous image acquisition is performed, and real-time image tracking and segmentation is performed with reference to the benchmark positioning reference points, including: Based on the real-time surgical process, continuous image acquisition is performed to obtain a continuous image sequence, wherein the continuous image sequence includes a plurality of continuous medical images; According to the continuous image sequence, positioning identification and motion trajectory analysis of the benchmark positioning reference point are performed to generate a benchmark point motion trajectory; Referring to the motion trajectory of the reference points, the continuous medical images are quickly positioned and aligned, and multi-scale feature extraction is performed on the aligned images to generate a multi-scale feature map.

4. A lightweight multi-scale biomedical image segmentation method as claimed in claim 3, characterized in that: Perform multi-scale feature extraction on the aligned images to capture local details and global information and generate multi-scale feature maps, including: Based on the continuous image sequence, extracting an initial medical image; For the initial medical image, after identifying and calibrating the benchmark positioning reference points, perform a first multi-scale feature extraction to obtain initial image features; Referring to the image acquisition order, the aligned continuous image sequence is traversed in turn, and multi-scale feature extraction is performed in combination with the initial image features to generate a multi-scale feature map.

5. A lightweight multi-scale biomedical image segmentation method as claimed in claim 4, characterized in that: Perform multi-scale feature extraction, including: Constructing a multi-scale processing layer, wherein the multi-scale processing layer includes a plurality of convolutional layers and a multi-scale pooling layer, and performing convolution processing on different regions of the continuous medical image according to the plurality of convolutional layers to obtain feature maps of different scales; The feature maps of different scales are aggregated through the multi-scale pooling layer to output the multi-scale feature map.

6. A lightweight multi-scale biomedical image segmentation method as claimed in claim 1, characterized in that: A feature fusion mechanism is introduced to fuse the multi-scale feature maps layer by layer to obtain a fused feature map, including: Based on the multi-scale feature map, extract high-level semantic features and low-level spatial detail features; A feature pyramid network is introduced to fuse the high-level semantic features with the low-level spatial detail features layer by layer. In each fusion layer, a jump connection is used to combine the high-resolution features with the low-resolution features to generate a fused feature map.

7. A lightweight multi-scale biomedical image segmentation method as claimed in claim 1, characterized in that: The fused feature map is subjected to lightweight segmentation decoding to obtain an image segmentation result, including: Adopting deep separable convolution technology to build a lightweight decoding network structure; Based on the lightweight decoding network structure, the fused feature map is segmented and decoded, and a segmentation result map is output.

8. The lightweight multi-scale biomedical image segmentation method according to claim 1, characterized in that: The real-time image tracking and lightweight segmentation decoding are implemented through a hardware acceleration module, and parallel computing is performed on a GPU or TPU to accelerate multi-scale feature extraction and segmentation decoding, and output a segmented image in real time.

Images