Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and the method is applied to the field of image processing, and comprises the steps: obtaining an initial scanning image of a target detection object; calling a preset first segmentation network to obtain an initial segmentation mask of the initial scanning image; and calling a preset second segmentation network to optimize the initial segmentation mask to obtain a target segmentation image. According to the scheme, the second segmentation network is used for accurately optimizing the fuzzy, fractured and misconnected regions of the trachea in the initial segmentation mask, and finally, the target segmentation image of which the topological continuity and the edge precision are remarkably enhanced is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to an image processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] The core challenge faced by existing airway segmentation imaging technology is that the airway terminal branches are small in diameter, and have weak grayscale differences with the surrounding lung parenchyma in the CT scan image. This low-contrast characteristic makes it difficult for the segmentation algorithm to accurately identify the boundary, resulting in errors such as blurred boundaries, structure breaks, and abnormal connections in the generated airway three-dimensional model. These segmentation defects directly distort the true topological structure of the airway, making it impossible for medical personnel to accurately evaluate the pathology or plan the surgery based on the distorted model, thereby seriously affecting the reliability of clinical diagnosis. SUMMARY

[0003] The present application provides an image processing method, device, electronic device, and storage medium, aiming to improve the imaging accuracy of airway scan images. The technical solution is as follows: In a first aspect, the embodiments of the present application provide an image processing method, comprising: obtaining an initial scan image of a target detection object; calling a preset first segmentation network to obtain an initial segmentation mask of the initial scan image; calling a preset second segmentation network to optimize the initial segmentation mask to obtain a target segmentation image.

[0004] In a second aspect, the embodiments of the present application provide an image processing device, comprising: an image obtaining unit configured to obtain an initial scan image of a target detection object; an image segmentation unit configured to call a preset first segmentation network to obtain an initial segmentation mask of the initial scan image; an image generating unit configured to call a preset second segmentation network to optimize the initial segmentation mask to obtain a target segmentation image.

[0005] In a third aspect, the embodiments of the present application provide an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program, when executed by the processor, implements the image processing method of any one of the above.

[0006] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program, when executed, implements the image processing method of any one of the above.

[0007] In the technical solution, the initial segmentation mask is generated by processing the initial scan image by the first segmentation network to establish the basic topology of the main air tube and branch air tube, thereby providing a key spatial reference for subsequent repair. The second segmentation network receives the initial scan image and the initial segmentation mask as inputs, so that the second segmentation network can align and fuse the complete gray scale information of the original image with the error distribution in the initial segmentation result, thereby guiding the second segmentation network to accurately optimize the areas of the air tube blur, fracture and misconnection in the initial segmentation mask, and finally generate a target segmentation image with significantly enhanced topology continuity and edge accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0009] Figure 1 is a scene schematic diagram of an image processing method provided by an embodiment of the present application; Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application; Figure 3 is a scene schematic diagram of an image processing method provided by an embodiment of the present application; Figure 4 is a flowchart of an image processing method provided by an embodiment of the present application; Figure 5 is a scene schematic diagram of an image processing method provided by an embodiment of the present application; Figure 6 is a flowchart of an image processing method provided by an embodiment of the present application; Figure 7 is a scene schematic diagram of an image processing method provided by an embodiment of the present application; Figure 8 is a structural schematic diagram of an image processing device provided by an embodiment of the present application; Figure 9 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to make the features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0011] The technical solutions in the present application will be described clearly and completely in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B: "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0012] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features.

[0013] The embodiments of the present application provide an image processing method, and the execution subject of the image processing method is an image processing device or an electronic device with an image processing device. The following will be described in detail, and it should be noted that the description order of the following embodiments is not limited as the preferred order of the embodiments. Please refer to Figure 1 , Figure 1 is a scene diagram of an image processing method provided by the embodiments of the present application. The specific process of the image processing method can be as follows: Please refer to Figure 1 , Figure 1 is a scene diagram of an image processing method provided by the embodiments of the present application. As shown in Figure 1 , the initial scan image refers to the original three-dimensional medical image of the target detection object obtained by the computer tomography technology, that is, the three-dimensional volume data composed of a series of continuous two-dimensional cross-sectional slices, wherein each pixel point is actually a voxel, which contains the X-ray attenuation value information of the corresponding voxel of the scanned part. In the embodiments of the present application, the initial scan image is preferably a chest scan image. It should be understood that the application range of the image processing method provided by the embodiments of the present application is not limited to the chest part, and can be extended and applied to similar image processing of scan images of other parts of the human body, such as the abdomen, the head, etc. The specific reference to the chest does not constitute a limitation on the protection scope of the present application.

[0014] The first segmentation network and the second segmentation network are based on an improved 3D U-Net architecture network, which includes a five-layer encoder and a five-layer decoder. Each layer of the encoder and the decoder is composed of two three-dimensional convolution layers, each convolution layer uses a convolution kernel with a size of 3 by 3 by 3 and a step of 1 for feature extraction, followed by a batch normalization layer to stabilize the training process, and a ReLU activation function to introduce a nonlinear transformation. In the encoder part, the third to fifth layers integrate a cross-space attention module, which compresses the channel dimension to one quarter of the original by a 1 by 1 by 1 convolution operation, and calculates a spatial attention weight matrix, thereby enhancing the feature expression of branch points and narrow areas. The decoder part uses a residual connection mechanism, and each decoding layer first performs upsampling through a three-dimensional transpose convolution, which uses a kernel with a size of 2 by 2 by 2 and a step of 2 to expand the feature map size, and then splices the upsampled features with the corresponding encoder layer features to improve gradient flow efficiency and alleviate the gradient vanishing problem. Finally, the output layer generates a single-channel binary segmentation mask through a 1 by 1 by 1 three-dimensional convolution layer and a Sigmoid activation function, thereby completing the segmentation task.

[0015] In actual application, the first segmentation network first receives the initial scan image and segments the main bronchus region and the branch bronchus region in the initial scan image. The first segmentation network outputs an initial segmentation mask based on the main bronchus region and the branch bronchus region, wherein the initial segmentation mask is a single-channel matrix with the same size as the input image, and the pixels with a value of 1 represent the bronchus region including the main bronchus and the branch, and the pixels with a value of 0 represent the background region.

[0016] The second segmentation network determines the bronchus fuzzy region, the branch broken region and the branch misconnection region in the initial segmentation mask by comparing the initial segmentation mask with the gold standard mask. After calling the second segmentation network to optimize the bronchus fuzzy region, the branch broken region and the branch misconnection region in the initial segmentation mask based on the gold standard mask, a target segmentation image is generated.

[0017] In the embodiments of the present application, the initial segmentation mask is generated by processing the initial scan image through the first segmentation network to establish the basic topological structure of the main bronchus and the branch bronchus, thereby providing a key spatial reference for subsequent repair. By pre-setting the second segmentation network to simultaneously receive the initial scan image and the initial segmentation mask as input, the second segmentation network can align and fuse the complete gray scale information of the original image with the error distribution in the initial segmentation result, thereby guiding the second segmentation network to accurately optimize the bronchus fuzzy, broken and misconnection regions in the initial segmentation mask, and finally generate a target segmentation image with significantly enhanced topological continuity and edge accuracy.

[0018] Based onFigure 1 The scenario shown below will be combined with Figures 2-7 , a detailed description of the image processing method provided by the embodiment of the application.

[0019] In the existing CT image tracheal segmentation technology, the tracheal end branch is difficult for the segmentation network to accurately identify the boundary region due to the small tube diameter and weak gray difference with the surrounding lung parenchyma. In a low-contrast environment, image noise interference aggravates the edge blurring effect, and the segmentation result often appears broken or under-segmentation phenomenon, which affects the topological continuity of tracheal three-dimensional reconstruction. The connection between the main trachea and the branch trachea is prone to lose the connectivity of the small branch due to local gray level mutation, resulting in non-physiological interruption of the end airway structure in the three-dimensional model.

[0020] If the above problems are not solved, the structural defects of the tracheal model will directly affect the positioning accuracy of the virtual endoscope system and increase the risk of instrument misplacement in the interventional surgery. The small lesions covered by the under-segmented region cannot trigger abnormal markers in the computer-aided diagnosis system, thereby affecting the accuracy of disease evaluation.

[0021] Based on the above situation, the image processing method provided by the embodiment of the application. Please refer to Figure 2 , Figure 3 is a flowchart of the image processing method provided by the embodiment of the application. As Figure 2 shown, the method of the embodiment of the application can include the following steps S101-S103.

[0022] S101, obtaining the initial scanning image of the target detection object.

[0023] Specifically, the target detection object is placed in the CT scanning device, and the CT scanning device rotates the X-ray tube and the detector around the target part of the target detection object according to the scanning parameters set according to the clinical requirements. The detector of the CT scanning device continuously acquires the X-ray signal attenuated after penetrating the target detection object, thereby obtaining the original two-dimensional projection data set.

[0024] The two-dimensional projection data is processed by using algorithms such as filtered back projection or iterative reconstruction, the X-ray attenuation coefficient is calculated through mathematical transformation, and the X-ray attenuation coefficient is converted into a gray value represented by Hounsfield unit, and then a set of continuous and high-resolution axial two-dimensional cross-sectional slice images are reconstructed, wherein the Hounsfield unit is used to quantify the standardized measurement unit of the relative attenuation degree of human tissue to X-ray.

[0025] The two-dimensional cross-sectional slice image is sequentially stacked along the vertical direction by the three-dimensional reconstruction technology to obtain the initial scanning image.

[0026] Please refer toFigure 3 , Figure 3 is a scene schematic diagram of an image processing method provided by an embodiment of the present application. As shown in the figure, the initial scan image is a certain chest CT cross-sectional image, which directly presents the anatomical structure of the human thoracic cavity in terms of gray difference; the large black areas on the left and right sides of the image are the gas-filled lungs, in which the branch-like distributed bronchus and blood vessel sections can be seen, the white part in the central region is the mediastinum, which contains organs such as the heart and large blood vessels, and the density difference between different tissues is different, that is, from the low-density black bronchus to the high-density white bone. Figure 3

[0027] S102, calling a preset first segmentation network to obtain an initial segmentation mask of the initial scan image.

[0028] Specifically, the first segmentation network is used to extract features from the initial scan image, the multi-layer three-dimensional convolution structure in the encoder of the first segmentation network is used to gradually capture multi-scale features from local details to global context, and the attention mechanism is embedded in the middle and high layer network to strengthen the feature response of the bronchus branch point and the narrow area; the decoder is used to reconstruct the spatial details through upsampling and residual connection, and finally the output layer generates a probability map representing the bronchus region, and the binary initial segmentation mask is obtained after thresholding the probability map.

[0029] S103, calling a preset second segmentation network to optimize the initial segmentation mask to obtain a target segmentation image.

[0030] Specifically, the second segmentation network receives the initial scan image and the initial segmentation mask as dual-channel input, identifies the bronchus fuzzy area by comparing the data loss of the initial segmentation mask with the preset bronchus boundary data in the gold standard mask, obtains the topological feature vector of the bronchus in the initial segmentation mask, wherein the topological feature vector includes branch length, angle and connection point coordinates, and compares the topological feature vector with the preset topological feature data to identify the branch broken area and the branch misconnection area of the bronchus by calculating the data difference value.

[0031] The second segmentation network is called to generate a preset style of target segmentation image after optimizing the bronchus fuzzy area, the branch broken area and the branch misconnection area of the initial segmentation mask based on the gold standard mask. For example, the edge contrast of the bronchus fuzzy area is enhanced to refine the bronchus boundary, the branch broken area is connected according to the topological continuity prior to restore the lumen connectivity, and the branch misconnection area is eliminated to correct the anatomical structure.

[0032] ​In the embodiments of the present application, the initial segmentation mask is generated by processing the initial scan image through the first segmentation network to determine the basic topology of the main bronchus and the branch bronchus in the image, thereby providing a key spatial reference for subsequent repair. The second segmentation network receives the initial scan image and the initial segmentation mask as input at the same time, so that the second segmentation network can align and fuse the complete gray scale information of the original image with the error distribution in the initial segmentation result, thereby guiding the second segmentation network to accurately optimize the areas of the bronchus blur, fracture and misconnection in the initial segmentation mask, and finally generate a target segmentation image with significantly enhanced topology continuity and edge accuracy.

[0033] In some of the above schemes of the present application, the generation process of the initial segmentation mask may cause the loss of the end details of the small branch bronchus. Based on the above situation, please refer to Figure 4 , Figure 4 is a flowchart of an image processing method provided by an embodiment of the present application. As shown in Figure 4 , the method of the present application can include the following steps S201-S202.

[0034] S201, determining the main bronchus region and the branch bronchus region in the initial scan image based on the first segmentation network.

[0035] In the embodiments of the present application, the first segmentation network can analyze the features of the initial scan image, and identify the regions conforming to the morphological characteristics of the main bronchus and the branch bronchus, wherein the main bronchus region is identified as a continuous tubular structure with a larger diameter, and the branch bronchus region is identified as a tree-like bifurcation structure extending from the end of the main bronchus.

[0036] It should be noted that the loss function used by the first segmentation network during training is:

[0037] wherein, the Dice loss, the cross-entropy loss, the skeleton loss, the Dice loss, the cross-entropy loss is used to determine the segmentation accuracy of the main bronchus and the branch bronchus, and the skeleton loss is used to constrain the center line continuity of the segmentation result, thereby preliminarily preserving the bronchus tree topology. The skeleton loss is specifically:

[0038] wherein, is a skeleton extraction function, and MAE is the mean absolute error.

[0039] Specifically, the first segmentation network extracts features from the initial scanned image and progressively downsamples them through multiple 3D convolutional layers in its encoder to capture multi-scale features. The third to fifth layers of the 3D convolutional layers integrate cross-spatial attention modules. The decoder uses a residual connection mechanism to upsample the acquired features and concatenates them with the corresponding layer features of the encoder through transposed convolution to progressively restore spatial details and reconstruct the tracheal structure.

[0040] The output layer of the first segmentation network is convolutionally processed and a sigmoid activation function is used to generate a probability value for each voxel in the initial scanned image. A probability map is then generated based on these probability values, where the probability value of each voxel represents the likelihood that it belongs to the trachea. Finally, the probabilities are calculated based on a preset probability threshold. Figure Two Values ​​are used to determine the location of the main tracheal region and the branch tracheal regions.

[0041] S202, an initial segmentation mask is generated based on the main trachea region and the branch trachea region.

[0042] Specifically, based on the main trachea region and branch trachea region determined in step S201 above, the probability map is binarized. That is, by setting a specific probability threshold, voxel points with probability values ​​higher than the threshold are assigned a value of 1 in the generated mask. The voxel points with a value of 1 together constitute the connected region of the main trachea and branch trachea, while voxel points with probability values ​​lower than the threshold are assigned a value of 0 as the background. Finally, a binary initial segmentation mask that completely corresponds to the spatial size of the initial scan image and contains only 0 and 1 values ​​is generated.

[0043] Please refer to the following: Figure 5 , Figure 5 This is a schematic diagram of a scene illustrating an image processing method provided in an embodiment of this application. For example... Figure 5 As shown, the red pixel area in the initial segmentation mask represents the trachea distribution area, and the black pixel area corresponds to the background area.

[0044] In this embodiment, the main trachea and branch trachea regions can be effectively segmented from the initial scan image, and an intuitive color initial segmentation mask can be generated. This improves the accuracy and completeness of tracheal segmentation, providing a good foundation for subsequent optimization. Furthermore, this scheme overcomes the limitations of traditional methods in handling complex tracheal structures by employing deep learning networks, especially significantly improving the segmentation effect for small branches.

[0045] In some of the above schemes of the present application, the topological structure data of the trachea is not effectively utilized when segmenting the main trachea region and the branch trachea region, resulting in insufficient segmentation accuracy in complex trachea structure regions, and easy to produce broken or missegmentation. Based on the above situation, in a feasible implementation, when performing the step of determining the main trachea region and the branch trachea region in the initial scan image based on the first segmentation network, the following steps are specifically performed: Obtain the trachea topological structure data in the initial scan image.

[0046] In the embodiments of the present application, the trachea topological structure data includes three-dimensional coordinates, connection relationship and diameter change information of the main trachea and the trachea branches.

[0047] Call the first segmentation network to segment the main trachea region and the branch trachea region from the initial scan image based on the trachea topological structure data.

[0048] Specifically, the first segmentation network performs down-sampling on the initial scan image through an encoder to extract image features of the initial scan image. A cross-space attention module is introduced in the third to fifth layers of the encoder. The anatomical prior information provided by the trachea topological structure data is used to calculate the spatial attention weight of the cross-layer features, and the response features of the connection between the main trachea and the branch trachea and the small branch region are highlighted.

[0049] The decoder of the first segmentation network performs up-sampling on the feature map through residual connection and transposed convolution, and splices the up-sampling result with the same layer features of the encoder to gradually restore the spatial details and reconstruct the complete tree-like topological structure of the trachea. The network output layer of the first segmentation network generates a binary probability map through convolution and Sigmoid activation function, so as to accurately segment the main trachea region and its continued branch trachea region in accordance with the hierarchical relationship of the trachea.

[0050] In the embodiments of the present application, by obtaining and utilizing the trachea topological structure data implied in the initial scan image, and calling the first segmentation network to perform segmentation based on the data, the anatomical prior knowledge of the trachea is integrated into the feature learning process of the network; this integration mechanism guides the network to focus on strengthening the identification of small trachea branches and their end structures during feature extraction according to the inherent tree-like hierarchy and spatial connection relationship of the trachea. This method significantly improves the segmentation ability of the small branch end with weak gray difference in low contrast environment, effectively suppresses the broken or misjudgment caused by edge blur and noise interference, so as to ensure that the generated trachea region not only has accurate shape, but also maintains correct topological connectivity from small trachea branches to their tips as a whole.

[0051] In some schemes of the above-mentioned schemes of the present application, when the initial segmentation mask is optimized, there is a problem that the trachea ambiguous region, the branch broken region and the branch misconnection region in the initial segmentation mask cannot be accurately recognized, resulting in that the trachea boundary is still not clear or the structure is broken after subsequent optimization, affecting the accuracy of trachea optimization. Based on the above situation, please refer to Figure 6 , Figure 6 is a flowchart of an image processing method provided by an embodiment of the present application. As shown in Figure 6 , the method of the embodiment of the present application can include the following steps S301-S303.

[0052] S301, determining a trachea ambiguous region in the initial segmentation mask based on a second segmentation network.

[0053] It should be noted that the loss function used by the second segmentation network during training is:

[0054] Among them, is a boundary sensitive loss, is a topological persistent homology loss, is a contrast refinement loss.

[0055] The role of the boundary sensitive loss is to assign higher weights to the pixel points in the boundary region of the segmentation target in the process of training the second segmentation network. The boundary sensitive loss is used to optimize the recognition accuracy of the second segmentation network for the target contour, so as to generate a segmentation result with clearer and sharper boundary. The topological persistent homology loss is used to introduce topological constraints, by calculating the difference between the predicted result and the real label in the topological invariant, guiding the second segmentation network to learn the correct topological structure, avoiding unnecessary holes or breaks, and ensuring the topological correctness of the segmentation result. The contrast refinement loss is used to narrow the distance between the feature points inside the target region in the feature space, while pushing away the distance between the target and background feature points. Through this contrast learning mechanism, the feature representation is refined, so as to suppress noise and improve the detail accuracy and consistency of the final segmentation mask.

[0056] In this embodiment, the blurred tracheal region is determined by comparing the optimized segmentation mask with the tracheal boundary data using an edge detection algorithm to assess data loss. The tracheal boundary data refers to the tracheal boundary data in the tracheal mask gold standard. Specifically, it is a set of pixels precisely defined in the mask by medical personnel based on the grayscale gradient changes of the tracheal wall and surrounding tissues in the scanned image, representing the boundary between the internal cavity of the trachea and the external tissues. The tracheal boundary data is the primary basis for evaluating the accuracy of the tracheal edges in the segmentation results. The tracheal mask gold standard refers to the most authoritative binarized template of the tracheal region, precisely drawn manually on chest scan images by medical imaging personnel. It completely defines the topological structure from the main trachea to its various minor branches and even the tracheal terminal tract, serving as the ultimate benchmark for evaluating the accuracy and reliability of automatic segmentation algorithms.

[0057] S302, Based on the second segmentation network, determine the branch breakage region and branch misconnection region of the trachea in the initial segmentation mask.

[0058] In this embodiment, the branch breakage region and the branch misconnection region are determined by comparing the data matching degree between the topological feature vector of the trachea in the initial segmentation mask and the preset topological feature data. The preset topological feature data is a feature data in the gold standard of tracheal mask. It refers to the data model of the complete tree-like hierarchical structure and spatial connection relationship of the trachea from the main bronchus to the various minor branches, which is predefined in the mask by medical personnel based on their prior knowledge of the human trachea anatomy. For example, the standard connection relationship and length ratio parameters between tracheal branches, branch length, branch angle and coordinates of adjacent branch connection points, etc.

[0059] Specifically, by comparing the matching degree between the topological feature vector and the preset topological features, the branch fracture area and the branch misconnection area of ​​the trachea can be identified.

[0060] S303 optimizes the blurred tracheal region, broken branch region, and misconnected branch region to obtain the target segmentation image.

[0061] Specifically, the second segmentation network is invoked to enhance edge contrast in blurred areas based on the gold standard to refine tracheal boundaries. For branch breakage areas, luminal connectivity is restored by prioritizing topological continuity to connect breakpoints. For misconnected branches, abnormal connections are eliminated by referring to the topological structure of the gold standard. During the restoration process, the second segmentation network uses a multi-level feature alignment mechanism to verify the consistency of the restoration results with the gold standard at both spatial and topological levels, ensuring that the output image meets the gold standard requirements in terms of boundary sharpness, topological continuity, and anatomical accuracy.

[0062] It should be noted that although the blurred tracheal area, broken branch areas, and misconnected branch areas are explicitly identified as key repair targets, the core logic of this design lies in using focused local repair to drive global image optimization. When the second segmentation network optimizes specific defective areas, its repair operations naturally radiate to adjacent areas. For example, when dealing with broken branch areas, the second network optimizes the morphological transition of adjacent branches to maintain overall topological coherence; when enhancing blurred boundaries, it adjusts the edge sharpness of surrounding low-contrast areas.

[0063] Please refer to the following: Figure 7 , Figure 7 This is a schematic diagram of a scene illustrating an image processing method provided in an embodiment of this application. For example... Figure 7 As shown, the red pixel area represents the trachea. It can be seen that there are tracheal fracture areas and blurred areas in the area selected by the white box in the initial segmentation mask. After the tracheal fracture areas and blurred areas are repaired by the second segmentation network, the target segmentation image is obtained.

[0064] In this embodiment, by identifying blurred tracheal regions, broken branch regions, and misconnected branch regions in the initial segmentation mask, precise target regions are provided for subsequent image restoration. This effectively solves the problems of breakage and under-segmentation caused by the small diameter and blurred boundaries of the terminal tracheal branches in the prior art, improving the integrity and accuracy of tracheal segmentation. Consequently, the overall tracheal structure can be better presented, reducing the risk of missed diagnoses of terminal airway lesions and providing more reliable image support for clinical diagnosis.

[0065] In some of the solutions described above in this application, when determining the blurred tracheal region in the initial segmentation mask based on the second segmentation network, there is a problem of difficulty in accurately identifying the blurred tracheal boundary region. Because the grayscale value of the tracheal terminal branch differs little from the surrounding tissue in the initial scan image, relying solely on the network's internal calculations makes it difficult to distinguish the true boundary from noise interference, resulting in insufficient accuracy in locating the blurred region. Based on the above, in a feasible implementation, when performing the step of determining the blurred tracheal region in the initial segmentation mask based on the second segmentation network, the following steps are specifically performed: The preset edge detection algorithm is called to compare the data loss between the initial segmentation mask and the preset tracheal boundary data, and the blurred tracheal region in the initial segmentation mask is obtained.

[0066] In this embodiment, the edge detection algorithm uses the Canny or Sobel operator to extract the contour features of the trachea region in the initial segmentation mask. The trachea boundary data is derived from the accurately labeled trachea boundary contour in the gold standard trachea mask. Data loss is achieved by calculating the difference between the pixel values ​​at the edge of the initial segmentation mask and the corresponding positions of the trachea boundary data. When the pixel values ​​of a local area do not match, that area is determined to be a blurred trachea region.

[0067] In this embodiment, blurred tracheal regions in the initial segmentation mask can be identified. This effectively locates image areas requiring focused restoration, providing precise target regions for subsequent image restoration. This method avoids indiscriminate processing of the entire image, improving the targeting and efficiency of image restoration. Furthermore, comparison with standard tracheal boundary data allows for a more objective evaluation of the segmentation results, contributing to improved accuracy in tracheal segmentation.

[0068] In some of the solutions described above in this application, due to the small diameter of the terminal branches of the trachea and the small difference in grayscale value between them and the surrounding lung parenchyma in the initial scan image, it is difficult to accurately identify the branch breakage areas and misconnected branch areas of the trachea in the segmented image, resulting in the segmentation result failing to fully represent the tracheal topology. Based on the above, in a feasible implementation, when performing the step of determining the branch breakage areas and misconnected branch areas of the trachea in the initial segmentation mask based on the second segmentation network, the following steps are specifically performed: Obtain the topological feature vector of the trachea in the initial segmentation mask.

[0069] In this embodiment, the topological feature vector is generated by extracting the connection relationships, branch lengths, and node distributions of tracheal branches. The preset topological feature data is constructed based on an anatomical model of the standard tracheal structure. The comparison process employs a vector space similarity calculation method, identifying abnormal regions of the topological structure by calculating the Euclidean distance difference between the feature vector and the preset data. For example, a branch breakage region is characterized by a node connection count in the feature vector being lower than a preset threshold, while a branch misconnection region is characterized by abnormal connection vectors between non-adjacent nodes.

[0070] The topological feature vector is compared with the preset topological feature data to determine the branch breakage area and branch misconnection area of ​​the trachea in the initial segmentation mask.

[0071] Specifically, the tracheal skeleton is extracted using image processing algorithms, and topological features such as the number of branches, branch lengths, and branch angles are calculated based on these skeletons to form a topological feature vector. This topological feature vector is then compared with preset topological feature data, which can be normal tracheal topology data annotated by medical experts. If the feature vector of a certain region deviates from the standard value range, it is determined to be a branch break or misconnection. For example, if the connection number of a branch node is 1 and the path length is significantly shorter than the standard value, it is marked as a branch break region; if there is a connection vector between two nodes at non-adjacent levels, it is marked as a branch misconnection region.

[0072] In this embodiment, abnormal regions of the tracheal structure in the initial segmentation mask can be accurately identified, providing guidance for subsequent mask optimization. This helps improve the integrity and accuracy of tracheal segmentation, especially for the segmentation of small branches. Simultaneously, this method utilizes the topological features of the trachea for analysis, which, compared to simple image feature analysis, can better capture the integrity and continuity of the tracheal structure, thereby improving the ability to identify broken and misconnected regions.

[0073] In some of the solutions described above in this application, image segmentation is performed directly after obtaining the initial scanned image of the target object. However, the original scanned image may have large size differences or uneven grayscale distribution, leading to inconsistent input data to the segmentation network and affecting segmentation accuracy and stability. Based on the above, in a feasible implementation, after obtaining the initial scanned image of the target object, the following steps are specifically performed: Adjust the image size of the initial scanned image to the preset image size.

[0074] The initial scanned image after image size adjustment is subjected to image grayscale value normalization processing.

[0075] In this embodiment, image resizing is achieved by using bilinear interpolation to uniformly scale initial scanned images of different resolutions to a preset image size of 512×512×64 pixels. Image grayscale normalization is performed using a linear transformation method, mapping the resized image grayscale values ​​to the range of 0 to 1, ensuring a consistent grayscale distribution range for images from different scanning devices.

[0076] Specifically, the spatial dimensions of the image are adjusted to a preset 512×512×64 pixels using image resampling techniques such as trilinear interpolation to ensure the consistency of the input data. Subsequently, grayscale normalization is performed on the resized image, and the original intensity values ​​are linearly transformed to a continuous range of 0 to 1 using methods such as minimum or maximum scaling. This standardizes the image contrast and reduces intensity variation, providing a standardized input for the subsequent segmentation network.

[0077] For example, the original image size of the initial scanned image is 256×256×128 pixels and has a wide range of grayscale values. First, its voxels are resampled to 512×512×64 pixels using an interpolation algorithm. Then, the grayscale value of each pixel is scaled to between 0 and 1 based on the minimum and maximum intensity values ​​of the image.

[0078] In this embodiment, by eliminating the problem of inconsistent image scale caused by differences in imaging parameters between different CT scanning devices, the insufficiency of contrast caused by discrete grayscale distribution is improved, enabling the subsequent segmentation network to receive standardized input data, significantly improving the stability of the segmentation of the main trachea and small branch trachea regions, and reducing artifact interference and feature misjudgment caused by improper image preprocessing.

[0079] based on Figure 1 The following is a scene illustration, which will be combined with... Figure 8 This application provides a detailed description of the image processing apparatus provided in its embodiments. It should be noted that... Figure 8 The image processing apparatus in the present application is used to execute the present application. Figures 2-7 The methods shown in the embodiments are for illustrative purposes only, illustrating the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figures 2-7 In the embodiment shown, the image processing device 400 may include an image acquisition unit 401, an image segmentation unit 402, and an image generation unit 403, as detailed below: Image acquisition unit 401 is used to acquire an initial scan image of the target detection object; Image segmentation unit 402 is used to call a preset first segmentation network to obtain the initial segmentation mask of the initial scanned image; The image generation unit 403 is used to call a preset second segmentation network to optimize the initial segmentation mask and obtain the target segmentation image.

[0080] In some embodiments, the image segmentation unit 402 is also used to perform: The main trachea region and branch trachea region in the initial scan image are determined based on the first segmentation network; An initial segmentation mask is generated based on the main trachea region and the branch trachea region; In some embodiments, the image segmentation unit 402 is also used to perform: Acquire tracheal topology data from the initial scan image; The first segmentation network is invoked to segment the main trachea region and branch trachea regions from the initial scan image based on tracheal topology data.

[0081] In some embodiments, the image segmentation unit 402 is also used to perform: The blurred tracheal region in the initial segmentation mask is determined based on the second segmentation network; The branch breakage region and branch misconnection region of the trachea in the initial segmentation mask are determined based on the second segmentation network. The blurred tracheal region, broken branch region, and misconnected branch region are optimized to obtain the target segmentation image.

[0082] In some embodiments, the image generation unit 403 is also configured to perform: The preset edge detection algorithm is called to compare the data loss between the initial segmentation mask and the preset tracheal boundary data, and the blurred tracheal region in the initial segmentation mask is obtained.

[0083] In some embodiments, the image generation unit 403 is also configured to perform: Obtain the topological feature vector of the trachea in the initial segmentation mask; By comparing the topological feature vector with the preset topological feature data, the branch breakage region and the branch misconnection region of the trachea in the initial segmentation mask are obtained.

[0084] In some embodiments, the image acquisition unit 401 is also configured to perform: Adjust the image size of the initial scanned image to the preset image size; The initial scanned image after image size adjustment is subjected to image grayscale value normalization processing.

[0085] In this embodiment, an initial segmentation mask is generated by processing the initial scanned image through a first segmentation network to establish the basic topological structure of the main trachea and branch trachea, thus providing a crucial spatial reference for subsequent repair. A pre-defined second segmentation network simultaneously receives the initial scanned image and the initial segmentation mask as input. This allows the second segmentation network to align and fuse the complete grayscale information of the original image with the error distribution in the initial segmentation result. This guides the second segmentation network to precisely optimize the blurred, broken, and misconnected areas of the trachea in the initial segmentation mask using its optimization capabilities, ultimately generating a target segmented image with significantly enhanced topological continuity and edge accuracy.

[0086] Furthermore, the image processing apparatus provided in the above embodiments and the image processing method embodiment belong to the same concept, and the implementation process can be found in the method embodiment, which will not be repeated here.

[0087] The sequence numbers of the embodiments described above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] Please see Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 500 includes a processor 501 and a memory 502. The processor 501 and the memory 502 are electrically connected.

[0089] Processor 501 is the control center of electronic device 500 and may include one or more processing cores. Processor 501 connects various parts of the electronic device using various interfaces and lines. By running or calling computer programs stored in memory 502, and by calling data stored in memory 502, it executes various functions and processes data of the electronic device, thereby providing overall control over the electronic device. Optionally, processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 501 may integrate one or more of the following: CPU, Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user page, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 501 and may be implemented separately using a communication chip.

[0090] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the computer programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0091] Furthermore, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 502 may also include a memory controller to provide processor 501 with access to memory 502.

[0092] In this embodiment, the processor 501 in the electronic device 500 loads the instructions corresponding to the processes of one or more computer programs into the memory 502 according to the following steps, and the processor 501 runs the computer programs stored in the memory 502 to realize various functions, as follows: Obtain the initial scan image of the target object; The initial segmentation mask of the initial scanned image is obtained by calling the preset first segmentation network; The initial segmentation mask is optimized by calling the preset second segmentation network to obtain the target segmented image.

[0093] Optionally, the processor 501 executes the following steps: It calls a preset first segmentation network to obtain the initial segmentation mask of the initial scanned image. The main trachea region and branch trachea region in the initial scan image are determined based on the first segmentation network; An initial segmentation mask is generated based on the main trachea region and the branch trachea region.

[0094] Optionally, the processor 501, in executing the process of determining the main trachea region and branch trachea regions in the initial scan image based on the first segmentation network, specifically performs the following: Acquire tracheal topology data from the initial scan image; The first segmentation network is invoked to segment the main trachea region and branch trachea regions from the initial scan image based on tracheal topology data.

[0095] Optionally, the processor 501 executes the process of calling a preset second segmentation network to optimize the initial segmentation mask and obtain the target segmentation image. Specifically, the following steps are performed: The blurred tracheal region in the initial segmentation mask is determined based on the second segmentation network; The branch breakage region and branch misconnection region of the trachea in the initial segmentation mask are determined based on the second segmentation network. The blurred tracheal region, broken branch region, and misconnected branch region are optimized to obtain the target segmentation image.

[0096] Optionally, the processor 501, when executing the process of determining the tracheal blur region in the initial segmentation mask based on the second segmentation network, specifically performs the following: The preset edge detection algorithm is called to compare the data loss between the initial segmentation mask and the preset tracheal boundary data, and the blurred tracheal region in the initial segmentation mask is obtained.

[0097] Optionally, the processor 501, when executing the process of determining the branch breakage region and branch misconnection region of the trachea in the initial segmentation mask based on the second segmentation network, specifically performs the following: Obtain the topological feature vector of the trachea in the initial segmentation mask; By comparing the topological feature vector with the preset topological feature data, the branch breakage region and the branch misconnection region of the trachea in the initial segmentation mask are obtained.

[0098] Optionally, after acquiring the initial scan image of the target object, the processor 501 specifically executes the following: Adjust the image size of the initial scanned image to the preset image size; The initial scanned image after image size adjustment is subjected to image grayscale value normalization processing.

[0099] In this embodiment, an initial segmentation mask is generated by processing the initial scanned image through a first segmentation network to establish the basic topological structure of the main trachea and branch trachea, thus providing a crucial spatial reference for subsequent repair. A pre-defined second segmentation network simultaneously receives the initial scanned image and the initial segmentation mask as input. This allows the second segmentation network to align and fuse the complete grayscale information of the original image with the error distribution in the initial segmentation result. This guides the second segmentation network to precisely optimize areas of blurred, broken, or misconnected trachea in the initial segmentation mask, ultimately generating a target segmented image with significantly enhanced topological continuity and edge accuracy.

[0100] In addition, the device provided in this application embodiment may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute an image processing method provided in the above embodiment.

[0101] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a computer, it causes the computer to perform the above-described related method steps to implement the image processing method provided in the above embodiments.

[0102] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the image processing method provided in the above embodiments.

[0103] In this application, the apparatus, computer-readable storage medium, computer program product or chip provided in the embodiments are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0104] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0105] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the related couplings or direct couplings or communication connections shown or discussed may be through some interfaces; indirect couplings or communication connections between apparatuses or units may be electrical, mechanical, or other forms.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the initial scan image of the target object; The initial segmentation mask of the initial scanned image is obtained by calling a preset first segmentation network; The initial segmentation mask is optimized by calling a preset second segmentation network to obtain the target segmentation image.

2. The method according to claim 1, characterized in that, The step of calling a preset first segmentation network to obtain the initial segmentation mask of the initial scanned image includes: The main trachea region and branch trachea region in the initial scan image are determined based on the first segmentation network; An initial segmentation mask is generated based on the main trachea region and the branch trachea region.

3. The method according to claim 2, characterized in that, The step of determining the main trachea region and branch trachea regions in the initial scan image based on the first segmentation network includes: Acquire the tracheal topology data from the initial scan image; The first segmentation network is invoked to segment the main trachea region and the branch trachea region from the initial scan image based on the trachea topology data.

4. The method according to claim 1, characterized in that, The step of invoking a preset second segmentation network to optimize the initial segmentation mask to obtain the target segmentation image includes: The tracheal ambiguity region in the initial segmentation mask is determined based on the second segmentation network; Based on the second segmentation network, the branch breakage region and branch misconnection region of the trachea in the initial segmentation mask are determined; The blurred tracheal region, the broken branch region, and the misconnected branch region are optimized to obtain the target segmentation image.

5. The method according to claim 4, characterized in that, The step of determining the blurred tracheal region in the initial segmentation mask based on the second segmentation network includes: A preset edge detection algorithm is invoked to compare the data loss between the initial segmentation mask and the preset tracheal boundary data, thereby obtaining the blurred tracheal region in the initial segmentation mask.

6. The method according to claim 4, characterized in that, The step of determining the branch breakage region and branch misconnection region of the trachea in the initial segmentation mask based on the second segmentation network includes: Obtain the topological feature vector of the trachea in the initial segmentation mask; The topological feature vector is compared with the preset topological feature data to obtain the branch breakage region and branch misconnection region of the trachea in the initial segmentation mask.

7. The method according to claim 1, characterized in that, After acquiring the initial scanned image of the target object, the method further includes: Adjust the image size of the initial scanned image to a preset image size; The initial scanned image after image size adjustment is subjected to image grayscale value normalization processing.

8. An image processing apparatus, characterized in that, The device includes: The image acquisition unit is used to acquire the initial scan image of the target detection object; An image segmentation unit is used to call a preset first segmentation network to obtain the initial segmentation mask of the initial scanned image; The image generation unit is used to call a preset second segmentation network to optimize the initial segmentation mask and obtain the target segmentation image.

9. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the electronic device to perform the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the image processing method as described in any one of claims 1 to 7.

Images