Auxiliary image segmentation method for bronchoscopy

Abstract

The invention relates to an auxiliary image segmentation method for bronchoscopy, and relates to the field of medical image processing, and the method comprises the steps: carrying out the significant feature analysis of an airway image, and determining a plurality of airway significant regions; configuring an initial image segmentation granularity scheme according to the plurality of image saliency degrees; synchronously monitoring and acquiring user physiological features and lens operation features, performing image fuzziness prediction, and generating an image segmentation compensation coefficient; and adjusting the initial image segmentation granularity scheme to obtain a corrected image segmentation granularity scheme, and performing image segmentation on the plurality of airway salient regions. According to the method, the problems that the requirements of different lesion areas are difficult to adapt and false detection or missing detection is easily caused due to the fact that a segmentation strategy cannot be dynamically adjusted according to image features in a traditional method can be solved; according to the method, the segmentation precision and accuracy of the trachea image can be remarkably improved, and efficient and accurate image segmentation is realized, so that different types of lesion areas can be better identified, and false detection and missing detection are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and particularly to an image segmentation method for assisting bronchoscopy examination. Background Art

[0002] The quality of tracheal images is often affected by various factors, such as the physiological responses of patients (such as coughing, breathing, etc.), unstable lens operation, equipment cleanliness, environmental light, etc. This makes the edges of the lesion areas in tracheal images may be blurred or interfered by noise, bringing challenges to subsequent image analysis and lesion recognition.

[0003] Tracheal image segmentation is one of the key technologies in bronchoscopic image analysis, which involves extracting the regions of interest (such as lesion areas) in the image from the background for further analysis. However, traditional tracheal image segmentation methods mostly adopt fixed segmentation granularity and strategies, and it is difficult to dynamically adjust the segmentation granularity according to the specific features of the image, resulting in insufficient accuracy and reliability of the segmentation results, being difficult to meet the requirements of different lesion areas, and prone to false detection or missed detection. Summary of the Invention

[0004] Aiming at the technical problem in traditional airway image segmentation methods that it is difficult to meet the requirements of different lesion areas and prone to false detection or missed detection due to the inability to dynamically adjust the segmentation strategy according to image features, the present invention provides an image segmentation method for assisting bronchoscopy examination to solve this problem.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides an image segmentation method for assisting bronchoscopy examination, including: collecting airway images of the lesion area through a bronchoscope, analyzing the significant features of the airway images to determine multiple airway significant regions; configuring an initial image segmentation granularity scheme according to the multiple image saliencies of the multiple airway significant regions; synchronously monitoring and obtaining the user's physiological features and lens operation features during the airway image collection process, predicting the image blurriness, and generating an image segmentation compensation coefficient; adjusting the initial image segmentation granularity scheme according to the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme, and performing image segmentation on the multiple airway significant regions.

[0006] Preferably, the image segmentation method for bronchoscopy assistance further includes: constructing a saliency feature analyzer based on a Gaussian pyramid, where the saliency feature analyzer includes a brightness extraction unit, a color extraction unit, and an orientation extraction unit; performing multi-scale Gaussian downsampling on the airway image through the brightness extraction unit, the color extraction unit, and the orientation extraction unit to obtain multi-scale airway images; performing deviation analysis on the multi-scale airway images respectively based on the center-surround difference calculation principle to obtain an airway brightness feature map, an airway color feature map, and an airway orientation feature map, and performing weighted superposition processing to fuse and generate an airway saliency image; screening and segmenting the airway saliency image according to a saliency feature threshold, setting the regions with saliency greater than the saliency feature threshold as airway saliency regions to obtain multiple airway saliency regions, where each airway saliency region is marked with an image saliency degree.

[0007] Preferably, the image segmentation method for bronchoscopy assistance further includes: obtaining the position information of the multiple airway saliency regions in the airway image, mapping it into two-dimensional coordinate points, and constructing a two-dimensional distribution matrix; performing distribution dispersion analysis on the two-dimensional distribution matrix, and setting the reciprocal of the distribution dispersion as the saliency distribution concentration; configuring an initial image segmentation granularity scheme according to the saliency distribution concentration and the multiple image saliency degrees.

[0008] Preferably, the image segmentation method for bronchoscopy assistance further includes: if the saliency distribution concentration is greater than or equal to a predetermined reference value, selecting the maximum image saliency degree among the multiple image saliency degrees, and inputting the maximum image saliency degree into a saliency-segmentation granularity comparison library to output a first image segmentation granularity, which is set as the initial image segmentation granularity scheme; if the saliency distribution concentration is less than the predetermined reference value, inputting the multiple image saliency degrees into the saliency-segmentation granularity comparison library respectively to output multiple image segmentation granularities, which are set as the initial image segmentation granularity scheme.

[0009] Preferably, the image segmentation method for bronchoscopy assistance further includes: within the same time zone of airway image acquisition, synchronously monitoring and obtaining the user's breathing frequency, heart rate, and cough intensity through a sensor, which are set as user physiological characteristics; within the same time zone of airway image acquisition, synchronously monitoring and obtaining the motion characteristics of the airway lens, which are set as lens operation characteristics, where the lens operation characteristics include lens acceleration, lens rotation rate, and lens rotation angle.

[0010] Preferably, the image segmentation method for bronchoscopy examination assistance further includes: according to historical airway examination records, collecting a sample user physiological feature set, a sample lens operation feature set, and a sample trachea image set, and analyzing to obtain a sample image blurriness set; using the sample user physiological feature set, the sample lens operation feature set, and the sample image blurriness set as training data, and performing P-fold cross-validation to construct P training sets, where P is an integer greater than or equal to 20; using the P training sets to train a feedforward neural network respectively until the network converges, obtaining P image blurriness prediction branches, and integrally constructing an image blurriness prediction plugin; inputting the user physiological features and lens operation features into the image blurriness prediction plugin to perform image blurriness prediction and generate an image segmentation compensation coefficient.

[0011] Preferably, the image segmentation method for bronchoscopy examination assistance further includes: randomly selecting a first sample user physiological feature, a first sample lens operation feature, and a first sample trachea image; obtaining a first standard trachea image at the corresponding position under the first sample user physiological feature and the first sample lens operation feature; taking the first standard trachea image as a reference, performing a blurriness comparison on the first sample trachea image, and obtaining an initial image blurriness by calculating the gradient magnitude of the image, where the initial image blurriness is greater than or equal to 0 and less than 100; setting the ratio of the initial image blurriness to 100 as the first sample image blurriness and adding it to the sample image blurriness set.

[0012] Preferably, the image segmentation method for bronchoscopy examination assistance further includes: respectively performing feature change impact analysis according to the user physiological features and lens operation features, outputting a physiological change coefficient and an operation change coefficient, and summing them to obtain a comprehensive change coefficient, where the feature change impact is the impact of feature changes on the acquisition quality of airway images; multiplying the ratio of the obtained comprehensive change coefficient to the historical maximum comprehensive change coefficient by P and taking the integer to obtain the branch selection quantity K; randomly selecting K image blurriness prediction branches from the P image blurriness prediction branches of the image blurriness prediction plugin to perform image blurriness prediction on the user physiological features and lens operation features, obtaining K predicted image blurriness values, and calculating the mean value to obtain the image segmentation compensation coefficient.

[0013] Preferably, the image segmentation method for bronchoscopy examination assistance further includes: adding 1 to the image segmentation compensation coefficient and summing to obtain a granularity correction weight; adjusting the initial image segmentation granularity scheme according to the granularity correction weight to obtain a corrected image segmentation granularity scheme, where the corrected image segmentation granularity is the product of the initial image segmentation granularity and the granularity correction weight.

[0014] The beneficial effects of the present invention are as follows: By analyzing the significant features of the airway image, multiple airway significant regions are determined; then, according to the multiple image saliencies of the multiple airway significant regions, an initial image segmentation granularity scheme is configured; on the other hand, the user's physiological characteristics and lens operation characteristics during the acquisition process of the airway image are synchronously monitored; further, according to the user's physiological characteristics and lens operation characteristics, image blur prediction is performed to generate an image segmentation compensation coefficient; then, according to the image segmentation compensation coefficient, the initial image segmentation granularity scheme is adjusted to obtain a corrected image segmentation granularity scheme; finally, according to the corrected image segmentation granularity scheme, image segmentation is performed on the multiple airway significant regions. That is to say, by flexibly adjusting the image segmentation granularity according to the different characteristics of the lesion area and the change of image quality, the segmentation accuracy and accuracy of the airway image can be significantly improved, efficient and accurate image segmentation can be achieved, so as to better identify different types of lesion areas and effectively reduce false detection and missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flowchart of an image segmentation method for bronchoscopy examination assistance provided by the present invention; Figure 2 It is a schematic flowchart of determining multiple airway significant regions in an image segmentation method for bronchoscopy examination assistance provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0017] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0018] In the description of the present invention, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use the present invention. In the following description, details are set forth for purposes of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without the use of these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but rather to be in line with the broadest scope consistent with the principles and features disclosed in the present invention.

[0019] Embodiment, such as Figure 1 As shown, an embodiment of the present invention provides an image segmentation method for bronchoscopy assistance, specifically including the following steps: S10: Collect airway images of the lesion area through a bronchoscope, perform significant feature analysis on the airway images, and determine multiple airway significant areas.

[0020] Furthermore, as Figure 2 shown, step S10 of the present invention further includes: S11: Construct a significant feature analyzer based on a Gaussian pyramid, where the significant feature analyzer includes a brightness extraction unit, a color extraction unit, and an orientation extraction unit; S12: Perform multi-scale Gaussian downsampling on the airway images through the brightness extraction unit, the color extraction unit, and the orientation extraction unit to obtain multi-scale airway images; S13: Based on the center-surround difference calculation principle, perform deviation analysis on the multi-scale airway images respectively to obtain an airway brightness feature map, an airway color feature map, and an airway orientation feature map, and perform weighted superposition processing to fuse and generate an airway significant image; S14: Screen and segment the airway significant image according to a significant feature threshold, set the area with a significance greater than the significant feature threshold as an airway significant area, and obtain multiple airway significant areas, where each airway significant area is marked with an image significance.

[0021] Specifically, a bronchoscope enters the trachea and bronchi of a patient through a slender fiber optic lens to directly observe the lesions in the airway. During this process, a doctor can obtain airway images in real time and identify potential lesion areas, such as tumors, inflammations, bronchiectasis, foreign bodies, etc.; the lens is gently inserted into the patient's throat through the bronchoscope and gradually advanced along the airway to collect images of the trachea and bronchi in real time, and obtain airway images of the lesion area, and these lesions include masses, inflammations, infections, foreign bodies, tissue damage, etc.

[0022] The Gaussian pyramid is a multi-scale image processing method. By performing blurring and downsampling at different levels on the image, it can obtain the features of the image at different scales, enabling more detailed analysis of its content at each level when processing the image. First, a significant feature analyzer is constructed based on the Gaussian pyramid. The significant feature analyzer includes a brightness extraction unit, a color extraction unit, and an orientation extraction unit. The brightness extraction unit mainly focuses on the brightness information of the image (i.e., grayscale values). Usually, the brightness feature is an important indicator for describing the changes in bright and dark regions in the image. The lesion area in the bronchoscope image usually has a significant brightness difference from the normal tissue (for example, a tumor may be darker or brighter than the surrounding tissue). This module extracts the brightness feature on each layer of the Gaussian pyramid and generates a brightness feature map. The color extraction unit identifies the significant features in the bronchoscope image by analyzing the color information of the image. Color can provide visual perception information of the image, which helps to distinguish the normal airway from the lesion area. This unit mainly extracts the color features of the image, especially the saturation and contrast of the color, which can help to identify certain specific lesion areas (such as the colors of some types of tumors or inflammations are different from those of the normal airway). The orientation extraction unit analyzes the directional features of local regions in the image, mainly used to identify edge and texture features. In the bronchoscope image, the edges of the lesion area are usually more obvious and have directional features than the normal tissue, such as the edge of a tumor or the shape of a foreign object. Orientation extraction can help to effectively detect these edges and enhance the saliency of the relevant regions in the image. Through the multi-scale processing of the Gaussian pyramid and the comprehensive analysis of brightness, color, and orientation, the significant feature analyzer can effectively identify the important feature regions in the bronchoscope image.

[0023] Next, through the brightness extraction unit, the color extraction unit, and the orientation extraction unit, multi-scale Gaussian downsampling is performed on the airway image. Through Gaussian downsampling, the image is processed at different scales. Each layer of the Gaussian pyramid represents a different image resolution, and the image resolution is gradually reduced, enabling the image to show different features at multiple scales. By applying Gaussian blurring and downsampling operations to the airway image, images at multiple scales are generated. The resolution of each layer of the image is gradually reduced, which helps to capture features at different scales, such as small lesions and larger lesion areas. Multi-scale airway images are obtained.

[0024] Then, based on the center-surround difference calculation principle, deviation analysis is performed on the multi-scale airway images respectively. By calculating the difference between the central region and the surrounding region of the image, those regions with significant differences are identified. The central region usually contains the target or lesion, while the surrounding region is the background or normal tissue. By analyzing the difference between the two, the significant regions in the image can be highlighted. Among them, by comparing the brightness difference between the central and surrounding regions of the image, the regions with significant brightness changes are highlighted to generate a brightness feature map. In different color spaces (such as HSV, Lab, etc.), the color difference of the airway image is analyzed, the color difference between the central region and the surrounding region is calculated, and a color feature map is generated. The edge detection algorithm (such as Sobel operator, Canny edge detection) is used to process the image, and the edge information in different directions of the airway image is extracted to generate a direction feature map. Further, weighted superposition processing is performed on the airway brightness feature map, airway color feature map, and airway direction feature map. That is, for each feature map (brightness, color, direction), a weight is assigned. Different features have different contribution degrees to the significant region, so the setting of the weight will affect the generation of the final significant image. Generally speaking, the brightness feature and color feature have a greater impact on most lesion regions, while the direction feature helps to highlight the boundary of the lesion. Through weighted superposition, the brightness, color, and direction information are integrated together to obtain a significant image that can better highlight the lesion region, that is, the airway significant image. Through multi-scale downsampling and center-surround difference analysis, lesion information at different scales can be captured, especially small lesions and edge parts. The weighted superposition processing can fuse different features, effectively highlight the significant regions in the airway image, and improve the accuracy of image segmentation and analysis.

[0025] Configure a significant feature threshold, which is a key parameter. Through it, it can be controlled which regions are considered lesion regions and which regions are considered background regions. Regions with a significance greater than the threshold represent regions with strong feature differences, usually lesion regions, while regions with a significance less than the threshold may be normal or background regions. Among them, the significant feature threshold can be automatically calculated based on the statistical features of the image (such as mean, standard deviation, etc.) or a constant value can be selected according to experience. Then, according to the significant feature threshold, the airway significant image is screened and segmented, that is, the threshold comparison is performed on each pixel value in the airway significant image. If the significance of the pixel (the weighted result of the brightness, color, or direction feature shown in the image) is greater than the set threshold, then the pixel belongs to the significant region. Through threshold comparison, the significant regions in the airway image are screened out. The significant regions usually correspond to the parts with large feature differences in the airway image, such as lesion tissues, lesions, or abnormal structures. Through threshold screening, other background or irrelevant regions will be ignored, and multiple airway significant regions are obtained.

[0026] Next, perform connected component analysis on the salient regions in the image, connect adjacent salient regions together to form a complete region, each salient region will be marked and assigned a unique identifier, and each salient region is extracted separately and associated with its saliency value. Among them, the image saliency of the airway salient region is the average of the saliency values of multiple pixels within the region. In this way, each final airway salient region has a clear identifier and saliency value. By screening and segmenting the airway salient image according to the salient feature threshold, the lesion region in the airway image can be effectively extracted, and the saliency of each salient region can be accurately identified.

[0027] S20: Configure an initial image segmentation granularity scheme according to the multiple image saliencies of the multiple airway salient regions.

[0028] Furthermore, step S20 of the present invention further includes: S21: Obtain the position information of the multiple airway salient regions within the airway image, map it to two-dimensional coordinate points, and construct a two-dimensional distribution matrix; S22: Perform distribution discretization analysis on the two-dimensional distribution matrix, and set the reciprocal of the distribution discreteness as the significant distribution concentration.

[0029] Specifically, obtain the position information of the multiple airway salient regions within the airway image. The position information of the salient region refers to the central position of each salient region in the airway image. After image segmentation, the salient regions have been extracted and marked. Through connected component analysis or edge detection algorithms (such as Canny edge detection), the boundary pixels of each salient region can be determined. The position information of each salient region can be obtained by calculating its geometric center, which is the average coordinate (i.e., the centroid) of all pixel points within the region, representing the center of the region. For each salient region in the image, repeat the above steps to calculate and obtain the geometric center coordinates of each salient region, and each salient region will have a corresponding two-dimensional coordinate point. Then, map these two-dimensional coordinate points to a two-dimensional coordinate system to construct a two-dimensional distribution matrix. The two-dimensional distribution matrix is a matrix representing the distribution of salient regions, which represents the coordinate points in the image in two-dimensional space, and each element of the matrix corresponds to the distribution of a region in the image. The two-dimensional distribution matrix can reflect the spatial distribution of salient regions in the airway image. By analyzing the matrix, the aggregation degree of salient regions can be identified, which helps to further judge the type or location of the lesion.

[0030] Then, perform distribution discretization analysis on the two-dimensional distribution matrix. The distribution dispersion is used to measure whether the distribution of significant regions in the airway image is uniform. The larger the distribution dispersion, the more dispersed the significant regions are. Conversely, the more concentrated the distribution is. Among them, the distribution dispersion is a measure of the difference between each element and the mean value, and is obtained by calculating the sum of the squared differences between each element in the two-dimensional distribution matrix and the mean value, thereby obtaining the distribution dispersion; further, the reciprocal of the distribution dispersion is set as the significant distribution concentration degree. The significant distribution concentration degree reflects the aggregation degree of significant regions in the airway image. When the distribution dispersion is small (i.e., the significant regions are more concentrated), the significant distribution concentration degree is large, indicating that the significant regions are more concentrated in the image; conversely, when the distribution dispersion is large, the significant distribution concentration degree is small, indicating that the significant regions are more dispersed. The significant distribution concentration degree provides an effective index for measuring the concentration degree of significant regions, helps to optimize the image segmentation strategy, and improves the accuracy and precision of image segmentation.

[0031] S23: Configure an initial image segmentation granularity scheme according to the significant distribution concentration degree and the multiple image saliencies.

[0032] Furthermore, step S23 of the present invention further includes: S231: If the significant distribution concentration degree is greater than or equal to a predetermined reference value, select the maximum image saliency among the multiple image saliencies, and input the maximum image saliency into the saliency-segmentation granularity comparison library, and output the first image segmentation granularity, which is set as the initial image segmentation granularity scheme; S232: If the significant distribution concentration degree is less than the predetermined reference value, input the multiple image saliencies into the saliency-segmentation granularity comparison library respectively, and output multiple image segmentation granularities, which are set as the initial image segmentation granularity scheme.

[0033] Specifically, compare the calculated significant distribution concentration degree with the predetermined reference value to determine whether to use a globally unified segmentation granularity or dynamically adjust the segmentation granularity according to local region characteristics. The predetermined reference value is a threshold set according to the actual requirements and segmentation strategy of the airway image, and is used to judge the distribution characteristics of significant regions in the image, and can be set according to the actual scenario. If the significant regions are more concentrated, it means that the diseased regions or feature regions in the image are roughly concentrated in a certain area and the distribution is relatively uniform. At this time, a globally unified segmentation granularity can be selected, that is, a unified image segmentation granularity is used to process the entire image. If the significant regions are more dispersed, it means that the diseased regions or feature regions in the image are more dispersed and may have different characteristics. At this time, the segmentation granularity needs to be dynamically adjusted according to the characteristics of different significant regions, and a local image segmentation granularity strategy is adopted to meet the needs of different diseased regions.

[0034] Judge the significant distribution concentration degree according to a predetermined reference value. When the significant distribution concentration degree is greater than or equal to the predetermined reference value, it indicates that the significant regions in the image are relatively concentrated, and it is suitable to use a globally unified image segmentation granularity. Then, from multiple airway significant regions, select the region with the largest image saliency. The saliency values of these significant regions represent the significant degree of lesions or important features. Then, input the selected maximum image saliency into the saliency-segmentation granularity comparison library, and output the corresponding first image segmentation granularity according to this saliency value. Since the significant regions are relatively concentrated and a globally unified segmentation granularity is adopted, the output segmentation granularity is the initial image segmentation granularity scheme for subsequent image segmentation processing. Among them, the saliency-segmentation granularity comparison library is a database or lookup table containing the mapping relationship between saliency and the corresponding segmentation granularity. During the image segmentation process, by looking up this comparison library, the system can automatically select an appropriate segmentation granularity according to the saliency value of the image, thereby optimizing the segmentation process, improving the accuracy and efficiency of segmentation, and can be set after analyzing historical data. For example, high saliency > 0.8, matching fine granularity (higher resolution); medium saliency (0.5 to 0.8), matching medium granularity (moderate resolution), etc.

[0035] When the significant distribution concentration degree is less than the predetermined reference value, it means that the significant regions in the image are relatively dispersed, and it may be necessary to dynamically adjust the image segmentation granularity according to the characteristics of each region. Then, for each significant region, input its image saliency into the saliency-segmentation granularity comparison library respectively, and output the corresponding segmentation granularity according to the image saliency of each significant region. Since the significant regions are dispersed, different image segmentation granularities can be output for each region respectively, and each region will obtain a suitable segmentation granularity according to its saliency and characteristics, obtaining multiple image segmentation granularities, which are set as the initial image segmentation granularity scheme. This method of flexibly adjusting the segmentation granularity can effectively adapt to the needs of different images. When the significant regions are relatively concentrated, select a globally unified image segmentation granularity to improve the simplicity and integrity of segmentation; when the significant regions are dispersed, dynamically adjust the segmentation granularity according to the saliency of each region to enhance the fineness and pertinence of segmentation; thereby improving the accuracy of segmentation.

[0036] S30: Synchronously monitor and obtain the user's physiological characteristics and lens operation characteristics during the airway image acquisition process, predict the image blur degree, and generate an image segmentation compensation coefficient.

[0037] Furthermore, step S30 of the present invention further includes: S31: During the same time period of airway image acquisition, synchronously monitor and obtain the user's respiratory rate, heart rate, and cough intensity through sensors, and set them as the user's physiological characteristics; S32: During the same time period of airway image acquisition, synchronously monitor and obtain the motion characteristics of the airway lens, and set them as the lens operation characteristics, where the lens operation characteristics include lens acceleration, lens rotation rate, and lens rotation angle.

[0038] Specifically, during the process of airway image acquisition, by synchronously monitoring the user's physiological characteristics and the operation characteristics of the lens in real time, the image quality can be effectively optimized. Especially in a dynamic or blurred environment, it provides valuable information for image segmentation and subsequent analysis. First, during the same time period of airway image acquisition, synchronously monitor and obtain the user's respiratory rate, heart rate, and cough intensity through sensors. Among them, monitoring the user's respiratory rate helps to understand the user's respiratory state. The fluctuations in breathing may cause blurring of the bronchoscope image during the dynamic process. Therefore, the respiratory rate can be used as an indicator to evaluate the clarity and stability of the image; by measuring the user's heart rate, their physiological state can be reflected. A high or low heart rate may affect the user's operation stability, thus affecting the acquisition quality of the image; coughing may cause vibrations or blurring of the image. Monitoring the cough intensity can make compensation during image processing, thereby reducing the degradation of image quality caused by the user's physiological reactions, and setting the respiratory rate, heart rate, and cough intensity as the user's physiological characteristics.

[0039] On the other hand, during the same time period of airway image acquisition, synchronously monitor and obtain the motion characteristics of the airway lens, including lens acceleration, lens rotation rate, and lens rotation angle. Among them, the lens acceleration reflects the change rate of the lens movement. Excessive or violent movement may cause image blurring. Especially when the lens suddenly moves or stops, it may cause image distortion; the rotation rate of the lens directly affects the image stability. A higher rotation rate may cause image jitter or distortion, affecting subsequent image processing and analysis; the rotation angle of the lens helps to judge the perspective and focus of the image. If the lens angle changes too sharply, it may cause a decrease in the image quality of the local area; and set the lens acceleration, lens rotation rate, and lens rotation angle as the lens operation characteristics. By synchronously monitoring the user's physiological characteristics and lens operation characteristics in real time, the quality of the airway image can be more accurately evaluated, and then the image processing strategy can be dynamically adjusted according to these factors to improve the segmentation accuracy of the image.

[0040] Furthermore, step S30 of the present invention further includes: S33: According to the historical airway examination records, collect the sample user physiological characteristic set, the sample lens operation characteristic set, and the sample trachea image set, and analyze and obtain the sample image blurriness set.

[0041] Furthermore, step S33 of the present invention further includes: S331: Randomly select the physiological characteristics of the first sample user, the first sample lens operation characteristics, and the first sample trachea image; S332: Obtain the first standard trachea image at the corresponding position under the physiological characteristics of the first sample user and the first sample lens operation characteristics; S333: Based on the first standard trachea image, perform a fuzzy comparison on the first sample trachea image, and obtain the initial image blur by calculating the gradient magnitude of the image, where the initial image blur is greater than or equal to 0 and less than 100; S334: Set the ratio of the initial image blur to 100 as the first sample image blur and add it to the sample image blur set.

[0042] Specifically, according to the historical airway examination records, collect the physiological characteristic set of the sample user, the sample lens operation characteristic set, and the sample trachea image set. The physiological characteristics of the user include breathing, heart rate, and coughing. For example, the breathing rate refers to the number of breaths per minute, which reflects the patient's breathing state. Frequent breathing changes may affect the stability of the image; the heart rate refers to the number of heartbeats per minute, and the patient's heart rate may be affected by tension or other factors, thereby affecting the clarity of the bronchoscope image; coughing will cause image blurring and displacement; the lens operation characteristics include the operation of the bronchoscope itself, which will affect the image quality, especially when the movement of the lens is unstable. These lens operation characteristics can help judge the stability of the lens during the examination and provide a reference for image optimization.

[0043] Next, randomly select the physiological characteristics of the first sample user, the first sample lens operation characteristics, and the first sample trachea image. The first sample is any one in the sample dataset; then obtain the first standard trachea image at the corresponding position under the physiological characteristics of the first sample user and the first sample lens operation characteristics. This image should represent a clear trachea image taken under normal physiological conditions and stable lens operation, serving as a benchmark for subsequent comparison and blur analysis. Further, based on the first standard trachea image, perform a fuzzy comparison on the first sample trachea image, that is, compare the first standard trachea image and the first sample trachea image, and determine the clarity of the image by calculating the gradient magnitude. The gradient magnitude can be used to measure the change in image details. The gradient magnitude of a blurred image is lower, while that of a clear image is higher. First, calculate the gradient of each pixel point in the image (such as using the Sobel operator or other edge detection algorithms); then, calculate the gradient magnitude to obtain the edge information of the image. Less edge information indicates that the image is more blurred, and vice versa indicates that the image is more clear; evaluate the blur by calculating the difference in the gradient magnitude of the image. According to the result calculated from the gradient magnitude of the image, generate the initial image blur. The value of the initial image blur should be between 0 and 100, where a lower value indicates that the image is more clear, and a higher value indicates that the image is blurred.

[0044] Then, calculate the ratio of the initial image blur degree to 100 to obtain the standardized image blur degree, which is set as the first sample image blur degree and added to the sample image blur degree set as part of subsequent data analysis. By using the image gradient magnitude as the calculation basis and combining the comparison between the standard trachea image and the actual trachea image, the blur degree of the trachea image can be accurately evaluated. These blur degree data will be used to optimize the image segmentation granularity and dynamically adjust the segmentation strategy according to the clarity of the image, ensuring that a finer segmentation granularity is used when the image is blurred to avoid false detection or missed detection caused by image blur.

[0045] S34: Use the sample user physiological feature set, the sample lens operation feature set, and the sample image blur degree set as training data, and perform P-fold cross-validation to construct P training sets, where P is an integer greater than or equal to 20; S35: Use the P training sets to train a feedforward neural network respectively until the network converges to obtain P image blur prediction branches, and integrally construct an image blur prediction plugin.

[0046] Specifically, use the sample user physiological feature set, the sample lens operation feature set, and the sample image blur degree set as training data, and perform P-fold cross-validation, that is, divide the training data into P equal parts to obtain P data sets, where P is an integer greater than or equal to 20; then, select P times with replacement from the P data sets to construct the first training set; use the same method to select P times iteratively to obtain P training sets.

[0047] Next, use the P training sets to train a feedforward neural network respectively. The structure of the feedforward neural network includes an input layer (receiving data from user physiological features, lens operation features, and image blur degree), a hidden layer (performing feature learning and pattern recognition through multiple hidden layers), and an output layer (outputting the predicted image blur degree). Then, use the sample user physiological features and the sample lens operation features as inputs and the sample image blur degree as supervision to train the feedforward neural network. During the training process, use the backpropagation algorithm for training to adjust the network weights until the output of the network is close to the true blur degree value; during the training process, a loss function (such as mean squared error) will be used to measure the difference between the predicted value and the actual value, optimize the parameters of the network until the network converges; through multiple rounds of iteration until the loss value of the neural network is stable and low enough to meet the preset convergence standard, indicating that the model has been trained; obtain P image blur prediction branches, and integrally construct an image blur prediction plugin according to the P image blur prediction branches. This plugin can dynamically predict the blur degree of the image according to factors such as user physiological features and lens operation features, so as to provide a more accurate blur degree reference for image segmentation and diagnosis.

[0048] S36: Input the user's physiological characteristics and lens operation characteristics into the image blur prediction plug-in to perform image blur prediction and generate an image segmentation compensation coefficient.

[0049] Furthermore, step S36 of the present invention further includes: S361: Analyze the influence of feature changes based on the user's physiological characteristics and lens operation characteristics respectively, output a physiological change coefficient and an operation change coefficient, and sum them to obtain a comprehensive change coefficient, where the feature change influence is the influence of feature changes on the acquisition quality of airway images; S362: Multiply the ratio of the obtained comprehensive change coefficient to the historical maximum comprehensive change coefficient by P and round down to obtain the number of selected branches K; S363: Randomly select K image blur prediction branches from the P image blur prediction branches of the image blur prediction plug-in, perform image blur prediction on the user's physiological characteristics and lens operation characteristics to obtain K predicted image blur degrees, and calculate the mean value to obtain the image segmentation compensation coefficient.

[0050] Specifically, first, analyze the influence of feature changes based on the user's physiological characteristics. Among them, the user's physiological characteristics (such as respiratory rate, heart rate, cough intensity, etc.) will affect the acquisition quality of airway images. By analyzing the changes in these physiological characteristics, estimate the influence of physiological changes on image quality; that is, set an influence weight for each physiological characteristic (respiratory rate, heart rate, cough intensity), and analyze based on the influence of these characteristics on image quality to obtain the comprehensive physiological characteristic influence, denoted as the physiological change coefficient. On the other hand, analyze the influence of feature changes based on the lens operation characteristics, set weights for each operation characteristic (acceleration, rotation rate, rotation angle), and analyze based on the influence of these characteristics on image quality to obtain the comprehensive operation characteristic influence, denoted as the operation change coefficient; and add the physiological change coefficient and the operation change coefficient, and use the sum of the two as the comprehensive change coefficient to represent the comprehensive influence of overall feature changes on the acquisition quality of airway images.

[0051] Next, obtain the historical maximum comprehensive change coefficient. The historical maximum comprehensive change coefficient refers to the maximum value among all the comprehensive change coefficients recorded after multiple image acquisitions and analyses in historical data. This value represents the maximum degree of change observed historically and can be used as a standard to compare the changes during the current image acquisition process. Then, round the product of the ratio of the obtained comprehensive change coefficient to the historical maximum comprehensive change coefficient and P to obtain the number of selected branches K. Further, randomly select K image blur prediction branches from the P image blur prediction branches of the image blur prediction plug-in, and perform image blur prediction on the user's physiological characteristics and lens operation characteristics. Each branch predicts the image blur based on the user's physiological characteristics (such as respiratory rate, heart rate, cough intensity) and lens operation characteristics (such as lens acceleration, lens rotation rate, lens rotation angle) to obtain K predicted image blur values. Finally, calculate the mean of the K predicted image blur values to obtain the image segmentation compensation coefficient.

[0052] By dynamically adjusting the number of branches used for image blur prediction according to the influence degree of image acquisition, it is possible to select the most suitable prediction model or branch for processing in different image acquisition situations. This can ensure that in cases of image acquisition with greater influence, more branches are used for more refined blur prediction, while in cases of less influence, redundant calculations are avoided. Thus, under the premise of ensuring the accuracy of image blur prediction, unnecessary waste of computing power resources can be effectively reduced, and the accuracy and efficiency of blur prediction can be improved.

[0053] S40: Adjust the initial image segmentation granularity scheme according to the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme, and perform image segmentation on the multiple airway significant regions.

[0054] Furthermore, step S40 of the present invention further includes: S41: Add 1 to the image segmentation compensation coefficient and sum to obtain the granularity correction weight; S42: Adjust the initial image segmentation granularity scheme according to the granularity correction weight to obtain a corrected image segmentation granularity scheme, where the corrected image segmentation granularity is the product of the initial image segmentation granularity and the granularity correction weight.

[0055] Specifically, 1 is added to the image segmentation compensation coefficient, and the sum of the two is used as the granularity correction weight, which represents the degree of adjustment of image segmentation. This weight determines how much adjustment needs to be made to the initial image segmentation granularity scheme. Then, according to the granularity correction weight, the initial image segmentation granularity scheme is adjusted. The granularity correction weight is the product of the initial segmentation granularity and the compensation coefficient. Through this process, the segmentation granularity can be adjusted according to the actual situation of the image (such as the degree of blurriness, the size of the significant region, etc.). After adjustment, the segmentation granularity is more in line with the actual needs of the image, thereby improving the accuracy and precision of segmentation. Finally, through the corrected image segmentation granularity scheme, image segmentation is performed on multiple airway significant regions. Among them, the segmentation details of each significant region are optimized according to its significance and the adjusted granularity. This means that for some smaller lesion regions, the segmentation granularity will be more detailed; while for larger lesion regions, the segmentation granularity will be relatively coarser. By introducing the image segmentation compensation coefficient, the segmentation granularity can be adjusted according to the quality and characteristics of the actual image. For blurred or difficult-to-identify regions, the system automatically increases the segmentation granularity to avoid mis-segmentation; for clear regions, the granularity is reduced to improve efficiency.

[0056] An image segmentation method for bronchoscopy examination assistance provided by an embodiment of the present invention has at least the following technical effects: By performing significant feature analysis on the airway image to determine multiple airway significant regions; then configuring an initial image segmentation granularity scheme according to the multiple image significances of the multiple airway significant regions; on the other hand, synchronously monitoring and obtaining the user's physiological characteristics and lens operation characteristics during the airway image acquisition process; further predicting the image blurriness according to the user's physiological characteristics and lens operation characteristics to generate an image segmentation compensation coefficient; then adjusting the initial image segmentation granularity scheme according to the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme; finally, performing image segmentation on the multiple airway significant regions according to the corrected image segmentation granularity scheme. That is to say, by flexibly adjusting the image segmentation granularity according to the different characteristics of the lesion region and the change of image quality, the segmentation accuracy and accuracy of the airway image can be significantly improved, efficient and accurate image segmentation can be achieved, so as to better identify different types of lesion regions and effectively reduce false positives and missed detections.

[0057] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concept.

[0058] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An image segmentation method for bronchoscopy assistance, characterized in that, The method includes: Collecting airway images of the lesion area through a bronchoscope, analyzing the significant features of the airway images, and determining multiple airway significant regions; Configuring an initial image segmentation granularity scheme according to the multiple image saliencies of the multiple airway significant regions; Synchronously monitoring and obtaining the user's physiological characteristics and lens operation characteristics during the acquisition process of the airway images, predicting the image blurriness, and generating an image segmentation compensation coefficient; Adjusting the initial image segmentation granularity scheme according to the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme, and performing image segmentation on the multiple airway significant regions.

2. The image segmentation method for bronchoscopy examination assistance according to claim 1, wherein Analyzing the significant features of the airway images to determine multiple airway significant regions, including: Constructing a significant feature analyzer based on a Gaussian pyramid, where the significant feature analyzer includes a brightness extraction unit, a color extraction unit, and an orientation extraction unit; Performing multi-scale Gaussian downsampling on the airway images through the brightness extraction unit, the color extraction unit, and the orientation extraction unit to obtain multi-scale airway images; Based on the center-surround difference calculation principle, performing deviation analysis on the multi-scale airway images respectively to obtain an airway brightness feature map, an airway color feature map, and an airway orientation feature map, and performing weighted superposition processing to fuse and generate an airway significant image; Screening and segmenting the airway significant image according to a significant feature threshold, setting the regions with saliency greater than the significant feature threshold as airway significant regions, and obtaining multiple airway significant regions, where each airway significant region is marked with an image saliency.

3. The image segmentation method for bronchoscopy examination assistance according to claim 2, characterized in that, Configuring an initial image segmentation granularity scheme according to the multiple image saliencies of the multiple airway significant regions, including: Obtaining the position information of the multiple airway significant regions in the airway image, mapping it to two-dimensional coordinate points, and constructing a two-dimensional distribution matrix; Performing distribution discretization analysis on the two-dimensional distribution matrix, and setting the reciprocal of the distribution discreteness as the significant distribution concentration; Configuring an initial image segmentation granularity scheme according to the significant distribution concentration and the multiple image saliencies.

4. An image segmentation method for bronchoscopy examination assistance according to claim 3, characterized in that, Configuring an initial image segmentation granularity scheme according to the significant distribution concentration and the multiple image saliencies, including: If the significant distribution concentration is greater than or equal to a predetermined reference value, selecting the maximum image saliency among the multiple image saliencies, inputting the maximum image saliency into a saliency-segmentation granularity comparison library, outputting a first image segmentation granularity, and setting it as the initial image segmentation granularity scheme; If the significant distribution concentration is less than the predetermined reference value, inputting the multiple image saliencies into the saliency-segmentation granularity comparison library respectively, outputting multiple image segmentation granularities, and setting them as the initial image segmentation granularity scheme.

5. A method for image segmentation for assisting bronchoscopy according to claim 1, wherein Synchronously monitoring and obtaining the user's physiological characteristics and lens operation characteristics during the acquisition process of the airway images, including: During the same time zone of airway image acquisition, synchronously monitoring and obtaining the user's breathing frequency, heart rate, and cough intensity through a sensor, and setting them as the user's physiological characteristics; During the same time zone of airway image acquisition, synchronously monitoring and obtaining the motion characteristics of the airway lens, and setting them as the lens operation characteristics, where the lens operation characteristics include lens acceleration, lens rotation rate, and lens rotation angle.

6. A method for image segmentation for assisting bronchoscopy according to claim 5, characterized in that, Perform image blurriness prediction and generate an image segmentation compensation coefficient, including: Collect a set of sample user physiological characteristics, a set of sample lens operation characteristics, and a set of sample trachea images based on historical airway examination records, and analyze to obtain a set of sample image blurriness; Use the set of sample user physiological characteristics, the set of sample lens operation characteristics, and the set of sample image blurriness as training data, and perform P-fold cross-validation to construct P training sets, where P is an integer greater than or equal to 20; Use the P training sets to train a feedforward neural network respectively until the network converges, obtain P image blurriness prediction branches, and integrally construct an image blurriness prediction plugin; Input the user physiological characteristics and lens operation characteristics into the image blurriness prediction plugin to perform image blurriness prediction and generate an image segmentation compensation coefficient.

7. A method for image segmentation for assisting bronchoscopy according to claim 6, characterized in that, Analyze to obtain a set of sample image blurriness, including: Randomly select the first sample user physiological characteristics, the first sample lens operation characteristics, and the first sample trachea image; Obtain the first standard trachea image at the corresponding position under the first sample user physiological characteristics and the first sample lens operation characteristics; Taking the first standard trachea image as a reference, perform blurriness comparison on the first sample trachea image, and obtain the initial image blurriness by calculating the gradient magnitude of the image, where the initial image blurriness is greater than or equal to 0 and less than 100; Set the ratio of the initial image blurriness to 100 as the first sample image blurriness and add it to the set of sample image blurriness.

8. A method for image segmentation for assisting bronchoscopy according to claim 6, characterized in that, Input the user physiological characteristics and lens operation characteristics into the image blurriness prediction plugin to perform image blurriness prediction and generate an image segmentation compensation coefficient, including: Perform feature change impact analysis according to the user physiological characteristics and lens operation characteristics respectively, output a physiological change coefficient and an operation change coefficient, and sum them to obtain a comprehensive change coefficient, where the feature change impact is the impact of feature changes on the quality of airway image acquisition; Multiply the ratio of the obtained comprehensive change coefficient to the historical maximum comprehensive change coefficient by P and round down to obtain the number of branches selected K; Randomly select K image blurriness prediction branches from the P image blurriness prediction branches of the image blurriness prediction plugin, perform image blurriness prediction on the user physiological characteristics and lens operation characteristics, obtain K predicted image blurriness values, and calculate the mean value to obtain the image segmentation compensation coefficient.

9. A method for image segmentation for assisting bronchoscopy according to claim 8, characterized in that Adjust the initial image segmentation granularity scheme according to the image segmentation compensation coefficient, including: Use 1 plus the image segmentation compensation coefficient and sum to obtain a granularity correction weight; Adjust the initial image segmentation granularity scheme according to the granularity correction weight to obtain a corrected image segmentation granularity scheme, where the corrected image segmentation granularity is the product of the initial image segmentation granularity and the granularity correction weight.

Images