An image segmentation method for assisting bronchoscopy

Abstract

The present invention relates to an image segmentation method for assisting bronchoscopy, and relates to the field of medical image processing, including: performing significant feature analysis on airway images to determine multiple significant airway regions; configuring an initial image segmentation granularity scheme based on multiple image saliencies; synchronously monitoring and acquiring user physiological characteristics and lens operation characteristics, performing image blur prediction, and generating image segmentation compensation coefficients; adjusting the initial image segmentation granularity scheme to obtain a corrected image segmentation granularity scheme, and performing image segmentation on multiple significant airway regions. The present invention can solve the problem in traditional methods that the segmentation strategy cannot be dynamically adjusted according to image features, resulting in difficulty in adapting to the needs of different lesion regions and easily causing false detection or missed detection; it can significantly improve the segmentation precision and accuracy of tracheal images, achieve efficient and accurate image segmentation, thereby better identifying different types of lesion regions and effectively reducing false detection and missed detection.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to an image segmentation method for assisting bronchoscopic examination. Background Art

[0002] The quality of tracheal images is often affected by multiple factors, such as the patient's physiological reactions (such as coughing, breathing, etc.), unstable lens operation, equipment cleanliness, ambient light, etc., which may make the edges of the lesion area in the tracheal image blurred or interfered with by noise, posing challenges to subsequent image analysis and lesion identification.

[0003] Tracheal image segmentation is a key technology in bronchoscopic image analysis. It involves extracting regions of interest (such as lesions) from the background for further analysis. However, traditional tracheal image segmentation methods often use fixed segmentation granularity and strategies, making it difficult to dynamically adjust the granularity based on specific image features. This results in inaccurate and unreliable segmentation results, making it difficult to adapt to the needs of different lesion areas and prone to false or missed detections. Summary of the Invention

[0004] The present invention aims to solve the technical problem that traditional airway image segmentation methods cannot dynamically adjust the segmentation strategy according to image features, resulting in difficulty in adapting to the needs of different lesion areas and easily causing false detection or missed detection. An image segmentation method for assisting bronchoscopy examination is provided to solve the problem.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: The present invention provides an image segmentation method for assisting bronchoscopic examination, comprising: collecting airway images of the lesion area through a bronchoscope, performing significant feature analysis on the airway image, and determining multiple airway significant areas; configuring an initial image segmentation granularity scheme based on multiple image saliencies of the multiple airway significant areas; synchronously monitoring and obtaining the user's physiological characteristics and lens operation characteristics during the airway image collection process, performing image blur prediction, and generating an image segmentation compensation coefficient; adjusting the initial image segmentation granularity scheme based on the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme, and performing image segmentation on the multiple airway significant areas.

[0006] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: constructing a significant feature analyzer based on a Gaussian pyramid, wherein the significant feature analyzer includes a brightness extraction unit, a color extraction unit, and a direction extraction unit; performing multi-scale Gaussian downsampling on the airway image through the brightness extraction unit, the color extraction unit, and the direction extraction unit to obtain a multi-scale airway image; based on the center-surrounding difference calculation principle, performing deviation analysis on the multi-scale airway image respectively to obtain an airway brightness feature map, an airway color feature map, and an airway direction feature map, and performing weighted superposition processing to fuse and generate an airway significant image; screening and segmenting the airway significant image according to a significant feature threshold, setting the area with a significance greater than the significant feature threshold as an airway significant area, and obtaining multiple airway significant areas, wherein each airway significant area is marked with an image significance.

[0007] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: obtaining the position information of the multiple airway salient areas in the airway image, mapping them into two-dimensional coordinate points, and constructing a two-dimensional distribution matrix; performing distribution discrete analysis on the two-dimensional distribution matrix, and setting the inverse of the distribution discreteness as the significant distribution concentration; configuring the initial image segmentation granularity scheme based on the significant distribution concentration and the multiple image saliencies.

[0008] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: if the significant distribution concentration is greater than or equal to a predetermined benchmark value, selecting the maximum image saliency among the multiple image saliencies, and inputting the maximum image saliency into the saliency-segmentation granularity comparison library, outputting the first image segmentation granularity, and setting it as the initial image segmentation granularity scheme; if the significant distribution concentration is less than the predetermined benchmark value, respectively inputting the multiple image saliencies into the saliency-segmentation granularity comparison library, outputting multiple image segmentation granularities, and setting them as the initial image segmentation granularity scheme.

[0009] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: in the same time zone as the airway image acquisition, synchronously monitoring and acquiring the user's respiratory rate, heart rate and cough intensity through sensors, and setting them as the user's physiological characteristics; in the same time zone as the airway image acquisition, synchronously monitoring and acquiring the motion characteristics of the airway lens, and setting them as lens operation characteristics, wherein the lens operation characteristics include lens acceleration, lens rotation rate and lens rotation angle.

[0010] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: based on historical airway examination records, collecting a sample user physiological feature set, a sample lens operation feature set and a sample tracheal image set, and analyzing to obtain a sample image blur set; using the sample user physiological feature set, the sample lens operation feature set and the sample image blur set as training data, and performing P-fold crossover to construct P training sets, where P is an integer greater than or equal to 20; using the P training sets to train the feedforward neural network separately until the network converges, obtaining P image blur prediction branches, and integrating to construct an image blur prediction plug-in; inputting the user physiological characteristics and lens operation characteristics into the image blur prediction plug-in to perform image blur prediction and generate image segmentation compensation coefficients.

[0011] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: randomly selecting a first sample user physiological feature, a first sample lens operation feature and a first sample tracheal image; obtaining a first standard tracheal image at a corresponding position under the first sample user physiological feature and the first sample lens operation feature; using the first standard tracheal image as a reference, performing fuzzy comparison on the first sample tracheal image, and obtaining an initial image blur by calculating the gradient amplitude of the image, wherein the initial image blur is greater than or equal to 0 and less than 100; setting the ratio of the initial image blur to 100 as the first sample image blur, and adding it to the sample image blur set.

[0012] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: performing feature change impact analysis according to the user's physiological characteristics and lens operation characteristics, outputting a physiological change coefficient and an operation change coefficient, and summing them to obtain a comprehensive change coefficient, wherein the feature change impact is the impact of the feature change on the airway image acquisition quality; multiplying the ratio of the obtained comprehensive change coefficient to the historical maximum comprehensive change coefficient by P and rounding it up to obtain the number of branches selected K; randomly selecting K image blur prediction branches from the P image blur prediction branches of the image blur prediction plug-in, performing image blur prediction on the user's physiological characteristics and lens operation characteristics, obtaining K predicted image blurs, and calculating the average to obtain the image segmentation compensation coefficient.

[0013] Preferably, the image segmentation method for assisting bronchoscopy examination also includes: adding 1 to the image segmentation compensation coefficient and summing the result to obtain a granularity correction weight; adjusting the initial image segmentation granularity scheme according to the granularity correction weight to obtain a corrected image segmentation granularity scheme, wherein the corrected image segmentation granularity is the product of the initial image segmentation granularity and the granularity correction weight.

[0014] The beneficial effects of the present invention are as follows: by analyzing the significant features of the airway image, a plurality of airway significant areas are determined; then, according to the image saliencies of the plurality of airway significant areas, an initial image segmentation granularity scheme is configured; on the other hand, the physiological characteristics of the user and the lens operation characteristics in the airway image acquisition process are synchronously monitored and acquired; further, image blur is predicted based on the physiological characteristics of the user and the lens operation characteristics to generate an image segmentation compensation coefficient; then, according to the image segmentation compensation coefficient, the initial image segmentation granularity scheme is adjusted to obtain a corrected image segmentation granularity scheme; finally, according to the corrected image segmentation granularity scheme, the plurality of airway significant areas are image segmented. In other words, by flexibly adjusting the image segmentation granularity according to the different characteristics of the lesion area and the changes in image quality, the segmentation precision and accuracy of the airway image can be significantly improved, and efficient and accurate image segmentation can be achieved, thereby better identifying different types of lesion areas and effectively reducing false detection and missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of an image segmentation method for assisting bronchoscopy provided by the present invention;

[0016] Figure 2 A schematic diagram of a process for determining multiple significant airway regions in an image segmentation method for assisting bronchoscopy provided by the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0019] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0020] Examples, such as Figure 1 As shown, an embodiment of the present invention provides an image segmentation method for assisting bronchoscopy examination, which specifically includes the following steps:

[0021] S10: Acquire an airway image of the lesion area through a bronchoscope, perform significant feature analysis on the airway image, and determine multiple significant airway areas.

[0022] Further, if Figure 2 As shown, step S10 of the present invention further includes:

[0023] S11: Constructing a significant feature analyzer based on the Gaussian pyramid, wherein the significant feature analyzer includes a brightness extraction unit, a color extraction unit and a direction extraction unit; S12: Performing multi-scale Gaussian downsampling on the airway image through the brightness extraction unit, the color extraction unit and the direction extraction unit to obtain a multi-scale airway image; S13: Based on the center-surrounding difference calculation principle, performing deviation analysis on the multi-scale airway image respectively to obtain an airway brightness feature map, an airway color feature map and an airway direction feature map, and performing weighted superposition processing to fuse and generate an airway significant image; S14: Screening and segmenting the airway significant image according to the significant feature threshold, setting the area with a significance greater than the significant feature threshold as an airway significant area, and obtaining multiple airway significant areas, wherein each airway significant area is marked with an image significance.

[0024] Specifically, a bronchoscope is used to enter the patient's trachea and bronchi through a slender fiber optic lens to directly observe the lesions in the airway. During this process, doctors can obtain airway images in real time and identify potential lesion areas, such as tumors, inflammation, bronchiectasis, foreign bodies, etc.; the lens is gently inserted into the patient's throat through the bronchoscope and gradually advanced along the airway to collect images of the trachea and bronchi in real time, and obtain airway images of the lesion area, which include masses, inflammation, infection, foreign bodies, tissue damage, etc.

[0025] Gaussian pyramid is a multi-scale image processing method that can obtain the features of the image at different scales by blurring and downsampling the image at different levels, so that the content of each level can be analyzed more carefully when processing the image. First, a salient feature analyzer is constructed based on the Gaussian pyramid. The salient feature analyzer includes a brightness extraction unit, a color extraction unit, and a direction extraction unit. The brightness extraction unit focuses on the brightness information of the image (i.e., grayscale value). Generally, brightness features are an important indicator for describing the changes between bright and dark areas in an image. Lesions in bronchoscopic images often have significant brightness differences from normal tissue (e.g., tumors may be darker or brighter than surrounding tissue). This module extracts brightness features on each layer of the Gaussian pyramid and generates a brightness feature map. The color extraction unit identifies salient features in bronchoscopic images by analyzing the color information of the image. Color provides visual perception information of the image and helps distinguish normal airways from lesions. This unit mainly extracts color features of the image, especially color saturation and contrast, which can help identify certain specific lesions (e.g., the color of certain types of tumors or inflammation is different from that of normal airways). The direction extraction unit analyzes the directional features of local areas in the image and is mainly used to identify edges and texture features. In bronchoscopic images, the edges of lesions are often more obvious than those of normal tissue and have directional characteristics, such as the edges of tumors or the shape of foreign objects. Direction extraction can effectively detect these edges and enhance the saliency of related areas in the image. Through multi-scale processing of Gaussian pyramid and comprehensive analysis of brightness, color and direction, the salient feature analyzer can effectively identify important feature areas in bronchoscopic images.

[0026] Next, the airway image is subjected to multi-scale Gaussian downsampling through the brightness extraction unit, the color extraction unit, and the direction extraction unit. Through Gaussian downsampling, the image is processed at different scales. Each layer of the Gaussian pyramid represents a different image resolution, and the image resolution is gradually reduced so that the image exhibits different features at multiple scales. By applying Gaussian blur and downsampling operations to the airway image, images of multiple scales are generated. The resolution of each layer of the image is reduced layer by layer, which helps to capture features of different scales, such as small lesions and larger lesion areas. A multi-scale airway image is obtained.

[0027] Then, based on the principle of center-surround difference calculation, deviation analysis is performed on the multi-scale airway images. By calculating the difference between the central area and the surrounding areas of the image, those areas with significant differences are identified. The central area usually contains the target or lesion, while the surrounding area is the background or normal tissue. By analyzing the difference between the two, the significant areas in the image can be highlighted. Among them, by comparing the brightness difference between the center and the surrounding areas of the image, the areas with significant brightness changes are highlighted, and a brightness feature map is generated. In different color spaces (such as HSV, Lab, etc.), the color difference of the airway image is analyzed, and the color difference between the central area and the surrounding area is calculated to generate a color feature map. The image is processed using an edge detection algorithm (such as the Sobel operator and Canny edge detection) to extract edge information in different directions in the airway image and generate a directional feature map. The airway brightness feature map, airway color feature map, and airway direction feature map are further subjected to weighted superposition processing. That is, a weight is assigned to each feature map (brightness, color, direction). Different features contribute differently to the salient area, so the setting of the weight affects the generation of the final salient image. Generally speaking, brightness and color features have a greater impact on most lesion areas, while direction features help highlight the boundaries of the lesion. Through weighted superposition, the brightness, color, and direction information are combined to obtain a salient image that can better highlight the lesion area, namely the airway salient image. Through multi-scale downsampling and center-surround difference analysis, lesion information of different scales can be captured, especially small lesions and edge areas. The weighted superposition processing can fuse different features, effectively highlighting the salient areas in the airway image, and improving the accuracy of image segmentation and analysis.

[0028] The salient feature threshold is configured. This is a key parameter that controls which areas are considered lesions and which are considered background. Areas with saliency greater than the threshold represent areas with strong feature differences, typically lesions, while areas with saliency less than the threshold may be normal or background areas. The salient feature threshold can be automatically calculated based on image statistical features (such as mean and standard deviation) or a constant value selected empirically. Next, the airway salient image is screened and segmented based on the salient feature threshold. This involves performing a threshold comparison on each pixel in the airway salient image. If the pixel's saliency (a weighted result of brightness, color, or directional features displayed in the image) is greater than the set threshold, the pixel is considered a salient area. This threshold comparison identifies salient areas in the airway image. Salient areas typically correspond to areas with significant feature differences, such as lesions, lesions, or abnormal structures. This threshold screening ignores other background or irrelevant areas, resulting in multiple salient airway areas.

[0029] Next, a connected domain analysis is performed on the salient regions in the image, connecting adjacent salient regions to form a complete region. Each salient region is labeled and assigned a unique identifier. Each salient region is extracted individually and associated with its saliency value. The image saliency of an airway salient region is the average of the saliency values ​​of multiple pixels within the region. This results in each airway salient region having a clear identifier and saliency value. By screening and segmenting the airway salient image based on a salient feature threshold, the lesion region in the airway image can be effectively extracted and the saliency of each salient region accurately identified.

[0030] S20: configuring an initial image segmentation granularity scheme according to the plurality of image saliencies of the plurality of airway salient regions.

[0031] Furthermore, step S20 of the present invention further includes:

[0032] S21: Obtain the position information of the multiple airway salient areas in the airway image, map them into two-dimensional coordinate points, and construct a two-dimensional distribution matrix; S22: Perform distribution discrete analysis on the two-dimensional distribution matrix, and set the inverse of the distribution discreteness as the significant distribution concentration.

[0033] Specifically, the positional information of the multiple salient airway regions within the airway image is obtained. The positional information of the salient regions refers to the center position of each salient region in the airway image. After image segmentation, the salient regions are extracted and marked. Connected domain analysis or edge detection algorithms (such as Canny edge detection) can be used to determine the boundary pixels of each salient region. The positional information of each salient region can be obtained by calculating its geometric center. The geometric center is the average coordinate (i.e., the center of mass) of all pixels within the region, representing the center of the region. For each salient region in the image, the above steps are repeated to calculate and obtain the geometric center coordinates of each salient region. Each salient region has a corresponding two-dimensional coordinate point. Next, these two-dimensional coordinate points are mapped into a two-dimensional coordinate system to construct a two-dimensional distribution matrix. The two-dimensional distribution matrix represents the distribution of salient regions. It represents the coordinate points in the image in two-dimensional space, with each element of the matrix corresponding to the distribution of a region in the image. The two-dimensional distribution matrix can reflect the spatial distribution of salient regions in the airway image. By analyzing the matrix, the degree of clustering of salient regions can be identified, helping to further determine the type or location of the lesion.

[0034] Then, the two-dimensional distribution matrix is ​​subjected to a distribution dispersion analysis. The distribution dispersion is used to measure whether the distribution of the significant areas in the airway image is uniform. The larger the distribution dispersion, the more dispersed the distribution of the significant areas, and vice versa. The distribution dispersion is a measure of the difference between each element and the mean. It is obtained by calculating the sum of the square differences between each element and the mean in the two-dimensional distribution matrix to obtain the distribution dispersion. The inverse of the distribution dispersion is further set as the significant distribution concentration. The significant distribution concentration reflects the degree of aggregation of the significant areas in the airway image. When the distribution dispersion is small (i.e., the significant areas are more concentrated), the significant distribution concentration is large, indicating that the significant areas are more concentrated in the image. Conversely, when the distribution dispersion is large, the significant distribution concentration is small, indicating that the significant areas are more dispersed. The significant distribution concentration provides an effective indicator for measuring the concentration of significant areas, helping to optimize image segmentation strategies and improve the precision and accuracy of image segmentation.

[0035] S23: configuring an initial image segmentation granularity scheme according to the saliency distribution concentration and the plurality of image saliencies.

[0036] Furthermore, step S23 of the present invention further includes:

[0037] S231: If the saliency distribution concentration is greater than or equal to a predetermined benchmark value, select the maximum image saliency among the multiple image saliencies, and input the maximum image saliency into the saliency-segmentation granularity comparison library, output the first image segmentation granularity, and set it as the initial image segmentation granularity scheme; S232: If the saliency distribution concentration is less than the predetermined benchmark value, input the multiple image saliencies into the saliency-segmentation granularity comparison library respectively, output multiple image segmentation granularities, and set them as the initial image segmentation granularity scheme.

[0038] Specifically, the calculated concentration of significant distribution is compared with a predetermined benchmark value to decide whether to use a globally unified segmentation granularity or to dynamically adjust the segmentation granularity according to the characteristics of the local area. The predetermined benchmark value is a threshold value set according to the actual needs of the airway image and the segmentation strategy, which is used to judge the distribution characteristics of the significant area in the image and can be set according to the actual scene. If the significant area is relatively concentrated, it means that the lesion area or feature area in the image is roughly concentrated in a certain area and the distribution is relatively uniform. At this time, you can choose to use a globally unified segmentation granularity, that is, use a unified image segmentation granularity to process the entire image. If the distribution of significant areas is relatively dispersed, it means that the lesion area or feature area in the image is relatively dispersed and may have different characteristics. At this time, the segmentation granularity needs to be dynamically adjusted according to the characteristics of different significant areas, and a local image segmentation granularity strategy should be adopted to adapt to the needs of different lesion areas.

[0039] The significant distribution concentration is judged according to a predetermined benchmark value. When the significant distribution concentration is greater than or equal to the predetermined benchmark value, it indicates that the significant areas in the image are relatively concentrated, and it is suitable to use a globally unified image segmentation granularity; then, from multiple airway significant areas, the area with the largest image significance is selected, and the significance values ​​of these significant areas represent the significance of the lesions or important features; then the selected maximum image significance is input into the significance-segmentation granularity control library, and the corresponding first image segmentation granularity is output according to the significance value; since the significant areas are relatively concentrated, a globally unified segmentation granularity is adopted, so the output segmentation granularity is the initial image segmentation granularity scheme, which is used for subsequent image segmentation processing. Among them, the saliency-segmentation granularity reference library is a database or lookup table containing the mapping relationship between saliency and corresponding segmentation granularity. During the image segmentation process, by searching this reference library, the system can automatically select the appropriate segmentation granularity according to the saliency value of the image, thereby optimizing the segmentation process and improving the accuracy and efficiency of segmentation. It can be set based on historical data analysis. For example, high saliency > 0.8 matches fine granularity (higher resolution); medium saliency (0.5 to 0.8) matches medium granularity (moderate resolution), etc.

[0040] When the saliency distribution concentration is less than a predetermined benchmark value, it indicates that the salient regions in the image are relatively dispersed, and the image segmentation granularity may need to be dynamically adjusted based on the characteristics of each region. Next, for each salient region, its image saliency is entered into a saliency-segmentation granularity comparison library, and the corresponding segmentation granularity is output based on the image saliency of each salient region. Because salient regions are dispersed, a different image segmentation granularity can be output for each region. Each region will receive an appropriate segmentation granularity based on its saliency and characteristics, resulting in multiple image segmentation granularities, which are then set as the initial image segmentation granularity scheme. This flexible segmentation granularity adjustment method can effectively adapt to the needs of different images. When salient regions are relatively concentrated, a globally unified image segmentation granularity is selected to improve the simplicity and integrity of the segmentation. When salient regions are dispersed, the segmentation granularity is dynamically adjusted based on the saliency of each region, improving the precision and specificity of the segmentation, thereby improving segmentation accuracy.

[0041] S30: synchronously monitoring and acquiring the user's physiological characteristics and lens operation characteristics during the airway image acquisition process, performing image blur prediction, and generating image segmentation compensation coefficients.

[0042] Furthermore, step S30 of the present invention further includes:

[0043] S31: In the same time zone as the airway image acquisition, the user's respiratory rate, heart rate and cough intensity are synchronously monitored and acquired through sensors, and set as the user's physiological characteristics; S32: In the same time zone as the airway image acquisition, the motion characteristics of the airway lens are synchronously monitored and acquired, and set as the lens operation characteristics, wherein the lens operation characteristics include lens acceleration, lens rotation rate and lens rotation angle.

[0044] Specifically, during the airway image acquisition process, by synchronously monitoring the user's physiological characteristics and the operating characteristics of the lens in real time, the image quality can be effectively optimized, especially in dynamic or blurred environments, providing valuable information for image segmentation and subsequent analysis. First, in the same time zone as the airway image acquisition, the user's respiratory rate, heart rate, and cough intensity are synchronously monitored by sensors. Among them, monitoring the user's respiratory rate helps to understand the user's respiratory status. Fluctuations in breathing may cause blurring of bronchoscopic images during dynamic processes. Therefore, respiratory rate can be used as an indicator to evaluate image clarity and stability; by measuring the user's heart rate, their physiological status can be reflected. A high or low heart rate may affect the user's operational stability, thereby affecting the image acquisition quality; coughing may cause vibration or blurring of the image. Monitoring cough intensity can compensate for this during image processing, thereby reducing the image quality degradation caused by the user's physiological response, and setting the respiratory rate, heart rate, and cough intensity as user physiological characteristics.

[0045] Furthermore, within the same time zone as airway image acquisition, simultaneous monitoring is performed to acquire airway lens motion characteristics, including lens acceleration, lens rotation rate, and lens rotation angle. Lens acceleration reflects the rate of change of lens motion. Excessively rapid or violent movement can cause image blur, especially when the lens moves or stops suddenly, which can lead to image distortion. Lens rotation rate directly affects image stability. A high rotation rate can cause image jitter or distortion, impacting subsequent image processing and analysis. Lens rotation angle helps determine the image's viewing angle and focus. Sharp changes in lens angle can lead to localized image quality degradation. Lens acceleration, lens rotation rate, and lens rotation angle are defined as lens operation characteristics. By synchronously monitoring the user's physiological characteristics and lens operation characteristics in real time, airway image quality can be more accurately assessed. Image processing strategies can then be dynamically adjusted based on these factors to improve image segmentation accuracy.

[0046] Furthermore, step S30 of the present invention further includes:

[0047] S33: Based on the historical airway examination records, a sample user physiological feature set, a sample lens operation feature set, and a sample tracheal image set are collected, and the sample image fuzziness set is obtained by analysis.

[0048] Furthermore, step S33 of the present invention further includes:

[0049] S331: Randomly select the first sample user physiological characteristics, the first sample lens operation characteristics and the first sample tracheal image; S332: Obtain the first standard tracheal image at the corresponding position under the first sample user physiological characteristics and the first sample lens operation characteristics; S333: Taking the first standard tracheal image as a reference, perform fuzzy comparison on the first sample tracheal image, and obtain the initial image blur by calculating the gradient amplitude of the image, wherein the initial image blur is greater than or equal to 0 and less than 100; S334: Set the ratio of the initial image blur to 100 as the first sample image blur, and add it to the sample image blur set.

[0050] Specifically, based on historical airway examination records, sample user physiological feature sets, sample lens operation feature sets and sample tracheal image sets are collected. User physiological features include breathing, heartbeat and coughing. For example, respiratory rate refers to the number of breaths per minute, which reflects the patient's respiratory status. Frequent breathing changes may affect the stability of the image; heart rate refers to the number of heartbeats per minute. The patient's heart rate may be affected by tension or other factors, which in turn affects the clarity of the bronchoscopic image; coughing can cause image blur and displacement; lens operation characteristics include the operation of the bronchoscope itself, which will affect the image quality, especially when the lens movement is unstable. These lens operation characteristics can help judge the stability of the lens during the inspection process and provide a reference for image optimization.

[0051] Next, a first sample user physiological feature, a first sample lens operation feature, and a first sample tracheal image are randomly selected, where the first sample is any one of the sample datasets. A first standard tracheal image corresponding to the first sample user physiological feature and the first sample lens operation feature is then obtained. This image should represent a clear tracheal image captured under normal physiological conditions and stable lens operation, serving as a benchmark for subsequent comparison and blur analysis. Furthermore, using the first standard tracheal image as a benchmark, a fuzzy comparison is performed on the first sample tracheal image. This comparison involves comparing the first standard tracheal image with the first sample tracheal image. Image clarity is determined by calculating the image gradient magnitude. The gradient magnitude can be used to measure changes in image detail; blurred images have lower gradient magnitudes, while sharp images have higher gradient magnitudes. First, the gradient of each pixel in the image is calculated (for example, using the Sobel operator or other edge detection algorithms). Next, the gradient magnitudes are calculated to obtain image edge information. Less edge information indicates a more blurred image, while less edge information indicates a sharper image. Blur is assessed by calculating the difference in image gradient magnitudes. The initial image blur is generated based on the result of image gradient amplitude calculation. The value of the initial image blur should be between 0 and 100, where a lower value indicates a clearer image and a higher value indicates a blurred image.

[0052] The initial image blur is then calculated by comparing it to 100, resulting in a standardized image blur, which is then set as the first sample image blur and added to the sample image blur set for subsequent data analysis. By using the image gradient amplitude as the basis for calculation and comparing standard tracheal images with actual tracheal images, the blur of the tracheal image can be accurately assessed. This blur data is then used to optimize the image segmentation granularity, dynamically adjusting the segmentation strategy based on image clarity. This ensures finer-grained segmentation when the image is blurry, avoiding false or missed detections due to image blur.

[0053] S34: Using the sample user physiological feature set, the sample lens operation feature set and the sample image blur set as training data, and performing P-fold crossover to construct P training sets, where P is an integer greater than or equal to 20; S35: Using the P training sets, respectively train the feedforward neural network until the network converges, and obtain P image blur prediction branches, which are integrated to construct an image blur prediction plug-in.

[0054] Specifically, the sample user physiological feature set, the sample lens operation feature set and the sample image blur set are used as training data, and P-fold crossover is performed, that is, the training data is divided into P equal parts to obtain P data sets, where P is an integer greater than or equal to 20; then, the P data sets are selected with replacement P times to construct the first training set; and the same method is used to iteratively select P times to obtain P training sets.

[0055] Next, the P training sets are used to train feedforward neural networks, each of which includes an input layer (receiving data from user physiological characteristics, lens operation characteristics, and image blur), a hidden layer (performing feature learning and pattern recognition via multiple hidden layers), and an output layer (outputting predicted image blur). The feedforward neural network is then trained using sample user physiological characteristics and sample lens operation characteristics as input, and sample image blur as supervision. During training, a backpropagation algorithm is used to adjust network weights until the network output approaches the true blur value. During training, a loss function (such as mean squared error) is used to measure the difference between the predicted and actual values, optimizing the network parameters until the network converges. Multiple rounds of iterations are performed until the loss value of the neural network is stable and sufficiently low, meeting a preset convergence criterion, indicating that the model has been trained. P image blur prediction branches are then generated, and an image blur prediction plug-in is constructed based on these P image blur prediction branches. This plug-in can dynamically predict image blur based on factors such as user physiological characteristics and lens operation characteristics, thereby providing a more accurate blur reference for image segmentation and diagnosis.

[0056] S36: Inputting the user's physiological characteristics and lens operation characteristics into the image blur prediction plug-in to perform image blur prediction and generate image segmentation compensation coefficients.

[0057] Furthermore, step S36 of the present invention further includes:

[0058] S361: Perform feature change impact analysis based on the user's physiological characteristics and lens operation characteristics, output the physiological change coefficient and the operation change coefficient, and sum them to obtain the comprehensive change coefficient, wherein the feature change impact is the impact of the feature change on the airway image acquisition quality; S362: Multiply the ratio of the obtained comprehensive change coefficient to the historical maximum comprehensive change coefficient by P and round it up to obtain the number of branch selections K; S363: Randomly select K image blur prediction branches from the P image blur prediction branches of the image blur prediction plug-in, perform image blur prediction on the user's physiological characteristics and lens operation characteristics, obtain K predicted image blurs, and calculate the average to obtain the image segmentation compensation coefficient.

[0059] Specifically, first, a feature change impact analysis is performed based on the user's physiological characteristics. Physiological characteristics of the user (such as respiratory rate, heart rate, and cough intensity) can affect the quality of airway image acquisition. By analyzing changes in these physiological characteristics, the impact of these changes on image quality is estimated. Specifically, an impact weight is assigned to each physiological characteristic (respiratory rate, heart rate, and cough intensity). Based on the impact of these characteristics on image quality, a comprehensive physiological feature impact is obtained, which is designated as the physiological change coefficient. Secondly, a feature change impact analysis is performed based on the lens operation characteristics. A weight is assigned to each operation characteristic (acceleration, rotation rate, and rotation angle). Based on the impact of these characteristics on image quality, a comprehensive operation characteristic impact is obtained, which is designated as the operation change coefficient. The physiological change coefficient and the operation change coefficient are then added together, and the sum is taken as the comprehensive change coefficient, representing the combined impact of the overall feature changes on airway image acquisition quality.

[0060] Next, the historical maximum comprehensive variation coefficient is obtained. The historical maximum comprehensive variation coefficient refers to the maximum value of all comprehensive variation coefficients recorded in the historical data after multiple image acquisitions and analyses. This value represents the maximum degree of change observed in history and can be used as a standard to compare changes in the current image acquisition process. The ratio of the obtained comprehensive variation coefficient to the historical maximum comprehensive variation coefficient is then multiplied by P and rounded to the integer to obtain the number of branches selected, K. Further, within the P image blur prediction branches of the image blur prediction plug-in, K image blur prediction branches are randomly selected to predict image blur based on the user's physiological characteristics and lens operation characteristics. Each branch predicts image blur based on the user's physiological characteristics (such as breathing rate, heart rate, cough intensity) and lens operation characteristics (such as lens acceleration, lens rotation rate, lens rotation angle), obtaining K predicted image blurs. Finally, the K predicted image blurs are averaged to obtain the image segmentation compensation coefficient.

[0061] By dynamically adjusting the number of branches used for image blur prediction according to the degree of impact of image acquisition, the most suitable prediction model or branch can be selected for processing according to different image acquisition situations. This ensures that in image acquisition situations with greater impact, more branches are used for more refined blur prediction, while in situations with less impact, redundant calculations are avoided. This effectively reduces unnecessary waste of computing resources and improves the accuracy and efficiency of blur prediction while ensuring the accuracy of image blur prediction.

[0062] S40: adjusting the initial image segmentation granularity scheme according to the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme, and performing image segmentation on the plurality of airway salient regions.

[0063] Furthermore, step S40 of the present invention further includes:

[0064] S41: add 1 to the image segmentation compensation coefficient, and sum to obtain a granularity correction weight; S42: adjust the initial image segmentation granularity scheme according to the granularity correction weight to obtain a corrected image segmentation granularity scheme, wherein the corrected image segmentation granularity is the product of the initial image segmentation granularity and the granularity correction weight.

[0065] Specifically, the sum of 1 and the image segmentation compensation coefficient is used as a granularity correction weight, representing the degree of image segmentation adjustment. This weight determines the extent of the adjustment required to the initial image segmentation granularity scheme. Next, the initial image segmentation granularity scheme is adjusted based on the granularity correction weight, which is the product of the initial segmentation granularity and the compensation coefficient. This process allows the segmentation granularity to be adjusted based on the actual image conditions (such as the degree of blur and the size of the salient regions). This adjustment better meets the actual image requirements, thereby improving segmentation accuracy and precision. Finally, using the corrected image segmentation granularity scheme, multiple salient airway regions are segmented. The segmentation details of each salient region are optimized based on its saliency and the adjusted granularity. This means that the segmentation granularity is finer for smaller lesions, while the granularity is relatively coarser for larger lesions. By introducing the image segmentation compensation coefficient, the segmentation granularity can be adjusted based on the actual image quality and characteristics. For blurred or difficult-to-identify areas, the system automatically increases the segmentation granularity to avoid missegments; for clear areas, the granularity is reduced to improve efficiency.

[0066] An image segmentation method for assisting bronchoscopy provided by an embodiment of the present invention has at least the following technical effects:

[0067] By analyzing the significant features of the airway image, multiple airway significant areas are determined; then, based on the multiple image saliencies of the multiple airway significant areas, an initial image segmentation granularity scheme is configured; on the other hand, the physiological characteristics of the user and the lens operation characteristics during the airway image acquisition process are synchronously monitored and acquired; further, image blurriness is predicted based on the user physiological characteristics and the lens operation characteristics to generate an image segmentation compensation coefficient; then, the initial image segmentation granularity scheme is adjusted based on the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme; finally, the multiple airway significant areas are image segmented based on the corrected image segmentation granularity scheme. In other words, by flexibly adjusting the image segmentation granularity according to the different characteristics of the lesion area and the changes in image quality, the segmentation precision and accuracy of the airway image can be significantly improved, and efficient and accurate image segmentation can be achieved, thereby better identifying different types of lesion areas and effectively reducing false detections and missed detections.

[0068] Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts become known.

[0069] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An image segmentation method for assisting bronchoscopy, characterized in that: Methods include: collecting an airway image of the lesion area through a bronchoscope, performing significant feature analysis on the airway image, and determining a plurality of significant airway areas; configuring an initial image segmentation granularity scheme according to a plurality of image saliencies of the plurality of airway salient regions; Synchronously monitor and acquire the user's physiological characteristics and lens operation characteristics during airway image acquisition, predict image blur, and generate image segmentation compensation coefficients; Adjusting the initial image segmentation granularity scheme according to the image segmentation compensation coefficient to obtain a corrected image segmentation granularity scheme, and performing image segmentation on the multiple airway salient areas; The user's physiological characteristics and lens operation characteristics during the airway image acquisition process are monitored and acquired simultaneously, including: In the same time zone as the airway image acquisition, the user's respiratory rate, heart rate, and cough intensity are synchronously monitored by sensors and set as the user's physiological characteristics; In the same time zone as the airway image acquisition, the motion characteristics of the airway lens are synchronously monitored and obtained, which are set as lens operation characteristics, wherein the lens operation characteristics include lens acceleration, lens rotation rate and lens rotation angle; Among them, the image blur prediction and the generation of image segmentation compensation coefficients include: Based on historical airway examination records, a sample user physiological feature set, a sample lens operation feature set, and a sample tracheal image set are collected, and the sample image blur set is obtained by analysis; Using the sample user physiological feature set, the sample lens operation feature set, and the sample image blur set as training data, and performing P-fold crossover to construct P training sets, where P is an integer greater than or equal to 20; Using the P training sets, respectively training the feedforward neural network until the network converges, obtaining P image blur prediction branches, and integrating them to construct an image blur prediction plug-in; The user's physiological characteristics and lens operation characteristics are input into the image blur prediction plug-in to perform image blur prediction and generate image segmentation compensation coefficients.

2. The image segmentation method for assisting bronchoscopy according to claim 1, characterized in that: Performing significant feature analysis on the airway image to determine multiple significant airway regions, including: Constructing a salient feature analyzer based on a Gaussian pyramid, wherein the salient feature analyzer includes a brightness extraction unit, a color extraction unit, and a direction extraction unit; Performing multi-scale Gaussian downsampling on the airway image by the brightness extraction unit, the color extraction unit, and the direction extraction unit to obtain a multi-scale airway image; Based on the center-surround difference calculation principle, the multi-scale airway images are subjected to deviation analysis to obtain airway brightness feature maps, airway color feature maps, and airway direction feature maps, which are then weighted superposition processed and fused to generate an airway saliency image; The airway salient image is screened and segmented according to a salient feature threshold, and regions with saliency greater than the salient feature threshold are set as airway salient regions, thereby obtaining a plurality of airway salient regions, wherein each airway salient region is marked with an image saliency.

3. The image segmentation method for assisting bronchoscopy according to claim 2, characterized in that: Configuring an initial image segmentation granularity scheme according to the plurality of image saliencies of the plurality of airway salient regions includes: Obtaining position information of the plurality of airway salient regions in the airway image, mapping the information into two-dimensional coordinate points, and constructing a two-dimensional distribution matrix; Performing a distribution dispersion analysis on the two-dimensional distribution matrix, and setting the inverse of the distribution dispersion as the significant distribution concentration; An initial image segmentation granularity scheme is configured according to the saliency distribution concentration and the plurality of image saliencies.

4. The image segmentation method for assisting bronchoscopy according to claim 3, characterized in that: Configuring an initial image segmentation granularity scheme according to the saliency distribution concentration and the plurality of image saliencies, including: If the saliency distribution concentration is greater than or equal to a predetermined reference value, selecting a maximum image saliency among the multiple image saliencies, inputting the maximum image saliency into a saliency-segmentation granularity comparison library, outputting a first image segmentation granularity, and setting it as an initial image segmentation granularity scheme; If the saliency distribution concentration is less than a predetermined reference value, the plurality of image saliencies are respectively input into a saliency-segmentation granularity comparison library, and a plurality of image segmentation granularities are output and set as an initial image segmentation granularity scheme.

5. The image segmentation method for assisting bronchoscopy according to claim 1, characterized in that: Analyze and obtain the sample image blur set, including: Randomly selecting a first sample user physiological feature, a first sample lens operation feature, and a first sample tracheal image; Acquire a first standard trachea image at a corresponding position under the first sample user physiological characteristics and the first sample lens operation characteristics; Performing fuzzy comparison on the first sample trachea image based on the first standard trachea image, and obtaining an initial image fuzziness by calculating the gradient amplitude of the image, wherein the initial image fuzziness is greater than or equal to 0 and less than 100; The ratio of the initial image blur to 100 is set as the first sample image blur, and added to the sample image blur set.

6. The image segmentation method for assisting bronchoscopy according to claim 1, characterized in that: Inputting the user's physiological characteristics and lens operation characteristics into the image blur prediction plug-in to perform image blur prediction and generate image segmentation compensation coefficients, including: Performing feature change impact analysis based on the user's physiological characteristics and lens operation characteristics, outputting a physiological change coefficient and an operation change coefficient, and summing them to obtain a comprehensive change coefficient, wherein the feature change impact is the impact of the feature change on the airway image acquisition quality; The ratio of the obtained comprehensive variation coefficient to the historical maximum comprehensive variation coefficient is multiplied by P and rounded to the integer to obtain the number of branches to be selected, K; Among the P image blur prediction branches of the image blur prediction plug-in, K image blur prediction branches are randomly selected, and image blur prediction is performed on the user physiological characteristics and lens operation characteristics to obtain K predicted image blurs, and the image segmentation compensation coefficient is obtained by average calculation.

7. The image segmentation method for assisting bronchoscopy according to claim 6, characterized in that: Adjusting the initial image segmentation granularity scheme according to the image segmentation compensation coefficient includes: The sum of 1 and the image segmentation compensation coefficient is used as a granularity correction weight; The initial image segmentation granularity scheme is adjusted according to the granularity correction weight to obtain a corrected image segmentation granularity scheme, wherein the corrected image segmentation granularity is the product of the initial image segmentation granularity and the granularity correction weight.

Images