Boundary recognition method and system for bronchial fluorescence images based on visual detection

Abstract

This invention discloses a method and system for boundary recognition of bronchial fluorescence images based on visual detection. The invention relates to the technical field of visual detection, and determines multiple edge portions of the bronchus based on edge detection of the bronchial region; determines the bronchial boundary based on the recognition of these edge portions; and determines the actual morphology of the bronchus based on the bronchial boundary and the bronchial fluorescence image, ensuring the accuracy of bronchial boundary recognition and further improving the accuracy of the actual bronchial morphology. Therefore, a core review area is determined based on the actual bronchial morphology and the user's age; abnormal feature combinations are determined based on the recognition of the core review area; abnormal events are determined based on the abnormal feature combinations and the core review area; and the bronchial state level is determined based on the abnormal event, the actual bronchial morphology, and the surrounding environment of the bronchus, ensuring the accuracy of bronchial state level recognition.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a method and system for boundary recognition of bronchial fluorescence images based on visual inspection. Background Technology

[0002] With the development of technology, bronchial fluorescence images are required for bronchial detection. These images are formed by capturing images of the bronchi using visual inspection devices. Bronchial fluorescence images utilize fluorescence imaging technology to observe the actual morphology of the bronchi. In existing technologies, bronchial fluorescence images are acquired, and the boundaries of the bronchi are directly marked within these images. However, the determination of these boundaries is not precise, and the actual morphology of the bronchi determined based on these boundaries is also inaccurate, affecting the accuracy of identifying the bronchial condition level. Furthermore, this method relies solely on routine analysis of fluorescence images. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and system for boundary recognition of bronchial fluorescence images based on visual detection.

[0004] This invention provides a method for boundary recognition of bronchial fluorescence images based on visual detection, comprising:

[0005] The location of the bronchi is acquired, and corresponding visual detection is triggered based on the location of the bronchi to acquire bronchial fluorescence images;

[0006] The bronchial region and the surrounding area are determined based on the detection of bronchial fluorescence images, and multiple edge parts of the bronchus are determined based on the edge detection of the bronchial region.

[0007] The bronchial boundary is determined by identifying multiple edge portions of the bronchus, and the actual morphology of the bronchus is determined based on the bronchial boundary and bronchial fluorescence images.

[0008] The core re-examination area is determined based on the actual morphology of the bronchi and the user's age, and the abnormal feature combination is determined based on the identification of the core re-examination area;

[0009] Abnormal events are identified based on combinations of abnormal features and core re-examination areas, and the bronchial status level is determined based on the abnormal events, the actual morphology of the bronchus, and the peripheral environment of the bronchus.

[0010] This invention provides a boundary recognition system for bronchial fluorescence images based on visual detection. This system is applied to the aforementioned boundary recognition method for bronchial fluorescence images based on visual detection. The boundary recognition system for bronchial fluorescence images based on visual detection includes:

[0011] The fluorescence image module is used to acquire the location of the bronchi and trigger corresponding visual detection based on the location of the bronchi to acquire bronchial fluorescence images.

[0012] The image detection module is used to determine the bronchial region and the environmental region based on the detection of bronchial fluorescence images, and to determine multiple edge parts of the bronchus based on the edge detection of the bronchial region.

[0013] The morphology detection module is used to determine the bronchial boundary based on the identification of multiple edge parts of the bronchus, and to determine the actual morphology of the bronchus based on the bronchial boundary and bronchial fluorescence image.

[0014] The abnormal feature module is used to determine the core re-examination area based on the actual morphology of the bronchi and the user's age, and to determine the abnormal feature combination based on the identification of the core re-examination area;

[0015] The status level module is used to identify abnormal events based on combinations of abnormal features and core review areas, and to determine the status level of the bronchus based on the abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] (1) The present invention first acquires the position of the bronchus, triggers corresponding visual detection based on the position of the bronchus to acquire bronchial fluorescence images, determines the bronchus region and the environment region based on the detection of the bronchial fluorescence images, determines multiple edge parts of the bronchus based on the edge detection of the bronchus region, determines the boundary of the bronchus based on the identification of multiple edge parts of the bronchus, and determines the actual shape of the bronchus based on the boundary of the bronchus and the bronchial fluorescence images, ensuring the accuracy of the identification of the bronchus boundary, and taking into account the overall consideration of the bronchus boundary and the bronchial fluorescence images, further improving the accuracy of the actual shape of the bronchus.

[0018] (2) The present invention determines the core re-examination area based on the actual morphology of the bronchus and the user's age, determines the abnormal feature combination based on the identification of the core re-examination area, determines the abnormal event based on the abnormal feature combination and the core re-examination area, and determines the bronchus status level based on the abnormal event, the actual morphology of the bronchus and the peripheral environment of the bronchus. It takes into account the overall consideration of abnormal events, the actual morphology of the bronchus and the peripheral environment of the bronchus, and ensures the accuracy of the identification of the bronchus status level.

[0019] (3) This invention relates to the determination of the core re-examination area. Specifically, the core re-examination area is determined by combining the user's age information and the re-examination mapping relationship. The re-examination mapping relationship associates the user's age, gender, medical history and other factors with the risk level of the abnormal area. By further and more detailed surface identification and structural identification of the core re-examination area, a complete combination of abnormal features is formed to more comprehensively describe the abnormal situation of the core re-examination area, thereby gaining a more accurate understanding of the patient's bronchial status. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the boundary recognition method for bronchial fluorescence images based on visual detection in an embodiment of the present invention.

[0021] Figure 2 This is a flowchart illustrating step S11 in the boundary recognition method for bronchial fluorescence images based on visual detection in an embodiment of the present invention.

[0022] Figure 3 This is a flowchart illustrating step S12 in the boundary recognition method for bronchial fluorescence images based on visual detection in an embodiment of the present invention.

[0023] Figure 4 This is a flowchart illustrating step S13 in the boundary recognition method for bronchial fluorescence images based on visual detection in an embodiment of the present invention.

[0024] Figure 5 This is a flowchart illustrating step S14 in the boundary recognition method for bronchial fluorescence images based on visual detection in an embodiment of the present invention.

[0025] Figure 6 This is a flowchart illustrating step S15 in the boundary recognition method for bronchial fluorescence images based on visual detection in an embodiment of the present invention.

[0026] Figure 7 This is a schematic diagram of the structural composition of the bronchial fluorescence image boundary recognition system based on visual detection in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0028] Please see Figures 1 to 7 A boundary recognition method for bronchial fluorescence images based on visual detection is applied to the boundary recognition scenario of bronchial fluorescence images based on visual detection. This boundary recognition method for bronchial fluorescence images based on visual detection includes:

[0029] Step S11: Acquire the location of the bronchus, and trigger corresponding visual detection based on the location of the bronchus to acquire bronchial fluorescence images;

[0030] Step S12: Determine the bronchial region and the environmental region based on the detection of bronchial fluorescence images, and determine multiple edge parts of the bronchus based on the edge detection of the bronchial region;

[0031] Step S13: Determine the bronchial boundary based on the identification of multiple edge parts of the bronchus, and determine the actual morphology of the bronchus based on the bronchial boundary and bronchial fluorescence image.

[0032] Step S14: Determine the core re-examination area based on the actual morphology of the bronchus and the user's age, and determine the abnormal feature combination based on the identification of the core re-examination area;

[0033] Step S15: Determine the abnormal event based on the combination of abnormal features and the core re-examination area, and determine the bronchial status level based on the abnormal event, the actual morphology of the bronchus, and the peripheral environment of the bronchus.

[0034] refer to Figure 2 In step S11, the position of the bronchus is acquired, and the corresponding visual detection is triggered based on the position of the bronchus to acquire the bronchial fluorescence image.

[0035] In the specific implementation of this invention, the specific steps are as follows:

[0036] S111: Detect the position of the bronchi and determine the position of the bronchi based on the detected position. At the same time, collect multiple posture parameters of the user and determine the user's current posture based on the multiple posture parameters and the user's height.

[0037] S112: Determine the area to be detected based on the location of the bronchus and the user's current posture, and trigger the peripheral visual detection device according to the spatial position of the area to be detected, so that the visual detection device moves relative to the area to be detected to perform visual detection of the area to be detected, and output the acquired bronchial fluorescence image.

[0038] In the embodiments of this application, the position of the bronchus is detected and the position of the bronchus is determined based on the detected position. At the same time, multiple posture parameters of the user are collected, and the user's current posture is determined based on the multiple posture parameters and the user's height. This overall consideration of multiple posture parameters and the user's height ensures the accuracy of the user's current posture.

[0039] At this point, the position of the bronchus is detected to determine its specific location within the body. A bronchoscope or other endoscopic device is then used. These devices are typically equipped with position sensors, such as optical locators, electromagnetic locators, or inertial measurement units (IMUs). These sensors track the movement of the bronchoscope within the body in real time and feed its position data back to the computer system. The position sensors determine the position and orientation of the bronchoscope by sending and receiving signals (such as light signals, electromagnetic signals, or acceleration / angular velocity data). The computer system converts this data into three-dimensional coordinates, thereby constructing the trajectory of the bronchoscope within the body.

[0040] Based on data from position sensors, the current position of the bronchi is accurately determined. At this point, the computer system processes the data from the position sensors and registers it with pre-acquired bronchial structure diagrams (such as CT scans or MRI images). The registration process involves aligning real-time position data with static image data to accurately locate the bronchi in three-dimensional space. Meanwhile, the registration algorithms include iterative closest point (ICP) algorithms, rigid transformations, or affine transformations to ensure accurate alignment between real-time position data and static image data.

[0041] The system acquires the user's current posture information, including the position and angle of the head, torso, and limbs. External sensors, such as cameras, depth sensors, or wearable devices (such as accelerometers, gyroscopes, or magnetometers), are used to capture the user's motion data and convert it into posture parameters. Simultaneously, the camera and depth sensor use image processing algorithms to identify the user's body parts and posture. Wearable devices use built-in sensors to measure the user's acceleration, angular velocity, and magnetic field direction to infer the user's posture.

[0042] Combining posture parameters and height information, an algorithm is used to determine the user's current posture. The collected posture parameters are then input into machine learning or deep learning models, which have been trained to infer the user's posture based on the input data. Height information, as an additional input feature, helps improve the accuracy of posture recognition. Machine learning models, including Support Vector Machines (SVM), Random Forests, or Deep Neural Networks, learn from large amounts of posture data to establish a mapping between input parameters and posture. During the inference phase, the model predicts the user's current posture based on the input posture parameters and height information.

[0043] Specifically, suppose a doctor is performing a bronchoscopy on a patient who is 175 cm tall. The doctor first inserts the bronchoscope into the patient's body and activates a position sensor to track the movement of the bronchoscope. As the bronchoscope is inserted deeper, the position sensor continuously sends position data to the computer system. After receiving this data, the computer system registers it with a pre-acquired CT scan image of the patient. Through a registration algorithm, the computer system can accurately determine the position of the bronchoscope in the body and construct the trajectory of the bronchus in three-dimensional space.

[0044] Meanwhile, during the examination, the doctor used a camera to capture the patient's posture information; the camera captured the position and angle of the patient's head, torso, and limbs, and input this data into a machine learning model; the model combined the patient's height information (175 cm) to infer that the patient's current posture was upright and slightly forward-leaning; based on the location of the bronchi and the patient's current posture, the doctor determined the bronchial region to be examined and adjusted the angle and depth of the bronchoscope to capture high-quality fluorescence images, which were then used for further diagnosis and analysis.

[0045] Furthermore, the detection area is determined based on the location of the bronchus and the user's current posture. The peripheral visual detection device is triggered based on the spatial position of the detection area, causing the visual detection device to move relative to the detection area to perform visual detection and output the acquired bronchial fluorescence image. This comprehensive consideration of the bronchus location and the user's current posture ensures the accuracy of the detection area.

[0046] At this point, based on the precise location of the bronchi and the user's current posture, the bronchial region requiring visual inspection is determined. Then, combining the bronchoscope's position data (from step S111) and the user's posture information (also from step S111), 3D modeling or image processing techniques are used to determine the area to be inspected. This typically involves spatially aligning the bronchoscope's position with the user's posture to identify areas in the bronchi where problems exist or require further examination. Simultaneously, specialized software tools are needed to visualize and analyze this data so that the physician can visually see the location of the bronchi, the user's posture, and the relationships between them. Using this information, the physician determines the specific location and extent of the area to be inspected.

[0047] To ensure high-quality image acquisition, visual inspection devices (such as cameras on bronchoscopes) must be accurately moved to the inspection area. This is achieved using robotics, servo motors, or other precision motion control equipment. These devices are typically connected to a computer system, receiving spatial location information of the inspection area and planning the device's path accordingly. The triggering process involves sending control signals to the motion control equipment, containing information such as the target location and speed the device needs to reach. Upon receiving these signals, the equipment activates the appropriate motion control algorithm to ensure the device moves smoothly and accurately to the inspection area.

[0048] After the visual inspection device reaches the area to be inspected, high-quality image acquisition is performed. Simultaneously, once the visual inspection device reaches the target position, a camera or other image acquisition device is activated to capture fluorescence images of the bronchus. These images typically need to be acquired under specific lighting conditions to ensure image quality and clarity. Optionally, the image acquisition process involves adjusting parameters such as the camera's focal length, exposure time, and white balance to obtain the best image effect. At the same time, it is also necessary to ensure that the acquired images are correlated with the spatial location information of the area to be inspected for subsequent analysis and processing.

[0049] The acquired bronchial fluorescence images are presented to doctors or analysis systems for further diagnosis and analysis. At this time, the acquired image data is transmitted from the visual inspection device to the computer system and displayed and processed using specialized software tools. These tools typically provide image enhancement, annotation, and analysis functions to help doctors better understand and interpret the image information. Meanwhile, the transmission of image data involves the use of wired or wireless communication technologies, such as USB, Ethernet, or Wi-Fi. It is also necessary to ensure the integrity and security of the image data to prevent data loss or tampering.

[0050] Specifically, suppose a doctor is performing a bronchoscopy on a patient and has already completed the position detection and posture recognition in step S111. In step S112, the doctor first determines the area to be examined—a suspected lesion area located in the middle of the bronchus—based on the position of the bronchus and the patient's current posture. The doctor uses robotic technology to control the movement of the camera on the bronchoscope. The robotic system receives the spatial position information of the area to be examined and plans a movement path from the current position to the target position. As the camera moves, the doctor can see the situation inside the bronchus in real time and adjust the camera's angle and focus to obtain the best image effect.

[0051] Once the camera reaches the target location, the doctor initiates the image acquisition process. Under specific lighting conditions, the camera captures fluorescent images of the bronchi and transmits this data to the computer system. The doctor uses specialized software tools to enhance and process the images to better observe and analyze suspected lesion areas. This example demonstrates how the various parts of step S112 work together and their application in actual clinical examinations. The doctor can determine the area to be examined based on the location of the bronchi and the patient's current posture, and use robotics and image acquisition equipment to accurately capture fluorescent images of the bronchi, thus providing strong support for subsequent diagnosis and analysis.

[0052] In one embodiment of this application, it is assumed that there are three regions to be detected: A, B, and C. Based on the current bronchial position (weight 0.5), the user's posture (weight 0.3), and known lesion information (weight 0.2), a comprehensive score for each region is calculated. It is assumed that region A has a score of 0.8 (position score 0.4 + posture score 0.3 + lesion information score 0.1), region B has a score of 0.6, and region C has a score of 0.5. Because region A has the highest score, it is identified as the region to be detected. The system then triggers the camera on the bronchoscope to move to the position of region A and performs image acquisition according to preset parameters. The acquired bronchial fluorescence image is output to the doctor's display screen and labeled as an image from region A.

[0053] refer to Figure 3 In step S12, the bronchial region and the environmental region are determined based on the detection of the bronchial fluorescence image, and multiple edge portions of the bronchus are determined based on the edge detection of the bronchial region.

[0054] In the specific implementation of this invention, the specific steps are as follows:

[0055] S121: Acquire bronchial fluorescence images, determine multiple branch features based on the detection of bronchial fluorescence images, determine the relative distance between two adjacent branch features based on the position of multiple branch features, and determine the bronchial region based on the relative distance between two adjacent branch features and the connectivity between two adjacent branch features.

[0056] S122: In the bronchial region, bronchial features are highlighted based on image preprocessing of the bronchial region. At the same time, the bronchial features are traversed, and multiple edge contours of the bronchial features are marked according to the traversal of the bronchial features. Multiple edge parts of the bronchus are determined based on the edge detection of the multiple edge contours.

[0057] S123: Non-bronchial regions are determined based on the comparison between bronchial regions and bronchial fluorescence images; environmental regions are determined based on the color regions of non-bronchial and bronchial regions; and the peripheral environment of the bronchi is determined based on the identification of the environmental regions.

[0058] In the embodiments of this application, bronchial fluorescence images are acquired, multiple branch features are determined based on the detection of bronchial fluorescence images, the relative distance between two adjacent branch features is determined based on the position of the multiple branch features, and the bronchial region is determined based on the relative distance between two adjacent branch features and the connectivity between two adjacent branch features. This approach takes into account both the relative distance between two adjacent branch features and the connectivity between two adjacent branch features, ensuring the accuracy of the bronchial region.

[0059] At this point, high-quality fluorescence images of the bronchial interior are acquired for subsequent feature detection and region determination. Then, a bronchoscope or other endoscopic device equipped with fluorescence imaging technology is used to image the interior of the bronchus under appropriate fluorescence excitation light. Fluorescence imaging technology uses light of a specific wavelength to excite fluorescent substances in the bronchial tissue, causing them to emit fluorescence of a specific wavelength. By capturing these fluorescence signals, high-contrast images of the bronchus are obtained.

[0060] Multiple branching features of the bronchi, such as bifurcation points and bends, are identified in fluorescence images. Simultaneously, image processing algorithms (such as edge detection, morphological processing, and feature extraction) are used to analyze the fluorescence images to identify the branching features of the bronchi. These algorithms can automatically detect edges, contours, and shape features in the image to determine the branching structure of the bronchi. For example, edge detection algorithms identify the boundaries of the bronchial walls, while morphological processing algorithms are used to smooth the image, remove noise, and enhance features.

[0061] To understand the topology of the bronchi, the relative distance between two adjacent branch features is measured. After identifying the branch features, measurement tools in image processing software or custom algorithms are used to calculate the distance between adjacent branch features. This involves marking the positions of the branch features in the image and using geometric measurement methods to calculate the distance between them. The distance is both a straight-line distance and a curved distance along the bronchial path.

[0062] By combining the relative distances and connectivity of branch features, the overall region of the bronchus is determined. At this point, image processing or computer vision techniques are used to construct a 3D model or 2D region map of the bronchus, with the relative distances and connectivity of adjacent branch features serving as key input information. Simultaneously, graph theory algorithms (such as minimum spanning tree and shortest path algorithms) are used to connect adjacent branch features and construct the connectivity structure of the bronchus. Furthermore, techniques such as shape analysis and pattern recognition are also considered to further refine the bronchial region.

[0063] Specifically, suppose a doctor is performing a bronchoscopy on a patient and has already acquired a bronchial fluorescence image. In step S121, the doctor first uses an image processing algorithm to analyze the image, identifying multiple bronchial branch features, including several obvious bifurcation points and bends. Next, the doctor uses measurement tools to calculate the relative distances between adjacent branch features; for example, he finds that the straight-line distance from a bifurcation point to its adjacent bend is approximately 2 centimeters. At the same time, he also confirms the connectivity between these branch features, that is, they are connected through the bronchial walls.

[0064] Based on this information, the doctor used image processing software to construct a two-dimensional map of the bronchus. In this map, each branch feature was marked, and their relative positions and connectivity were accurately represented. Through this map, the doctor could clearly see the overall structure and topological features of the bronchus. This example demonstrates the application of the S121 procedure in actual clinical examination. By acquiring bronchial fluorescence images, identifying branch features, measuring relative distances, and determining connectivity, the doctor can accurately locate the bronchial region, providing strong support for subsequent diagnosis and treatment.

[0065] Furthermore, in the bronchial region, bronchial features are highlighted based on image preprocessing of the bronchial region. At the same time, the bronchial features are traversed, and multiple edge contours of the bronchial features are marked according to the traversal of the bronchial features. Multiple edge parts of the bronchus are determined based on the edge detection of multiple edge contours. This approach incorporates the overall consideration of edge detection of multiple edge contours, ensuring the accuracy of multiple edge parts of the bronchus.

[0066] At this point, image preprocessing techniques are used to enhance bronchial features in the bronchial region, making them clearer and easier to identify. Image enhancement, denoising, and contrast adjustment algorithms, as well as bronchus-specific filtering techniques, are applied to improve image quality. Adaptive histogram equalization is used to enhance image contrast, or a Gaussian filter is used to smooth the image and reduce noise. For bronchial features, morphological operations (such as dilatation and erosion) are applied to highlight the bronchial wall or branching structures.

[0067] In the preprocessed image, the traversal is performed along the direction of the bronchi to facilitate subsequent edge contour marking. At this time, an image traversal algorithm, such as depth-first search (DFS) or breadth-first search (BFS), is used to traverse along the continuous features of the bronchi, starting from the known bronchial origin or feature points. During the traversal, the traversal path needs to be adjusted according to the curvature and direction of the bronchi to ensure accurate tracking of the entire bronchial structure. This requires combining image gradient information or edge detection results to guide the traversal process.

[0068] During the traversal, the edge contours of bronchial features are marked, providing a basis for subsequent edge detection. At this point, when traversing bronchial features, edge detection algorithms (such as the Canny edge detector) or morphological gradient methods are used to identify and mark the edge contours. Edge detection algorithms can identify points in the image with significant brightness changes, which usually correspond to the edges of the bronchial walls. When marking edge contours, it is necessary to ensure the integrity and continuity of the contours for subsequent analysis.

[0069] Based on the marked edge contours, multiple specific edge parts of the bronchus are identified, such as the bronchial wall and branching points. At this point, the marked edge contours are further analyzed, such as contour tracking and shape matching, to identify different edge parts of the bronchus. Simultaneously, shape matching is performed by combining prior knowledge of the bronchus (such as shape and size), or contour analysis algorithms are used to identify specific edge features (such as the sharpness of branching points and the smoothness of the bronchial wall).

[0070] Specifically, suppose a doctor is using image processing software to analyze a preprocessed image of the bronchial region. In step S122, the doctor first applies adaptive histogram equalization to enhance image contrast and uses morphological operations to highlight bronchial wall features. Next, starting from the known bronchial origin, the doctor uses a depth-first search algorithm to traverse along the direction of the bronchus. During the traversal, the doctor uses a Canny edge detector to identify and mark the edge contours of bronchial features, which clearly delineate the edges of the bronchial walls.

[0071] The doctor further analyzed the marked edge contours; through contour tracking and shape matching algorithms, the doctor identified several specific edge parts of the bronchus, including the bronchial wall and branching points; for example, the doctor found a distinct branching point with a sharp edge contour. This example demonstrates the application of step S122 in actual clinical examination; through image preprocessing, traversing bronchial features, marking edge contours, and identifying edge parts, the doctor can accurately identify and analyze the structural features of the bronchus, providing strong support for subsequent diagnosis and treatment.

[0072] Therefore, the non-bronchial region is determined by comparing the bronchial region with the bronchial fluorescence image, the environmental region is determined by the color regions of the non-bronchial region and the bronchial region, and the peripheral environment of the bronchus is determined by the identification of the environmental region. This method takes into account the overall consideration of the color regions of the non-bronchial region and the bronchial region, ensuring the accuracy of the environmental region.

[0073] At this point, the non-bronchial region is distinguished by comparing the bronchial region with the entire bronchial fluorescence image. Then, region segmentation or classification algorithms in image processing are used to differentiate between the bronchial and non-bronchial regions based on their feature differences in the image (such as brightness, color, texture, etc.). Simultaneously, one or more feature spaces are defined to describe the features of the bronchial and non-bronchial regions. Then, clustering algorithms (such as K-means, DBSCAN, etc.) or classification algorithms (such as support vector machines, neural networks, etc.) are applied to segment or classify these feature spaces. Finally, the location and extent of the non-bronchial region are determined based on the algorithm's output.

[0074] In the nonbronchial region, the environmental area, i.e., the specific tissues or structures around the bronchi, is further determined based on color characteristics. At this point, color space conversion and color segmentation techniques are used to map the color characteristics of the nonbronchial region to a specific color space, and segmentation is performed according to methods such as color thresholds or color histograms. Commonly used color spaces include RGB, HSV, and Lab. After selecting a suitable color space, the color histogram of the nonbronchial region is calculated, and color thresholds are determined based on the peak or valley values ​​of the histogram. Then, these thresholds are applied to segment the nonbronchial region into different color regions, each corresponding to a specific tissue or structure.

[0075] Based on the identified environmental region, a comprehensive analysis of the bronchial periphery is performed, including the health status of surrounding tissues and the presence of lesions or abnormalities. At this point, image processing techniques are used for further analysis and interpretation of the environmental region, involving advanced image processing techniques such as morphological manipulation, texture analysis, and feature extraction. Optionally, morphological manipulation can be used to detect abnormal structures (such as masses or nodules) in the environmental region, or texture analysis can be used to assess the homogeneity and heterogeneity of surrounding tissues. Furthermore, specific features (such as area, perimeter, and roundness) are extracted to quantify the attributes of the environmental region and compared with medical standards to determine the presence of lesions or abnormalities.

[0076] Specifically, suppose a doctor is using advanced image processing software to analyze a bronchial fluorescence image. In steps S123, the doctor first determines the location and extent of non-bronchial regions by comparing the bronchial region with the entire image. These non-bronchial regions include surrounding tissues such as lung tissue, blood vessels, and airway walls. The doctor then uses color space conversion and color segmentation techniques to map the color features of the non-bronchial regions into the HSV color space and determines color thresholds based on the color histogram. Finally, these thresholds are applied to segment the non-bronchial regions into different color regions, each corresponding to a specific tissue or structure, such as red lung tissue and blue blood vessels.

[0077] Combining prior medical knowledge with image processing techniques, the doctors further analyzed the surrounding environment. Through morphological manipulation, they detected an abnormal structure near the bronchus, similar in shape and size to a pulmonary nodule. Furthermore, they used texture analysis to assess the homogeneity and heterogeneity of the surrounding lung tissue, discovering that the texture characteristics of this area differed from normal lung tissue. Based on these analyses, the doctors determined an abnormality in the peribronchial environment and suspected the abnormal structure was a pulmonary nodule. This discovery is crucial for subsequent diagnosis and treatment, as pulmonary nodules are one of the early signs of lung cancer.

[0078] In one embodiment of this application, a color region matching table is collected, as shown in Table 1:

[0079] Table 1 Color Area Matching Table

[0080]

[0081] In this color area matching table, different tissues or structures are distinguished according to the value range of the HSV color space; for example, the red area corresponds to lung tissue, the blue area corresponds to blood vessels, and the green area corresponds to airway walls or other tissues.

[0082] In this example, the lung tissue region, vascular region, and airway wall region were assigned weights of 0.4, 0.3, and 0.3, respectively, and scores were given based on their health status; the final calculated composite score was 83, indicating that the peripheral environment of the bronchi was generally healthy, but there was mild inflammation in the airway wall region.

[0083] refer to Figure 4 In step S13, the bronchial boundary is determined based on the identification of multiple edge portions of the bronchus, and the actual morphology of the bronchus is determined based on the bronchial boundary and the bronchial fluorescence image.

[0084] In the specific implementation of this invention, the specific steps are as follows:

[0085] S131: Collect multiple edge portions of the bronchus, determine multiple edge segments based on the synchronous detection of multiple edge portions of the bronchus; form a continuous whole segment based on the synthesis of multiple edge segments, and use the continuous whole segment as the boundary of the bronchus.

[0086] S132: Match the bronchial boundary to the bronchial fluorescence image and output the bronchial morphological region;

[0087] S133: Mark the corners of the bronchial morphological region and determine the actual region of the bronchus based on the optimization of the corners of the bronchial morphological region; determine the actual morphology of the bronchus based on the identification of the actual region of the bronchus.

[0088] In the embodiments of this application, multiple edge portions of the bronchus are collected, and multiple edge segments are determined based on the synchronous detection of the multiple edge portions of the bronchus. A continuous overall line segment is formed based on the synthesis of the multiple edge segments. This continuous overall line segment serves as the boundary of the bronchus, which takes into account the overall consideration of the synchronous detection of the multiple edge portions of the bronchus and ensures the accuracy of the multiple edge segments.

[0089] At this point, image processing techniques are used to extract the bronchial edge portions from the bronchial image. These edge portions typically correspond to the junction between the bronchial wall and surrounding tissues. Edge detection algorithms, such as the Sobel operator and the Canny edge detector, are then used to process the bronchial image. These algorithms can identify areas in the image where brightness or color changes significantly, thereby determining the bronchial edge. Before applying the edge detection algorithm, the image needs to be preprocessed, such as denoising and contrast enhancement, to improve the accuracy of edge detection. The output of the edge detection algorithm is usually a set of edge points or edge line segments, which constitute the bronchial edge portion.

[0090] After acquiring multiple edge portions of the bronchus, these edge portions are simultaneously detected to determine multiple edge segments. These segments can more accurately describe the shape and boundary of the bronchus. At this point, line segment detection or contour tracking algorithms in image processing are used to connect the acquired edge portions into continuous line segments. These line segments should be as close as possible to the actual boundary of the bronchus and maintain coherence with each other. Line segment detection or contour tracking algorithms involve complex image processing techniques, such as Hough transform and dynamic programming. In practical applications, it is necessary to select an appropriate algorithm based on the specific characteristics of the bronchial image and adjust the algorithm parameters to obtain the best results.

[0091] Multiple edge segments are merged into a single, coherent line segment to form a clear boundary of the bronchus. This boundary is used for subsequent tasks such as image analysis, lesion detection, or 3D reconstruction. At this stage, image processing algorithms such as segment merging or contour smoothing are used to connect multiple edge segments into a coherent whole. During the merging process, it is necessary to ensure the coherence and smoothness of the line segments to avoid broken or jagged boundaries. Furthermore, segment merging or contour smoothing algorithms involve various image processing techniques, such as morphological operations and interpolation algorithms. In practical applications, it is necessary to select an appropriate algorithm based on the specific characteristics of the bronchial image and the requirements of subsequent tasks, and make necessary parameter adjustments.

[0092] Specifically, suppose we are using advanced medical image processing software to analyze a bronchoscopic image. In step S131, the image is first processed using the Canny edge detection algorithm to acquire multiple edge portions of the bronchus. These edge portions are presented as edge points distributed around the bronchial wall. Next, a contour tracking algorithm is used to connect these edge points into continuous line segments. These line segments represent the shape and boundary of the bronchus in the image. However, due to factors such as image noise and uneven lighting, these line segments are not completely continuous or smooth.

[0093] To address this issue, line segment merging and contour smoothing algorithms are used to process these line segments. These algorithms can identify and connect adjacent line segments while smoothing irregular parts on the line segments. After processing, a coherent and smooth overall line segment is obtained, which accurately describes the boundary of the bronchus. This boundary is used for subsequent image analysis tasks, such as lesion detection and 3D reconstruction. For example, this boundary is used to extract the geometric features of the bronchus, such as length and diameter, to further assess the health status of the bronchus. In addition, this boundary also serves as a benchmark for 3D reconstruction, helping to build a 3D model of the bronchus for more in-depth medical research and treatment planning.

[0094] Furthermore, multiple edge portions of the bronchus are collected, and multiple edge segments are determined based on the synchronous detection of these multiple edge portions. A coherent overall line segment is formed by synthesizing these multiple edge segments, and this coherent overall line segment serves as the boundary of the bronchus. The introduction of a coherent overall line segment as the boundary of the bronchus ensures the accuracy of bronchus boundary identification and takes into account both the bronchus boundary and the overall bronchus fluorescence image, further improving the accuracy of the actual morphology of the bronchus.

[0095] At this point, multiple edge segments of the bronchus are collected. After collecting these edge segments, they need to be connected into continuous line segments to more accurately describe the shape and boundary of the bronchus. Image processing algorithms such as contour tracking or line fitting are used to connect the detected edge points into line segments. These line segments should closely approximate the actual boundary of the bronchus and maintain continuity. Contour tracking algorithms typically start from a starting point and gradually track the edge points until they return to the starting point or reach a predetermined termination condition. Line fitting algorithms use methods such as least squares and Hough transform to fit the edge points into straight or curved line segments.

[0096] Multiple edge segments are combined into a single, coherent line segment to form a clear, continuous boundary for the bronchi. This is achieved using image processing algorithms such as segment merging or contour smoothing to connect adjacent segments and smooth transitions between them. These algorithms ensure that the resulting boundary segments are both coherent and smooth. Segment merging algorithms involve steps such as overlap detection and endpoint matching between segments; contour smoothing algorithms use morphological operations and filters to smooth irregularities on the segments.

[0097] Specifically, suppose we are using medical image processing software to analyze a high-resolution bronchial CT image; the Canny edge detection algorithm is used to process the image to identify the edges of the bronchi; during the processing, the threshold parameters of the algorithm are adjusted to ensure that clear edge information can be detected.

[0098] After acquiring edge information, a contour tracking algorithm is used to connect these edge points into continuous line segments. These line segments represent the shape and boundary of the bronchus in the image. Due to noise and artifacts in the image, some line segments are not completely continuous or smooth. In order to obtain a continuous and smooth bronchial boundary, line segment merging and contour smoothing algorithms are used to process the detected line segments. These algorithms can identify and connect adjacent line segments while smoothing irregular parts on the line segments. After processing, a clear bronchial boundary line segment is obtained, which accurately describes the shape and location of the bronchus.

[0099] This boundary segment is used for subsequent image analysis tasks, such as 3D reconstruction and lesion detection. For example, this boundary segment can be used to extract the geometric features of the bronchus, such as length and diameter, in order to further assess the health status of the bronchus. In addition, this boundary segment also serves as a benchmark for 3D reconstruction, helping to build a 3D model of the bronchus for more in-depth medical research and treatment planning.

[0100] In one embodiment of this application, a preset matching line segment ID matching table is collected. The matching line segment ID matching table records the similarity between different edge line segments, thereby helping to determine which line segments should be merged; the matching line segment ID matching table is shown in Table 2:

[0101] Table 2 Matching Line Segment IDs Table

[0102]

[0103] In this matching line segment ID matching table, the line segment ID is a unique identifier, the start point coordinates and end point coordinates define the endpoints of the line segment, the similarity score represents the similarity between the current line segment and other line segments (calculated based on factors such as distance, direction, and length), and the matching line segment ID represents the ID of the other line segment that is most similar to the current line segment.

[0104] refer to Figure 5 In step S14, the core re-examination area is determined based on the actual morphology of the bronchus and the user's age, and the abnormal feature combination is determined based on the identification of the core re-examination area.

[0105] In the specific implementation of this invention, the specific steps are as follows:

[0106] S141: Based on the actual morphology of the bronchi, multiple branch tubes are determined and their actual morphology is marked. The multiple branch tubes are the various parts of the bronchi.

[0107] S142: In each branch pipe body, the corresponding surface abnormality features are determined based on the detection of the branch pipe body. Multiple abnormal areas are determined based on the actual shape of multiple branch pipe bodies and the corresponding surface abnormality features. The core re-examination area is determined based on the multiple abnormal areas, the user's age, and the re-examination mapping relationship.

[0108] S143: Perform surface identification and structural identification on the core review area, determine the first sub-anomaly feature based on the surface identification of the core review area, determine the second sub-anomaly feature based on the structural identification of the core review area, and determine the combination of anomaly features based on the first sub-anomaly feature and the second sub-anomaly feature.

[0109] In the embodiments of this application, multiple branch tubes are determined based on the actual morphology of the bronchus, and the actual morphology of the multiple branch tubes is marked. The multiple branch tubes are introduced as various parts of the bronchus.

[0110] At this point, the actual morphology of the bronchial branches is identified and delineated to break them down into multiple manageable branch bodies. Medical imaging techniques, such as computed tomography (CT) or magnetic resonance imaging (MRI), are used to acquire three-dimensional images of the bronchi. Then, image processing algorithms (such as region growing, thresholding, and morphological manipulation) are used to automatically identify the main bronchial trunk and its branches. During the delineation process, the complexity and variability of the bronchi must be considered; the morphology of the bronchi varies due to individual differences, disease states, or scanning conditions. Therefore, the algorithm needs to be flexible and robust to adapt to different situations.

[0111] After identifying the main trunk and branches of the bronchi, these parts need to be determined as multiple independent branch tubes. At this point, based on the morphological characteristics and connections of the bronchi, the main trunk and branches are divided into multiple continuous tube segments. These segments are straight segments, curved segments, or complex structures at branch points. When determining the branch tubes, it is necessary to ensure the integrity and continuity of each tube segment. This requires the use of techniques such as image registration, interpolation, or smoothing to process noise, artifacts, or discontinuous areas in the image.

[0112] Each identified branch tube is marked, and its actual morphological information is recorded. At this time, image processing software or database system is used to assign a unique identifier to each branch tube and record its morphological characteristics such as position, length, diameter, and branch angle. This information is used for subsequent image analysis, lesion detection, or three-dimensional reconstruction. During the marking process, it is necessary to ensure the accuracy and consistency of the information, which requires the use of standardization or calibration techniques to process the size and position information in the image.

[0113] Specifically, suppose we are using advanced medical image analysis software to analyze a high-resolution bronchial CT image; the software's automatic recognition function is used to identify the main bronchus and its branches; the software uses advanced image processing algorithms to detect the edges and connections of the bronchi, thereby dividing them into multiple parts; after identifying the main bronchus and its branches, the software's segmentation function is used to determine these parts as multiple independent branch tubes; each branch tube is a continuous tube segment with unique morphological features and connections.

[0114] The software's labeling function assigns a unique identifier to each bronchial tube and records its actual morphological information, including the location, length, diameter, and branching angle of each tube. This information will be used for subsequent image analysis tasks, such as lesion detection and 3D reconstruction. For example, during the labeling process, an abnormally large diameter of a bronchial tube may be detected, which could be due to a tumor or inflammation. Recording this information provides strong support for subsequent diagnosis and treatment. At the same time, this information is also used to construct a 3D model of the bronchus for more in-depth medical research and treatment planning.

[0115] Furthermore, within each branch pipe, the corresponding surface abnormality features are determined based on the detection of that branch pipe. Multiple abnormal areas are determined based on the actual shape of multiple branch pipes and their corresponding surface abnormalities. The core review area is determined based on the multiple abnormal areas, the user's age, and the review mapping relationship. This comprehensive approach takes into account the multiple abnormal areas, the user's age, and the review mapping relationship, ensuring the accuracy of the core review area.

[0116] At this point, abnormal features on the surface of each branch tube are identified, indicating potential lesions or abnormalities;

[0117] Methods: Medical imaging analysis techniques, such as computed tomography (CT) or magnetic resonance imaging (MRI), combined with image processing algorithms (such as filtering, edge detection, texture analysis, etc.), are used to detect minute changes or irregularities on the surface of the branch tubes. Abnormal features include nodules, masses, stenosis, thickening, calcification, etc. The algorithm needs to be able to distinguish between normal anatomical structures and abnormal features, and accurately quantify the size, shape and location of these features.

[0118] After identifying surface anomalies, it is necessary to determine which areas have anomalies by considering the actual shape of the branch pipe. At this point, the diameter, length, curvature of the branch pipe, as well as the type, size, and location of the surface anomalies are taken into account, and image processing or machine learning algorithms are used to identify the anomalies. Simultaneously, 3D reconstruction technology is used to visualize the internal structure of the branch pipe and to conduct a comprehensive analysis in conjunction with the surface anomalies. The determination of anomalies requires consideration of multiple factors, such as the number, density, and distribution of anomalies.

[0119] After identifying multiple abnormal areas, it is necessary to combine the user's age information and the follow-up mapping relationship to determine which areas are the core follow-up areas, i.e., areas that require special attention. At this point, clinical guidelines, statistical data, or expert experience are used to establish a follow-up mapping relationship, associating factors such as the user's age, gender, and medical history with the risk level of the abnormal areas. Then, the core follow-up areas are determined based on these risk levels. At the same time, the follow-up mapping relationship is a pre-set model that takes into account the interaction between multiple factors. When determining the core follow-up areas, it is necessary to weigh the importance of different factors and consider the potential progression or deterioration of the disease.

[0120] Specifically, suppose a bronchial CT scan is being analyzed on a 60-year-old male patient. CT scan images and image processing algorithms are used to detect abnormal features on the surface of each branch bronchus. For example, in a branch of the right main bronchus, a nodule with a diameter of 5 mm is detected, which is irregular in shape and has blurred edges. Combining the actual morphology of the branch bronchus with the characteristics of the detected nodule, the area where the nodule is located is identified as an abnormal region. At the same time, it is also noted that the diameter of the branch bronchus is slightly increased, which is due to luminal narrowing caused by the nodule. Therefore, this area is marked as one of the abnormal regions.

[0121] Considering the patient's age of 60, belonging to the middle-aged and elderly population, and the nodule characteristics meeting certain high-risk lesion criteria (such as irregular shape and blurred edges), the area where the nodule is located was identified as the core follow-up area based on the follow-up mapping relationship. This means that this area will require special attention in subsequent follow-up examinations or treatments in order to promptly detect and treat potential lesions. Through the analysis of step S142, abnormal areas in the bronchi can be identified more accurately, and the core follow-up area can be determined by combining the patient's age and the follow-up mapping relationship, thereby providing strong support for subsequent diagnosis and treatment.

[0122] Therefore, surface and structural identification are performed on the core review area. The first sub-anomaly feature is determined based on the surface identification of the core review area, and the second sub-anomaly feature is determined based on the structural identification of the core review area. The combination of anomaly features is determined based on the first and second sub-anomaly features, which takes into account the overall consideration of the first and second sub-anomaly features and ensures the accuracy of the combination of anomaly features.

[0123] At this point, a detailed analysis of the surface features of the core re-examination area is performed to identify any abnormalities or irregularities. High-resolution medical imaging (such as CT or MRI) and image processing techniques are then used to conduct a thorough surface analysis of the core re-examination area. This includes using 3D reconstruction techniques to visualize the surface and applying algorithms such as texture analysis and edge detection to identify subtle surface changes. Simultaneously, surface recognition involves image preprocessing, such as denoising and contrast enhancement, to improve the accuracy of the analysis. Furthermore, the impact of image resolution and scanning conditions on the analysis results must be considered.

[0124] Based on surface identification, the first sub-abnormal feature related to the core re-examination area is determined. At this time, based on the results of surface identification, any features that do not conform to the normal anatomical structure, such as surface irregularities, nodules, masses, ulcers, etc., are identified and these features will be recorded as the first sub-abnormal feature. Determining the first sub-abnormal feature requires combining clinical experience and professional knowledge to distinguish between normal variations and true abnormalities. In addition, the size, shape, location, and relationship of the abnormal feature with surrounding tissues also need to be considered.

[0125] The internal structure of the core re-examination area is analyzed to detect any structural abnormalities or lesions. At this time, medical imaging analysis techniques, such as 3D reconstruction, volume rendering, and morphological analysis, are used to conduct a detailed analysis of the internal structure of the core re-examination area. This involves the assessment of bronchial walls, lumens, branch structures, etc. At the same time, structural identification needs to consider the impact of factors such as image resolution, contrast, and noise on the analysis results. In addition, clinical guidelines and professional knowledge are needed to interpret the analysis results.

[0126] Based on structural identification, the second sub-abnormal features related to the core re-examination area are determined. At this time, based on the results of structural identification, any structural abnormalities, such as thickened vessel walls, narrowed lumens, abnormal branch structures, etc., are identified. These features will be recorded as second sub-abnormal features. Determining the second sub-abnormal features requires consideration of the range, degree, relationship with surrounding tissues, and etiology of the abnormal features.

[0127] By combining the first and second sub-abnormal features, a complete combination of abnormal features is formed to more comprehensively describe the abnormalities in the core re-examination area. At this point, the first and second sub-abnormal features are summarized and classified, and the combination of abnormal features is determined based on their nature, location, and interrelationships. This involves quantitative assessment of abnormal features, etiological inference, and risk assessment. Determining the combination of abnormal features requires combining clinical experience and professional knowledge to ensure the accuracy and reliability of the analysis. In addition, the interactions and potential impacts between abnormal features also need to be considered.

[0128] Specifically, suppose we are analyzing a patient's bronchial CT scan images and have identified a suspected nodule area in the right main bronchus as the core area for review. High-resolution CT images and 3D reconstruction techniques are used to visualize the surface of this area. Through careful observation and analysis, it is found that the surface of this area is uneven and contains a small nodular protrusion. Based on the surface identification results, the first sub-abnormal feature of this area is determined to be "uneven surface and presence of a nodular protrusion," a feature suggesting the presence of some abnormality or lesion in this area.

[0129] Further analysis of the internal structure of the region revealed that the bronchial wall around the nodule was thickened and the lumen was slightly narrowed. These structural changes corroborated the first sub-abnormal feature, further supporting the hypothesis that there was an abnormality or lesion in the region. Based on the results of structural identification, the second sub-abnormal feature of the region was determined to be "thickened bronchial wall and narrowed lumen". This feature provides more information about the nature of the abnormality or lesion.

[0130] Combining the first and second sub-abnormal features, a complete abnormal feature combination was formed: "uneven surface with nodular protrusions, thickened vessel wall and narrowed lumen". This abnormal feature combination provides a comprehensive description of the abnormality in the core re-examination area and provides strong support for subsequent diagnosis and treatment.

[0131] refer to Figure 6 In step S15, abnormal events are determined based on the combination of abnormal features and the core review area, and the bronchial status level is determined based on the abnormal events, the actual morphology of the bronchus, and the peripheral environment of the bronchus.

[0132] In the specific implementation of this invention, the specific steps are as follows:

[0133] S151: Collect abnormal feature combinations, determine the corresponding abnormal list based on the matching of abnormal feature combinations and the actual morphology of the bronchi, and present the various parts of the bronchi; determine abnormal events based on the abnormal list and the core re-examination area.

[0134] S152: Collect the peripheral environment of the bronchus, determine the first state coefficient based on the peripheral environment of the bronchus and the abnormal event, and determine the second state coefficient based on the peripheral environment of the bronchus and the actual morphology of the bronchus.

[0135] S153: Match the corresponding state level mapping relationship based on the actual morphology of the bronchus and the previous detection information of the bronchus, and determine the state level of the bronchus based on the state level mapping relationship, the first state coefficient and the second state coefficient.

[0136] In the embodiments of this application, abnormal feature combinations are collected, and a corresponding abnormal list is determined based on the matching of abnormal feature combinations with the actual morphology of the bronchi. This abnormal list presents various parts of the bronchi. Abnormal events are determined based on the abnormal list and the core review area, which takes into account the overall consideration of the abnormal list and the core review area, and ensures the accuracy of abnormal events.

[0137] At this point, the abnormal feature combinations identified in the previous steps are collected. These feature combinations describe the abnormalities or lesions present in the bronchi. At this point, the abnormal feature combinations that have been identified in the previous analysis steps (such as S143) are obtained. These features include nodules, masses, thickened bronchial walls, stenosis of the lumen, etc. Ensure that the collected abnormal feature combinations are accurate, complete, and match the patient's medical imaging data.

[0138] The collected abnormal feature combinations are matched with the actual morphology of the bronchi to identify abnormalities in various parts of the bronchi. At this point, medical imaging analysis techniques, such as 3D reconstruction and volume rendering, are used to locate the abnormal feature combinations to specific locations within the bronchi. This requires considering the anatomical structure and branching of the bronchi to determine the specific location of the abnormality. The matching process needs to consider the size, shape, location, and relationship with surrounding tissues of the abnormal features to ensure the accuracy and completeness of the abnormality list.

[0139] After identifying abnormalities in various parts of the bronchi, the specific abnormal event is further determined by combining information from the core re-examination area. At this point, the abnormality list is analyzed to identify the abnormal features most relevant to the core re-examination area. Combined with the patient's medical history, symptoms, and other information, the specific abnormal event is determined. This requires considering factors such as the nature, size, and rate of progression of the abnormal feature. When determining the abnormal event, multiple factors need to be considered comprehensively to ensure the accuracy and reliability of the diagnosis. In addition, the etiology and pathophysiological mechanisms also need to be considered to provide guidance for subsequent treatment.

[0140] Specifically, suppose a patient undergoes a bronchial CT scan and the following abnormal features are identified: a nodular protrusion in the middle segment of the right main bronchus, and the wall of the bronchus in this area is thickened; the analysis results show that the two abnormal features are "nodular protrusion in the middle segment of the right main bronchus" and "thickened wall".

[0141] Using 3D reconstruction technology, these two abnormal features were located in the middle segment of the right main bronchus. The abnormality list showed a nodular protrusion in this area and thickening of the bronchial wall, which are associated with a certain lesion. Combining the patient's medical history (such as long-term smoking history), symptoms (such as cough, sputum production, dyspnea, etc.) and the information in the abnormality list, the abnormal event of this patient was determined to be "suspected lung cancer in the middle segment of the right main bronchus". The core re-examination area is the middle segment of the right main bronchus, and the nodular protrusion and thickening of the bronchial wall in this area are key evidence to support this diagnosis.

[0142] Furthermore, the peripheral environment of the bronchus is collected, and a first state coefficient is determined based on the peripheral environment of the bronchus and abnormal events. A second state coefficient is determined based on the peripheral environment of the bronchus and the actual morphology of the bronchus. This comprehensive consideration of the peripheral environment of the bronchus and the actual morphology of the bronchus ensures the accuracy of the second state coefficient.

[0143] At this point, the peribronchial environment is collected to assess the impact of the abnormal event on the peribronchial environment, thereby determining the first state coefficient. Imaging data of the peribronchial environment are then analyzed to assess whether the abnormal event (such as a tumor or inflammation) has caused compression, infiltration, or damage to surrounding tissues. Based on the degree and extent of the impact, a predetermined scoring standard or model is used to determine the first state coefficient. When determining the first state coefficient, factors such as the nature, size, location, and rate of progression of the abnormal event need to be considered. The scoring standard or model is developed based on clinical experience, research results, or expert consensus.

[0144] The second state coefficient is determined by assessing the coordination between the actual morphology of the bronchus and its surrounding environment. This involves comparing the actual morphology of the bronchus (e.g., lumen size, wall thickness, branching structure) with the characteristics of its surrounding environment (e.g., surrounding tissue density, vascular distribution, lymph node size). Based on the coordination and consistency between the two, a predetermined scoring standard or model is used to determine the second state coefficient. When determining the second state coefficient, the interaction and mutual influence between the bronchus and its surrounding environment must be considered. The scoring standard or model involves assessments of multiple aspects, such as morphological matching, functional coordination, and pathophysiological consistency.

[0145] Specifically, suppose a patient undergoes a bronchial CT scan and is diagnosed with "suspected lung cancer in the middle segment of the right main bronchus" as an abnormal event; the CT scan obtains images of the bronchus and its surrounding environment; the analysis shows that there is a nodular protrusion in the middle segment of the right main bronchus, and the wall of the bronchus in this area is thickened; at the same time, the density of the surrounding lung tissue is increased, some blood vessels are compressed, and there are signs of enlarged lymph nodes.

[0146] Assess the impact of an abnormal event (suspected lung cancer) on the peribronchial environment; the abnormal event had a significant impact on the peribronchial environment due to the compression of surrounding tissues by nodular protrusions and signs of lymph node enlargement; the first state coefficient was determined to be 0.7 (indicating moderate impact) according to the predetermined scoring criteria.

[0147] The analysis compared the actual morphology of the bronchus with its surrounding environment. The analysis revealed inconsistencies between the nodular protrusions and thickened walls of the mid-segment of the right main bronchus and the surrounding increased lung tissue density, vascular compression, and lymph node enlargement. This indicated poor coordination between the actual morphology of the bronchus and its surrounding environment. Based on a predetermined scoring standard, the second state coefficient was determined to be 0.5 (indicating poor coordination). This example demonstrates how step S152 combines medical imaging analysis, abnormal events, and information from the surrounding environment to determine two key coefficients (the first state coefficient and the second state coefficient) for bronchial status. These coefficients provide crucial information for subsequent status assessment and treatment planning.

[0148] Therefore, by matching the actual morphology of the bronchus with the corresponding state level mapping relationship based on the previous detection information of the bronchus, and determining the state level of the bronchus based on the state level mapping relationship, the first state coefficient, and the second state coefficient, the overall consideration of the state level mapping relationship, the first state coefficient, and the second state coefficient is taken into account, ensuring the accuracy of the bronchus state level. At the same time, the overall consideration of abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus is taken into account, further ensuring the accuracy of the identification of the bronchus state level.

[0149] At this point, a state level mapping relationship is found that matches the current actual morphology of the bronchus with previous test information. This state level mapping relationship is typically a pre-defined table or model used to associate the morphological characteristics and test information of the bronchus with a specific state level. Information on the current actual morphology of the bronchus is collected, including lumen size, wall thickness, and branching structures. Then, previous test information is reviewed, such as historical lesion progression, treatment response, and imaging changes. This information is compared with the pre-defined state level mapping relationship to find the best match. Simultaneously, multiple factors need to be considered during the matching process, including the trend of bronchial morphological changes and the reliability and consistency of previous test information. Since the state level mapping relationship is based on clinical experience, research results, or expert consensus, its accuracy and applicability must be ensured.

[0150] After establishing a state level mapping relationship that matches the actual bronchial morphology and previous test information, the bronchial state level is further determined by combining the first and second state coefficients. At this point, the first and second state coefficients are substituted into the matched state level mapping relationship, and the bronchial state level is calculated according to the rules or algorithms within the mapping relationship. The state level is a numerical value, classification label, or descriptive term used to represent the current state or degree of lesion of the bronchus. Simultaneously, when determining the state level, the weights and interrelationships of the first and second state coefficients need to be considered. These coefficients reflect different aspects of information, such as the impact of abnormal events on the surrounding environment and the coordination between the bronchial morphology and the surrounding environment. Therefore, when calculating the state level, it is necessary to ensure that these coefficients are reasonably considered and balanced.

[0151] Specifically, suppose a patient, after undergoing a series of tests, has the following information determined:

[0152] Actual morphology of the bronchus: The lumen of the middle segment of the right main bronchus is narrowed and the wall is thickened;

[0153] Previous examination information: Last year's CT scan showed slight thickening of the vessel wall in this area, and this year the stenosis of the lumen has worsened;

[0154] First state coefficient: 0.6 (indicating that the abnormal event has a slight impact on the surrounding environment);

[0155] Second state coefficient: 0.4 (indicating poor coordination between bronchial morphology and the surrounding environment);

[0156] Collect actual bronchial morphology information: narrowing of the lumen and thickening of the wall in the middle segment of the right main bronchus; review previous test information: slight wall thickening last year, and the narrowing has worsened this year; compare with the preset state level mapping relationship: find the mapping relationship that matches this information, which indicates that the luminal narrowing and wall thickening are associated with a specific state level.

[0157] The first state coefficient (0.6) and the second state coefficient (0.4) are substituted into the matched state level mapping relationship; the state level of the bronchus is calculated according to the rules or algorithms in the mapping relationship; for example, if the mapping relationship stipulates that luminal stenosis and wall thickening are associated with the state level "moderate lesion", and the state level is "moderate lesion" when the first state coefficient and the second state coefficient are in the range of 0.5-0.7, then the bronchial state level of this patient is determined to be "moderate lesion"; through this example, we can see how step S153 combines the actual morphology of the bronchus, previous test information, and the first and second state coefficients to determine the bronchial state level. This step provides an important basis for subsequent treatment planning and prognostic assessment.

[0158] In one embodiment of this application, a matching table of state level mapping relationships is collected, as shown in Table 3:

[0159] Table 3: Matching Table for Status Level Mapping Relationships

[0160]

[0161] Now, here is the information for a patient:

[0162] Actual morphology of the bronchus: narrow lumen and thickened wall;

[0163] Previous test information: The lumen was slightly narrowed last year, but the narrowing has worsened this year, and the lumen wall has thickened.

[0164] First state coefficient: 0.6;

[0165] Second state coefficient: 0.4;

[0166] Based on the matching table of the status level mapping relationship, the row that best matches the patient's information is found: luminal stenosis, bronchial wall thickening; the degree of stenosis gradually worsens, and the bronchial wall thickening; the first status coefficient ranges from 0.5 to 0.7; the second status coefficient ranges from 0.3 to 0.6; therefore, the patient's bronchial status level is "moderate lesion".

[0167] Please see Figure 7 , Figure 7 This is a schematic diagram of the structural composition of the bronchial fluorescence image boundary recognition system based on visual detection according to an embodiment of the present invention; the bronchial fluorescence image boundary recognition system based on visual detection includes:

[0168] The fluorescence image module 21 is used to acquire the position of the bronchus and trigger corresponding visual detection based on the position of the bronchus to acquire the fluorescence image of the bronchus.

[0169] Image detection module 22 is used to determine the bronchial region and the environmental region based on the detection of bronchial fluorescence images, and to determine multiple edge parts of the bronchus based on the edge detection of the bronchial region.

[0170] The morphology detection module 23 is used to determine the boundary of the bronchus based on the identification of multiple edge parts of the bronchus, and to determine the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image.

[0171] Abnormal feature module 24 is used to determine the core re-examination area based on the actual morphology of the bronchus and the user's age, and to determine the abnormal feature combination based on the identification of the core re-examination area;

[0172] The status level module 25 is used to determine abnormal events based on combinations of abnormal features and core review areas, and to determine the status level of the bronchus based on the abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus.

[0173] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for boundary recognition of bronchial fluorescence images based on visual detection, characterized in that, include: The location of the bronchi is acquired, and corresponding visual detection is triggered based on the location of the bronchi to acquire bronchial fluorescence images; The bronchial region and the surrounding area are determined based on the detection of bronchial fluorescence images, and multiple edge parts of the bronchus are determined based on the edge detection of the bronchial region. The bronchial boundary is determined by identifying multiple edge portions of the bronchus, and the actual morphology of the bronchus is determined based on the bronchial boundary and bronchial fluorescence image. This process includes: acquiring multiple edge portions of the bronchus; determining multiple edge segments based on simultaneous detection of these edge portions; forming a continuous overall line segment based on the synthesis of these multiple edge segments, which serves as the bronchial boundary; matching the bronchial boundary to the bronchial fluorescence image and outputting the bronchial morphological region; marking the corners of the bronchial morphological region and determining the actual bronchial region based on optimization of these corners; and determining the actual morphology of the bronchus based on the identification of the actual bronchial region. The core re-examination area is determined based on the actual morphology of the bronchi and the user's age, and the abnormal feature combination is determined based on the identification of the core re-examination area; Abnormal events are identified based on combinations of abnormal features and core re-examination areas. The bronchial status level is determined based on the abnormal events, the actual morphology of the bronchus, and the peripheral environment of the bronchus. This includes: collecting combinations of abnormal features; determining a corresponding abnormal list based on the matching of abnormal feature combinations with the actual morphology of the bronchus; and identifying abnormal events based on the abnormal list and core re-examination areas.

2. The boundary recognition method for bronchial fluorescence images based on visual detection according to claim 1, characterized in that, The location of the bronchus is acquired, and corresponding visual detection is triggered based on the location of the bronchus to acquire bronchial fluorescence images, including: The position of the bronchi is detected and determined based on the detected position. At the same time, multiple posture parameters of the user are collected and the user's current posture is determined based on the multiple posture parameters and the user's height. The detection area is determined based on the location of the bronchus and the user's current posture. The peripheral visual detection device is triggered based on the spatial position of the detection area, so that the visual detection device moves relative to the detection area to perform visual detection of the detection area and output the acquired bronchial fluorescence image.

3. The boundary recognition method for bronchial fluorescence images based on visual detection according to claim 1, characterized in that, The process of determining the bronchial region and the surrounding area based on bronchial fluorescence image detection, and determining multiple edge portions of the bronchus based on edge detection of the bronchial region, includes: The bronchial fluorescence image is acquired, and multiple branch features are determined based on the detection of the bronchial fluorescence image. The relative distance between two adjacent branch features is determined based on the position of the multiple branch features, and the bronchial region is determined based on the relative distance between two adjacent branch features and the connectivity between two adjacent branch features. In the bronchial region, bronchial features are highlighted based on image preprocessing of the bronchial region. At the same time, the bronchial features are traversed, and multiple edge contours of the bronchial features are marked according to the traversal of the bronchial features. Multiple edge portions of the bronchus are determined based on the edge detection of the multiple edge contours. Non-bronchial regions are determined by comparing bronchial regions with bronchial fluorescence images. Environmental regions are determined based on the color regions of non-bronchial and bronchial regions. The peripheral environment of the bronchi is determined based on the identification of environmental regions.

4. The boundary recognition method for bronchial fluorescence images based on visual detection according to claim 1, characterized in that, The core re-examination area is determined based on the actual morphology of the bronchi and the user's age. Abnormal feature combinations are then determined based on the identification of the core re-examination area, including: Based on the actual morphology of the bronchi, multiple branch tubes are identified and their actual morphology is marked. These multiple branch tubes are considered as various parts of the bronchi. In each branch pipe, the corresponding surface abnormality features are determined based on the detection of the branch pipe. Multiple abnormal areas are determined based on the actual shape of multiple branch pipes and their corresponding surface abnormalities. The core re-examination area is determined based on the multiple abnormal areas, the user's age, and the re-examination mapping relationship.

5. The boundary recognition method for bronchial fluorescence images based on visual detection according to claim 4, characterized in that, The process of determining the core re-examination area based on the actual morphology of the bronchi and the user's age, and determining abnormal feature combinations based on the identification of the core re-examination area, also includes: Surface and structure identification are performed on the core review area. The first sub-anomaly feature is determined based on the surface identification of the core review area, and the second sub-anomaly feature is determined based on the structure identification of the core review area. The combination of anomaly features is determined based on the first and second sub-anomaly features.

6. The boundary recognition method for bronchial fluorescence images based on visual detection according to claim 1, characterized in that, The method of identifying abnormal events based on combinations of abnormal features and core re-examination areas, and determining the bronchial status level based on the abnormal events, the actual morphology of the bronchus, and the peripheral environment of the bronchus, further includes: The peripheral environment of the bronchus is collected, and the first state coefficient is determined based on the peripheral environment of the bronchus and the abnormal event. The second state coefficient is determined based on the peripheral environment of the bronchus and the actual morphology of the bronchus. Based on the actual morphology of the bronchus and the previous detection information of the bronchus, the corresponding state level mapping relationship is matched, and the state level of the bronchus is determined according to the state level mapping relationship, the first state coefficient and the second state coefficient.

7. A boundary recognition system for bronchial fluorescence images based on visual detection, characterized in that, The visual detection-based bronchial fluorescence image boundary recognition system is applied to the visual detection-based bronchial fluorescence image boundary recognition method as described in any one of claims 1-6, wherein the visual detection-based bronchial fluorescence image boundary recognition system comprises: The fluorescence image module is used to acquire the location of the bronchi and trigger corresponding visual detection based on the location of the bronchi to acquire bronchial fluorescence images. The image detection module is used to determine the bronchial region and the environmental region based on the detection of bronchial fluorescence images, and to determine multiple edge parts of the bronchus based on the edge detection of the bronchial region. The morphology detection module is used to determine the bronchial boundary based on the identification of multiple edge portions of the bronchus, and to determine the actual morphology of the bronchus based on the bronchial boundary and bronchial fluorescence image. This includes: acquiring multiple edge portions of the bronchus; determining multiple edge segments based on the simultaneous detection of these edge portions; forming a continuous overall line segment based on the synthesis of these multiple edge segments, which serves as the bronchial boundary; matching the bronchial boundary to the bronchial fluorescence image and outputting the bronchial morphological region; marking the corners of the bronchial morphological region and determining the actual bronchial region based on the optimization of these corners; and determining the actual morphology of the bronchus based on the identification of the actual bronchial region. The abnormal feature module is used to determine the core re-examination area based on the actual morphology of the bronchi and the user's age, and to determine the abnormal feature combination based on the identification of the core re-examination area; The status level module is used to identify abnormal events based on combinations of abnormal features and core review areas, and to determine the status level of the bronchus based on the abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus. This includes: collecting combinations of abnormal features; determining a corresponding abnormal list based on the matching of abnormal feature combinations and the actual morphology of the bronchus, which presents various parts of the bronchus; and identifying abnormal events based on the abnormal list and core review areas.

Images