Bronchial fluorescence image boundary identification method and system based on visual detection

Abstract

The invention discloses a boundary identification method and system for a bronchus fluorescence image based on visual detection, and relates to the technical field of visual detection, and the method comprises the steps: determining a plurality of edge parts of a bronchus according to the edge detection of a bronchus region; the boundary of the bronchial tube is determined according to the recognition of the plurality of edge parts of the bronchial tube, and the actual form of the bronchial tube is determined based on the boundary of the bronchial tube and the bronchial tube fluorescence image, so that the recognition accuracy of the boundary of the bronchial tube is ensured, and the accuracy of the actual form of the bronchial tube is further improved. Therefore, a core review area is determined based on the actual form of the bronchus and the age of the user, and an abnormal feature combination is determined according to the recognition of the core review area; the abnormal event is determined according to the abnormal feature combination and the core review area, and the state level of the bronchial tube is determined according to the abnormal event, the actual form of the bronchial tube and the peripheral environment of the bronchial tube, so that the identification accuracy of the state level of the bronchial tube is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and in particular to a method and system for identifying the boundaries of bronchial fluorescence images based on visual detection. Background Art

[0002] With the development of science and technology, corresponding bronchial fluorescence images are needed for bronchi detection. Bronchial fluorescence images are formed by photographing the bronchi with visual detection components. Bronchial fluorescence images use fluorescence imaging technology to observe the actual morphology of the bronchi. In the existing technology, bronchial fluorescence images are collected and the boundaries of the bronchi are directly marked in the bronchial fluorescence images. The determination of the bronchial boundaries is not accurate, and the actual morphology of the bronchi determined based on the inference of the boundaries is also not accurate, which affects the accuracy of the recognition of the status level of the bronchus. Moreover, it is only based on conventional analysis of fluorescence images. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides a method and system for boundary recognition of bronchial fluorescence images based on visual detection.

[0004] An embodiment of the present invention provides a method for identifying boundaries of bronchial fluorescence images based on visual detection, comprising: Acquiring the location of the bronchus, and triggering corresponding visual detection based on the location of the bronchus to acquire a bronchial fluorescence image; determining a bronchial region and an environmental region based on detection of a bronchial fluorescence image, and determining a plurality of bronchial edge portions based on edge detection of the bronchial region; determining the boundary of the bronchus based on identification of a plurality of edge portions of the bronchus, and determining the actual morphology of the bronchus based on the boundary of the bronchus and a bronchial fluorescence image; Determine a core review area based on the actual morphology of the bronchus and the user's age, and determine an abnormal feature combination based on the identification of the core review area; Abnormal events are determined based on the combination of abnormal features and the core review area, and the bronchial status grade is determined based on the abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus.

[0005] An embodiment of the present invention provides a bronchial fluorescence image boundary recognition system based on visual detection. The bronchial fluorescence image boundary recognition system based on visual detection is applied to the above-mentioned bronchial fluorescence image boundary recognition method based on visual detection. The bronchial fluorescence image boundary recognition system based on visual detection includes: A fluorescence imaging module is used to acquire the location of the bronchi and trigger corresponding visual detection based on the location of the bronchi to acquire bronchial fluorescence images; An image detection module, configured to determine a bronchial region and an environmental region based on detection of a bronchial fluorescence image, and to determine multiple edge portions of the bronchus based on edge detection of the bronchial region; a morphology detection module, configured to determine the boundary of the bronchus based on identification of multiple edge portions of the bronchus, and determine the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image; An abnormal feature module is used to determine a core review area based on the actual morphology of the bronchus and the user's age, and to determine an abnormal feature combination based on the identification of the core review area; The status grade module is used to determine abnormal events based on the abnormal feature combination and the core review area, and to determine the status grade of the bronchus based on the abnormal events, the actual morphology of the bronchus and the surrounding environment of the bronchus.

[0006] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention first acquires the position of the bronchus, triggers corresponding visual detection based on the position of the bronchus to acquire a bronchial fluorescence image, determines the bronchial region and the environmental region based on the detection of the bronchial fluorescence image, and determines multiple edge portions of the bronchus based on the edge detection of the bronchial region; determines the boundary of the bronchus based on the identification of multiple edge portions of the bronchus, and determines the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image, thereby ensuring the accuracy of the identification of the boundary of the bronchus and being compatible with the overall consideration of the boundary of the bronchus and the bronchial fluorescence image, further improving the accuracy of the actual morphology of the bronchus; (2) The present invention determines a core review area based on the actual morphology of the bronchus and the age of the user, determines an abnormal feature combination based on the identification of the core review area, determines an abnormal event based on the abnormal feature combination and the core review area, and determines the bronchial status level based on the abnormal event, the actual morphology of the bronchus, and the surrounding environment of the bronchus. This takes into account the overall consideration of the abnormal event, the actual morphology of the bronchus, and the surrounding environment of the bronchus, thereby ensuring the accuracy of the identification of the bronchial status level. (3) The present invention relates to the determination of a core review area. Specifically, the core review area is determined by combining the user's age information and a review mapping relationship. The review mapping relationship associates factors such as the user's age, gender, and medical history with the risk level of the abnormal area. By further performing more detailed surface recognition and structural recognition on the core review area, a complete abnormal feature combination is formed to more comprehensively describe the abnormal situation of the core review area, thereby more accurately grasping the patient's bronchial status. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 1 is a flow chart of a method for boundary recognition of bronchial fluorescence images based on visual detection in an embodiment of the present invention; Figure 2 1 is a flow chart of step S11 in the boundary recognition method of bronchial fluorescence images based on visual detection in an embodiment of the present invention; Figure 3 1 is a flow chart of step S12 in the method for boundary recognition of bronchial fluorescence images based on visual detection in an embodiment of the present invention; Figure 4 3 is a flow chart of step S13 in the boundary recognition method of bronchial fluorescence images based on visual detection in an embodiment of the present invention; Figure 5 4 is a flow chart of step S14 in the boundary recognition method of bronchial fluorescence images based on visual detection in an embodiment of the present invention; Figure 6 4 is a flow chart of step S15 in the boundary recognition method of bronchial fluorescence images based on visual detection in an embodiment of the present invention; Figure 7 Schematic diagram of the structure of a bronchial fluorescence image boundary recognition system based on visual detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0008] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0009] See also Figures 1 to 7 A method for boundary recognition of bronchial fluorescence images based on visual detection is applied to a boundary recognition scenario of bronchial fluorescence images based on visual detection. The method for boundary recognition of bronchial fluorescence images based on visual detection includes: Step S11: collecting the position of the bronchus, and triggering corresponding visual detection based on the position of the bronchus to collect a bronchial fluorescence image; Step S12: determining a bronchial region and an environmental region based on detection of the bronchial fluorescence image, and determining multiple edge portions of the bronchus based on edge detection of the bronchial region; Step S13: determining the boundary of the bronchus according to the identification of the multiple edge portions of the bronchus, and determining the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image; Step S14: determining a core review area based on the actual morphology of the bronchus and the age of the user, and determining an abnormal feature combination based on the identification of the core review area; Step S15: determining an abnormal event based on the abnormal feature combination and the core review area, and determining the bronchial status level based on the abnormal event, the actual morphology of the bronchus, and the surrounding environment of the bronchus; refer to Figure 2In step S11, the position of the bronchus is collected, and corresponding visual detection is triggered based on the position of the bronchus to collect a bronchial fluorescence image; In the specific implementation process of the present invention, the specific steps are: S111: Detecting the position of the bronchus and determining the position of the bronchus based on the bronchial position detection. Simultaneously, collecting multiple posture parameters of the user and determining the current posture of the user based on the multiple posture parameters and the user's height. S112: Determine the area to be detected based on the position of the bronchus and the current posture of the user, and trigger the visual detection part on the peripheral side according to the spatial position of the area to be detected, so that the visual detection part moves relative to the area to be detected to perform visual detection on the area to be detected, and output the collected bronchial fluorescence image.

[0010] In an embodiment of the present application, the position of the bronchus is detected, and the position of the bronchus is determined based on the position detection of the bronchus. At the same time, multiple posture parameters of the user are collected, and the current posture of the user is determined based on the multiple posture parameters and the user's height. This takes into account the overall consideration of multiple posture parameters and the user's height to ensure the accuracy of the user's current posture.

[0011] At this time, the position of the bronchus is detected to determine the specific position of the bronchus in the body; at this time, a bronchoscope or other endoscopic equipment is used, which is usually equipped with position sensors such as optical locators, electromagnetic locators or inertial measurement units (IMUs). These sensors can track the movement of the bronchoscope in the body in real time and feed back its position data to the computer system; the position sensor determines the position and direction of the bronchoscope by sending and receiving signals (such as light signals, electromagnetic signals or acceleration / angular velocity data); the computer system converts these data into three-dimensional coordinates, thereby constructing the trajectory of the bronchoscope in the body.

[0012] Based on the data from the position sensor, the current position of the bronchus is accurately determined; at this time, the computer system processes the data from the position sensor and aligns it with a pre-acquired bronchial structure map (such as a CT scan or MRI image); the registration process involves aligning the real-time position data with the static image data to accurately locate the bronchus in three-dimensional space; at the same time, the registration algorithm includes an iterative closest point (ICP) algorithm, rigid transformation or affine transformation, etc. to ensure accurate alignment between the real-time position data and the static image data.

[0013] Obtain the user's current posture information, including the position and angle of the head, torso, and limbs; at this time, use external sensors such as cameras, depth sensors, or wearable devices (such as accelerometers, gyroscopes, or magnetometers) to capture the user's motion data and convert it into posture parameters; at the same time, cameras and depth sensors use image processing algorithms to identify the user's body parts and posture; wearable devices use built-in sensors to measure the user's acceleration, angular velocity, and magnetic field direction to infer the user's posture.

[0014] Combining posture parameters and height information, an algorithm is used to determine the user's current posture. At this point, the collected posture parameters are input into machine learning or deep learning models, which have been trained to infer the user's posture based on the input data. At the same time, height information serves as an additional input feature to help improve the accuracy of posture recognition. At the same time, machine learning models include support vector machines (SVMs), random forests, or deep neural networks. These models establish a mapping relationship between input parameters and posture by learning a large amount of posture data. In the inference stage, the model predicts the user's current posture based on the input posture parameters and height information.

[0015] Specifically, suppose a doctor is performing a bronchoscopy on a patient who is 175 cm tall. The doctor first inserts the bronchoscope into the patient's body and activates a position sensor to track the movement of the bronchoscope. As the bronchoscope goes deeper, the position sensor continuously sends position data to the computer system. After receiving this data, the computer system aligns it with a previously acquired CT scan image of the patient. Through the alignment algorithm, the computer system can accurately determine the position of the bronchoscope in the body and construct the trajectory of the bronchus in three-dimensional space.

[0016] At the same time, the doctor uses a camera to capture the patient's posture information during the examination; the camera captures the position and angle of the patient's head, torso and limbs, and inputs this data into the machine learning model; the model combines the patient's height information (175 cm) to infer that the patient's current posture is upright and slightly leaning forward; based on the position of the bronchi and the patient's current posture, the doctor determines the bronchial area to be inspected and adjusts the angle and depth of the bronchoscope to capture high-quality fluorescent images, which are then used for further diagnosis and analysis.

[0017] Furthermore, the area to be detected is determined according to the position of the bronchus and the current posture of the user, and the visual detection part on the surrounding side is triggered according to the spatial position of the area to be detected, so that the visual detection part moves relative to the area to be detected to perform visual detection on the area to be detected, and output the collected bronchial fluorescence image, which is compatible with the overall consideration of the position of the bronchus and the current posture of the user, and ensures the accuracy of the area to be detected.

[0018] At this time, based on the precise position of the bronchus and the user's current posture, the bronchial area that needs to be visually inspected is determined; at this time, combined with the position data of the bronchoscope (from step S111) and the user's posture information (also from step S111), three-dimensional modeling or image processing technology is used to determine the area to be inspected, which usually involves spatially aligning the position of the bronchoscope with the user's posture to identify areas in the bronchus where problems exist or that require further inspection; at the same time, specialized software tools are needed to visualize and analyze these data so that the doctor can intuitively see the position of the bronchi, the user's posture, and the relationship between them; through this information, the doctor determines the specific position and range of the area to be inspected.

[0019] Ensure that the visual detection part (such as the camera on the bronchoscope) can be accurately moved to the area to be inspected so that high-quality image acquisition can be performed; at this time, robotics, servo motors or other precision motion control devices are used to control the movement of the visual detection part. These devices are usually connected to a computer system and can receive the spatial position information of the area to be inspected and plan the movement path of the visual detection part based on this information; at the same time, the triggering process involves sending control signals to the motion control device. These signals contain information such as the target position and speed to which the visual detection part needs to move; after receiving these signals, the device will start the corresponding motion control algorithm to ensure that the visual detection part can move smoothly and accurately to the area to be inspected.

[0020] After the visual inspection part reaches the area to be inspected, high-quality image acquisition is performed; at the same time, once the visual inspection part reaches the target position, the camera or other image acquisition device is started to capture the fluorescent image of the bronchi. These images usually need to be acquired under specific lighting conditions to ensure the quality and clarity of the image; optionally, the image acquisition process involves adjusting the camera's focal length, exposure time, white balance and other parameters to obtain the best image effect; at the same time, it is also necessary to ensure that the acquired image is associated with the spatial position information of the area to be inspected for subsequent analysis and processing.

[0021] The acquired bronchial fluorescence images are presented to the physician or analysis system for further diagnosis and analysis. At this point, the acquired image data is transferred from the visual inspection piece to a computer system and displayed and processed using specialized software tools. These tools typically provide functions such as image enhancement, annotation, and analysis to help physicians better understand and interpret the image information. At the same time, the transmission of image data involves the use of wired or wireless communication technologies, such as USB, Ethernet, or Wi-Fi. At the same time, the integrity and security of the image data also need to be ensured to prevent data loss or tampering.

[0022] Specifically, assume that a doctor is performing a bronchoscopy on a patient and has completed position detection and posture recognition in step S111; in step S112, the doctor first determines the area to be detected based on the position of the bronchus and the patient's current posture - a suspected lesion area located in the middle of the bronchus; the doctor uses robotic technology to control the movement of the camera on the bronchoscope; the robotic system receives the spatial position information of the area to be detected and plans a movement path from the current position to the target position; as the camera moves, the doctor sees the situation inside the bronchus in real time and adjusts the angle and focal length of the camera to obtain the best image effect.

[0023] When the camera reaches the target position, the doctor starts the image acquisition process; under specific lighting conditions, the camera captures the fluorescent image of the bronchi and transmits the data to the computer system; the doctor uses specialized software tools to enhance and process the image to better observe and analyze the suspected lesion area; through this example, we can see how the various parts of step S112 work together and their application in actual clinical examinations; the doctor can determine the area to be inspected based on the position of the bronchi and the patient's current posture, and use robotic technology and image acquisition equipment to accurately capture the fluorescent image of the bronchi, thereby providing strong support for subsequent diagnosis and analysis.

[0024] In one embodiment of the present application, it is assumed that there are three areas to be detected: A, B and C; the comprehensive score of each area is calculated based on the current bronchial position (weight 0.5), the user's posture (weight 0.3) and the known lesion information (weight 0.2); it is assumed that the score of area A is 0.8 (position score 0.4 + posture score 0.3 + lesion information score 0.1), the score of area B is 0.6, and the score of area C is 0.5; because area A has the highest score, it is determined as the area to be detected; the system then triggers the camera on the bronchoscope to move to the position of area A and performs image acquisition according to preset parameters; the acquired bronchial fluorescence image is output to the doctor's display screen and marked as an image from area A.

[0025] refer to Figure 3 In step S12, the bronchial region and the environmental region are determined based on the detection of the bronchial fluorescence image, and multiple edge portions of the bronchus are determined based on the edge detection of the bronchial region; In the specific implementation process of the present invention, the specific steps are: S121: Acquire a bronchial fluorescence image, determine a plurality of branch features based on detection of the bronchial fluorescence image, determine a relative distance between two adjacent branch features based on positions of the plurality of branch features, and determine a bronchial region based on the relative distance between the two adjacent branch features and a connectivity state between the two adjacent branch features; S122: In the bronchus region, highlighting bronchus features based on image preprocessing of the bronchus region, traversing along the bronchus features, marking multiple edge contours of the bronchus features according to the traversal of the bronchus features, and determining multiple edge portions of the bronchus according to edge detection of the multiple edge contours; S123: Determine the non-bronchial area based on the comparison between the bronchial area and the bronchial fluorescence image, determine the environmental area based on the color areas of the non-bronchial area and the bronchial area, and determine the bronchial surrounding environment based on the identification of the environmental area.

[0026] In an embodiment of the present application, a bronchial fluorescence image is collected, a plurality of branch features are determined based on the detection of the bronchial fluorescence image, the relative distance between two adjacent branch features is determined based on the positions of the plurality of branch features, and the bronchial region is determined according to the relative distance between the two adjacent branch features and the connectivity status of the two adjacent branch features. This combines the overall consideration of the relative distance between the two adjacent branch features and the connectivity status of the two adjacent branch features to ensure the accuracy of the bronchial region.

[0027] At this time, high-quality fluorescence images of the inside of the bronchi are obtained for subsequent feature detection and area determination. At this time, a bronchoscope or other endoscopic equipment equipped with fluorescence imaging technology is used to image the inside of the bronchi under appropriate fluorescence excitation light. Fluorescence imaging technology uses light of a specific wavelength to excite fluorescent substances in bronchial tissue, causing them to emit fluorescence of a specific wavelength. By capturing these fluorescence signals, high-contrast images of the bronchi are obtained.

[0028] Multiple branching features of the bronchi, such as bifurcation points and curved segments, are identified in the fluorescence images. At the same time, image processing algorithms (such as edge detection, morphological processing, and feature extraction) are used to analyze the fluorescence images to identify the branching features of the bronchi. These algorithms can automatically detect edges, contours, and shape features in the images to determine the branching structure of the bronchi. For example, edge detection algorithms identify the boundaries of the bronchial wall, while morphological processing algorithms are used to smooth the image, remove noise, and enhance features.

[0029] The relative distance between two adjacent branch features is measured to understand the topological structure of the bronchi. At this time, after identifying the branch features, the measurement tools in the image processing software or custom algorithms are used to calculate the distance between adjacent branch features. At the same time, it involves marking the positions of the branch features in the image and using geometric measurement methods to calculate the distance between them. The distance is the straight-line distance and the curved distance along the bronchial path.

[0030] The overall area of the bronchus is determined by combining the relative distance and connectivity status of branch features. At this time, image processing or computer vision technology is used to construct a three-dimensional model or two-dimensional regional map of the bronchus, in which the relative distance and connectivity status of adjacent branch features serve as key input information. At the same time, graph theory algorithms (such as minimum spanning tree and shortest path algorithm) are used to connect adjacent branch features and construct the connectivity structure of the bronchus. At the same time, the use of shape analysis, pattern recognition and other technologies to further refine the bronchial area is also considered.

[0031] Specifically, suppose a doctor is performing a bronchoscopy on a patient and has collected a bronchial fluorescence image; in step S121, the doctor first uses an image processing algorithm to analyze the image and identifies multiple bronchial branch features, including several obvious bifurcation points and curved segments; next, the doctor uses a measurement tool to calculate the relative distance between adjacent branch features; for example, he finds that the straight-line distance from a bifurcation point to its adjacent curved segment is about 2 cm; at the same time, he also confirms the connectivity between these branch features, that is, they are connected through the bronchial wall.

[0032] Based on this information, the doctor uses image processing software to construct a two-dimensional regional map of the bronchus. In this map, each branch feature is marked, and their relative positions and connectivity relationships are accurately represented. Through this regional map, the doctor can clearly see the overall structure and topological characteristics of the bronchi. This example demonstrates the application of the S121 step in actual clinical examinations. By acquiring bronchial fluorescence images, identifying branch features, measuring relative distances, and determining connectivity status, the doctor can accurately determine the bronchial region, providing strong support for subsequent diagnosis and treatment.

[0033] Furthermore, in the bronchial area, the bronchial features are highlighted based on the image preprocessing of the bronchial area. At the same time, the bronchial features are traversed, and multiple edge contours of the bronchial features are marked according to the traversal of the bronchial features. The multiple edge parts of the bronchus are determined based on the edge detection of the multiple edge contours, which is compatible with the overall consideration of the edge detection of the multiple edge contours and ensures the accuracy of the multiple edge parts of the bronchus.

[0034] At this time, image preprocessing technology is used to enhance the bronchial features in the bronchial area to make them clearer and easier to identify; at this time, preprocessing algorithms such as image enhancement, denoising, contrast adjustment, and bronchial-specific filtering techniques are applied to improve image quality; at the same time, adaptive histogram equalization is used to enhance the contrast of the image, or a Gaussian filter is used to smooth the image and reduce noise; for bronchial features, morphological operations (such as dilation and erosion) need to be applied to highlight the tube wall or branch structure.

[0035] In the preprocessed image, traversal is performed along the direction of the bronchus for subsequent edge contour marking. At this time, an image traversal algorithm, such as depth-first search (DFS) or breadth-first search (BFS), is used to start from the known bronchial starting point or feature point and traverse along the continuous features of the bronchus. During the traversal process, the traversal path needs to be adjusted according to the curvature and direction of the bronchus to ensure that the entire bronchial structure can be accurately tracked. This requires combining image gradient information or edge detection results to guide the traversal process.

[0036] During the traversal process, the edge contours of the bronchial features are marked to provide a basis for subsequent edge detection. At this time, when the bronchial features are traversed, edge detection algorithms (such as the Canny edge detector) or morphological gradient methods are used to identify and mark the edge contours. The edge detection algorithm can identify points in the image with significant brightness changes, which usually correspond to the edges of the bronchial wall. When marking the edge contours, it is necessary to ensure the integrity and continuity of the contours for subsequent analysis.

[0037] Based on the marked edge contours, multiple specific edge parts of the bronchus are determined, such as the tube wall and branch points. At this time, the marked edge contours are further analyzed, such as contour tracking and shape matching, to determine the different edge parts of the bronchus. At the same time, shape matching is performed in combination with prior knowledge of the bronchus (such as shape and size), or a contour analysis algorithm is used to identify specific edge features (such as the sharpness of the branch points, the smoothness of the tube wall, etc.).

[0038] Specifically, assume that a doctor is using image processing software to analyze a pre-processed bronchial region image; in step S122, the doctor first applies adaptive histogram equalization to enhance the contrast of the image, and uses morphological operations to highlight the bronchial wall features; next, the doctor starts from the known bronchial starting point and uses a depth-first search algorithm to traverse along the direction of the bronchus; during the traversal process, the doctor uses a Canny edge detector to identify and mark the edge contours of the bronchial features, which clearly outline the edges of the bronchial wall.

[0039] The doctor further analyzed the marked edge contours; through contour tracking and shape matching algorithms, the doctor determined multiple specific edge parts of the bronchus, including the tube wall, branch points, etc.; for example, the doctor found an obvious branch point, whose edge contour showed a sharp shape feature; through this example, we can see the application of step S122 in actual clinical examinations; through image preprocessing, traversing bronchial features, marking edge contours and determining edge parts, the doctor can accurately identify and analyze the structural characteristics of the bronchi, providing strong support for subsequent diagnosis and treatment.

[0040] Therefore, the non-bronchial area is determined based on the comparison between the bronchial area and the bronchial fluorescence image, the environmental area is determined according to the color areas of the non-bronchial area and the bronchial area, and the lateral environment of the bronchus is determined based on the identification of the environmental area. This combines the overall consideration of the color areas of the non-bronchial area and the bronchial area to ensure the accuracy of the environmental area.

[0041] At this time, the non-bronchial area is distinguished by comparing the bronchial area with the entire bronchial fluorescence image; at this time, the regional segmentation or classification algorithm in the image processing technology is used to distinguish the bronchial area and the non-bronchial area based on the feature differences in the image (such as brightness, color, texture, etc.); at the same time, one or more feature spaces are defined to describe the features of the bronchial area and the non-bronchial area; then, a clustering algorithm (such as K-means, DBSCAN, etc.) or a classification algorithm (such as support vector machine, neural network, etc.) is applied to perform segmentation or classification in these feature spaces; finally, the position and range of the non-bronchial area are determined based on the output results of the algorithm.

[0042] In the non-bronchial area, the environmental area, that is, the specific tissue or structure around the bronchi, is further determined based on the color characteristics. At this time, the color features in the non-bronchial area are mapped to a specific color space using color space conversion and color segmentation techniques, and segmented according to methods such as color threshold or color histogram. At the same time, commonly used color spaces include RGB, HSV, Lab, etc. After selecting a suitable color space, the color histogram of the non-bronchial area is calculated, and the color threshold is determined based on the peak or valley value of the histogram. Then, these thresholds are applied to segment the non-bronchial area into different color regions, each of which corresponds to a specific tissue or structure.

[0043] Based on the identified environmental area, the surrounding environment of the bronchus is comprehensively analyzed, including the health status of the surrounding tissues, the presence of lesions or abnormalities, etc.; at this time, the environmental area is further analyzed and interpreted in combination with image processing technology, which involves advanced image processing technologies such as morphological operations, texture analysis, and feature extraction; optionally, morphological operations are used to detect abnormal structures (such as masses, nodules, etc.) in the environmental area, or texture analysis is used to evaluate the uniformity and heterogeneity of the surrounding tissues; in addition, specific features (such as area, circumference, circularity, etc.) are extracted to quantify the attributes of the environmental area and compared with medical standards to determine whether there are lesions or abnormalities.

[0044] Specifically, assume that a doctor is using advanced image processing software to analyze a bronchial fluorescence image; in step S123, the doctor first determines the location and range of the non-bronchial area by comparing the bronchial area with the entire image. These non-bronchial areas include lung tissue, blood vessels, airway walls and other surrounding tissues; the doctor uses color space conversion and color segmentation technology to map the color features in the non-bronchial area to the HSV color space, and determines the color threshold based on the color histogram; then, these thresholds are applied to segment the non-bronchial area into different color areas, each of which corresponds to a specific tissue or structure, such as red lung tissue, blue blood vessels, etc.

[0045] The doctors combined prior medical knowledge with image processing technology to conduct further analysis of the environmental area. Through morphological operations, they detected an abnormal structure near the bronchus, which was similar in shape and size to a lung nodule. In addition, they used texture analysis to evaluate the uniformity and heterogeneity of the surrounding lung tissue and found that the texture characteristics of this area were different from those of normal lung tissue. Based on these analysis results, they determined that there was an abnormality in the surrounding environment of the bronchus and suspected that the abnormal structure was a lung nodule. This finding is of great significance for subsequent diagnosis and treatment, as lung nodules are one of the early signs of lung cancer.

[0046] In one embodiment of the present application, a color region matching table is collected, and the color region matching table is shown in Table 1: Table 1 Color region matching table

[0047] In this color region matching table, different tissues or structures are distinguished according to the value range of the HSV color space; for example, the red region corresponds to lung tissue, the blue region corresponds to blood vessels, and the green region corresponds to airway walls or other tissues.

[0048] In this example, the lung tissue area, blood vessel area, and airway wall area were assigned weights of 0.4, 0.3, and 0.3, respectively, and given scores based on their health status; the final calculated composite score was 83, indicating that the bronchial surrounding environment was healthy overall, but there was mild inflammation in the airway wall area.

[0049] refer to Figure 4 In step S13, the boundary of the bronchus is determined based on the identification of multiple edge portions of the bronchus, and the actual morphology of the bronchus is determined based on the boundary of the bronchus and the bronchial fluorescence image; In the specific implementation process of the present invention, the specific steps are: S131: collecting multiple edge portions of the bronchus, determining multiple edge segments based on the simultaneous detection of the multiple edge portions of the bronchus; and forming a coherent overall segment based on the synthesis of the multiple edge segments, wherein the coherent overall segment serves as the boundary of the bronchus. S132: matching the boundary of the bronchus to the bronchial fluorescence image and outputting the morphological region of the bronchus; S133: Mark the corners of the morphological region of the bronchus, and determine the actual region of the bronchus based on the optimization of the corners of the morphological region of the bronchus; determine the actual morphology of the bronchus based on the identification of the actual region of the bronchus.

[0050] In an embodiment of the present application, multiple edge portions of the bronchus are collected, and multiple edge segments are determined based on the synchronous detection of the multiple edge portions of the bronchus; a coherent overall segment is formed based on the synthesis of the multiple edge segments, and the coherent overall segment serves as the boundary of the bronchus, which is compatible with the overall consideration of the synchronous detection of the multiple edge portions of the bronchus, thereby ensuring the accuracy of the multiple edge segments.

[0051] At this time, image processing technology is used to extract the edge of the bronchus from the bronchial image. These edge parts usually correspond to the junction of the bronchial wall and the surrounding tissue. At this time, edge detection algorithms such as the Sobel operator and the Canny edge detector are used to process the bronchial image. These algorithms can identify areas in the image where the brightness or color changes significantly, thereby determining the edge of the bronchus. Before applying the edge detection algorithm, the image needs to be preprocessed, such as denoising and contrast enhancement, to improve the accuracy of edge detection. The output of the edge detection algorithm is usually a set of edge points or edge segments, which constitute the edge of the bronchus.

[0052] After capturing multiple edge portions of the bronchus, multiple edge line segments are determined by synchronously detecting these edge portions. These line segments can more accurately describe the shape and boundary of the bronchus. At this point, line segment detection or contour tracking algorithms in image processing are used to connect the captured edge portions into continuous line segments. These line segments should be as close as possible to the actual boundary of the bronchus and maintain continuity with each other. Line segment detection or contour tracking algorithms involve complex image processing techniques such as Hough transform and dynamic programming. In practical applications, it is necessary to select an appropriate algorithm based on the specific characteristics of the bronchial image and adjust the algorithm parameters to achieve the best results.

[0053] Multiple edge line segments are combined into a coherent whole line segment to form a clear boundary of the bronchus, which is used for subsequent tasks such as image analysis, lesion detection, or three-dimensional reconstruction. At this time, the line segment merging or contour smoothing algorithm in image processing is used to connect the multiple edge line segments into a coherent whole. During the merging process, it is necessary to ensure the continuity and smoothness of the line segments to avoid breaks or jagged boundaries. At the same time, the line segment merging or contour smoothing algorithm involves a variety of image processing techniques, such as morphological operations and interpolation algorithms. In practical applications, it is necessary to select an appropriate algorithm based on the specific characteristics of the bronchial image and the requirements of subsequent tasks, and make necessary parameter adjustments.

[0054] Specifically, assume that an advanced medical image processing software is being used to analyze a bronchoscopic image; in step S131, the image is first processed using the Canny edge detection algorithm to collect multiple edge portions of the bronchi, which are presented in the form of edge points and are distributed around the bronchial wall; next, a contour tracking algorithm is used to connect these edge points into continuous line segments, which present the shape and boundary of the bronchi in the image; however, due to factors such as image noise and uneven illumination, these line segments are not completely coherent or smooth.

[0055] To address this issue, line segment merging and contour smoothing algorithms are used to process these line segments. These algorithms can identify and connect adjacent line segments while smoothing irregular parts of the line segments. After processing, a coherent and smooth overall line segment is obtained, which accurately describes the boundary of the bronchus. This boundary is used for subsequent image analysis tasks such as lesion detection and 3D reconstruction. For example, this boundary is used to extract the geometric features of the bronchus, such as length and diameter, to further assess the health of the bronchus. In addition, this boundary also serves as a benchmark for 3D reconstruction, helping to build a 3D model of the bronchus for more in-depth medical research and treatment planning.

[0056] Furthermore, multiple edge portions of the bronchus are collected, and multiple edge segments are determined based on the synchronous detection of the multiple edge portions of the bronchus; a coherent overall segment is formed based on the synthesis of the multiple edge segments, and the coherent overall segment is used as the boundary of the bronchus. The introduction of a coherent overall segment as the boundary of the bronchus ensures the accuracy of the recognition of the boundary of the bronchus, and is compatible with the overall consideration of the boundary of the bronchus and the bronchial fluorescence image, further improving the accuracy of the actual morphology of the bronchus.

[0057] At this time, multiple edge portions of the bronchus are collected. After collecting multiple edge portions of the bronchus, these edge portions need to be connected into continuous line segments to more accurately describe the shape and boundary of the bronchus; at this time, the contour tracking or line segment fitting algorithm in image processing is used to connect the detected edge points into line segments. These line segments should be as close as possible to the actual boundary of the bronchus and maintain continuity; the contour tracking algorithm usually starts from a starting point and gradually tracks along the edge points until it returns to the starting point or reaches the set termination condition; the line segment fitting algorithm uses methods such as least squares method and Hough transform to fit the edge points into straight lines or curve segments.

[0058] Multiple edge segments are combined into a coherent overall segment to form a clear and continuous boundary of the bronchus; at this time, the segment merging or contour smoothing algorithm in image processing is used to connect adjacent segments and smooth the transition areas between the segments. These algorithms ensure that the generated boundary segments are both coherent and smooth; the segment merging algorithm involves steps such as overlap detection and endpoint matching between segments; the contour smoothing algorithm uses morphological operations, filters and other technologies to smooth the irregular parts on the segments.

[0059] Specifically, assume that a high-resolution bronchial CT image is being analyzed using a medical image processing software. The image is processed using the Canny edge detection algorithm to identify the edges of the bronchus. During the processing, the threshold parameters of the algorithm are adjusted to ensure that clear edge information can be detected.

[0060] After collecting edge information, a contour tracking algorithm is used to connect these edge points into continuous line segments, which show the shape and boundary of the bronchi in the image. Due to the presence of noise and artifacts in the image, some line segments are not completely coherent or smooth. To obtain a coherent and smooth bronchial boundary, the detected line segments are processed using line segment merging and contour smoothing algorithms. These algorithms can identify and connect adjacent line segments and smooth irregular parts on the line segments. After processing, a clear bronchial boundary line segment is obtained, which accurately describes the shape and position of the bronchus.

[0061] This boundary segment is used for subsequent image analysis tasks, such as 3D reconstruction and lesion detection. For example, this boundary segment is used to extract the geometric features of the bronchus, such as length and diameter, to further assess the health of the bronchus. In addition, this boundary segment also serves as a benchmark for 3D reconstruction, helping to build a 3D model of the bronchus for more in-depth medical research and treatment planning.

[0062] In one embodiment of the present application, a preset matching line segment ID matching table is collected. The matching line segment ID matching table records the similarities between different edge line segments, thereby helping to determine which line segments should be merged. The matching line segment ID matching table is shown in Table 2: Table 2 Matching line segment ID matching table

[0063] In this matching segment ID matching table, the segment ID is a unique identifier, the start point coordinates and the end point coordinates define the endpoints of the segment, the similarity score represents the similarity between the current segment and other segments (calculated based on factors such as distance, direction, and length), and the matching segment ID represents the ID of the other segment that is most similar to the current segment.

[0064] Reference Figure 5 In step S14, a core review area is determined based on the actual morphology of the bronchus and the age of the user, and an abnormal feature combination is determined based on the identification of the core review area; In the specific implementation process of the present invention, the specific steps are: S141: determining a plurality of branch tubes based on the actual morphological division of the bronchus, and marking the actual morphologies of the plurality of branch tubes, wherein the plurality of branch tubes serve as various parts of the bronchus; S142: In each branch tube, a corresponding surface abnormality feature is determined based on the detection of the branch tube, multiple abnormal regions are determined based on the actual shapes of the multiple branch tubes and the corresponding surface abnormality features, and a core review region is determined based on the multiple abnormal regions, the user's age, and the review mapping relationship; S143: Perform surface recognition and structural recognition on the core review area, determine a first sub-abnormal feature based on the surface recognition of the core review area, determine a second sub-abnormal feature based on the structural recognition of the core review area, and determine an abnormal feature combination based on the first sub-abnormal feature and the second sub-abnormal feature.

[0065] In an embodiment of the present application, multiple branch tube bodies are determined based on the division of the actual morphology of the bronchi, and the actual morphologies of the multiple branch tube bodies are marked. The multiple branch tube bodies are introduced as various parts of the bronchi.

[0066] At this point, the actual morphology of the branch trachea is identified and delineated so that it can be decomposed into multiple manageable branch tubes; at this point, medical image analysis techniques, such as computed tomography (CT) or magnetic resonance imaging (MRI), are used to obtain three-dimensional images of the bronchi; then, image processing algorithms (such as region growing, threshold segmentation, morphological operations, etc.) are used to automatically identify the main trunk and various branches of the bronchi; in the delineation process, the complexity and variability of the bronchi need to be considered; the morphology of the bronchi varies due to individual differences, disease status or scanning conditions; therefore, the algorithm needs to have a certain degree of flexibility and robustness to adapt to different situations.

[0067] After identifying the main trunk and branches of the bronchus, these parts need to be determined as multiple independent branch tubes; at this time, based on the morphological characteristics and connection relationships of the bronchi, the main trunk and branches are divided into multiple continuous tube segments, which are straight segments, curved segments or complex structures at branch points; when determining the branch tubes, it is necessary to ensure the integrity and continuity of each tube segment, which requires the use of image registration, interpolation or smoothing techniques to deal with noise, artifacts or discontinuous areas in the image.

[0068] Each identified branch tube is marked and its actual morphological information is recorded. At this point, image processing software or a database system is used to assign a unique identifier to each branch tube and record its position, length, diameter, branch angle and other morphological features. This information is used for subsequent image analysis, lesion detection or three-dimensional reconstruction tasks. During the marking process, it is necessary to ensure the accuracy and consistency of the information, which requires the use of standardized or calibrated techniques to process the size and position information in the image.

[0069] Specifically, suppose that an advanced medical imaging analysis software is being used to analyze a high-resolution bronchial CT image; the automatic recognition function in the software is used to identify the main trunk and branches of the bronchus; the software uses advanced image processing algorithms to detect the edges and connection relationships of the bronchi, thereby dividing them into multiple parts; after identifying the main trunk and branches of the bronchus, the segmentation function of the software is used to determine these parts as multiple independent branch tubes; each branch tube is a continuous tube segment with unique morphological characteristics and connection relationships.

[0070] The software's labeling function assigns a unique identifier to each branch tube and records its actual morphological information, including the location, length, diameter, and branching angle of each branch tube. This information will be used for subsequent image analysis tasks such as lesion detection and 3D reconstruction. For example, during the labeling process, it will be found that the diameter of a branch tube is abnormally enlarged, which is caused by a tumor or inflammation. By recording this information, it provides strong support for subsequent diagnosis and treatment. At the same time, this information is also used to construct a 3D model of the bronchus for more in-depth medical research and treatment planning.

[0071] Furthermore, in each branch tube body, the corresponding surface abnormality features are determined based on the detection of the branch tube body, and multiple abnormal areas are determined based on the actual morphology of the multiple branch tube bodies and the corresponding surface abnormality features. The core review area is determined based on the multiple abnormal areas, the age of the user and the review mapping relationship, which is compatible with the overall consideration of multiple abnormal areas, the age of the user and the review mapping relationship to ensure the accuracy of the core review area.

[0072] At this point, abnormal features on the surface of each branch vessel are identified that indicate potential lesions or abnormalities; Methods: Medical imaging analysis techniques, such as computed tomography (CT) or magnetic resonance imaging (MRI), combined with image processing algorithms (such as filtering, edge detection, and texture analysis) are used to detect subtle changes or irregularities in the surface of branch vessels. Abnormal features include nodules, masses, stenosis, thickening, and calcification. The algorithm needs to be able to distinguish normal anatomical structures from abnormal features and accurately quantify the size, shape, and position of these features.

[0073] After identifying abnormal surface features, it is necessary to determine which parts have abnormal areas based on the actual morphology of the branch tube. At this time, the diameter, length, curvature of the branch tube, and the type, size, and location of the abnormal surface features are comprehensively considered, and image processing or machine learning algorithms are used to identify abnormal areas. At the same time, three-dimensional reconstruction technology is used to visualize the internal structure of the branch tube and conduct a comprehensive analysis based on the abnormal surface features. The determination of abnormal areas requires consideration of multiple factors, such as the number, density, and distribution of abnormal features.

[0074] After multiple abnormal areas are identified, it is necessary to combine the user's age information and the review mapping relationship to determine which areas are the core review areas, that is, the areas that need to be focused on. At this time, clinical guidelines, statistical data or expert experience are used to establish a review mapping relationship, and the user's age, gender, medical history and other factors are associated with the risk level of the abnormal area. Then, the core review areas are determined based on these risk levels. At the same time, the review mapping relationship is a preset model that takes into account the interaction between multiple factors. When determining the core review areas, it is necessary to weigh the importance of different factors and consider the potential progression or deterioration of the lesion.

[0075] Specifically, assume that a 60-year-old male patient is undergoing a bronchial CT scan analysis; the CT scan image and image processing algorithm are used to detect abnormal features on the surface of each branch tube; for example, a nodule with a diameter of 5 mm is detected in a branch of the right main bronchus, which has an irregular shape and blurred edges; combining the actual morphology of the branch tube and the detected nodule features, the area where the nodule is located is determined to be an abnormal area; at the same time, it is also noted that the diameter of the branch tube is slightly increased, which is due to the stenosis of the lumen caused by the nodule; therefore, this area is marked as one of the abnormal areas.

[0076] Considering that the patient is 60 years old and belongs to the middle-aged and elderly population, and the nodule characteristics meet the indications of certain high-risk lesions (such as irregular shape, blurred edges, etc.), the area where the nodule is located is determined as the core review area according to the review mapping relationship. This means that in subsequent reviews or treatments, this area will require special attention in order to promptly detect and treat potential lesions; through the analysis of step S142, the abnormal area in the bronchi can be identified more accurately, and the core review area can be determined in combination with the patient's age and review mapping relationship, thereby providing strong support for subsequent diagnosis and treatment.

[0077] Therefore, surface recognition and structural recognition are performed on the core review area, and the first sub-abnormality feature is determined based on the surface recognition of the core review area. The second sub-abnormality feature is determined based on the structural recognition of the core review area. The abnormality feature combination is determined based on the first sub-abnormality feature and the second sub-abnormality feature, which is compatible with the overall consideration of the first sub-abnormality feature and the second sub-abnormality feature, and ensures the accuracy of the abnormality feature combination.

[0078] At this point, the surface features of the core review area are analyzed in detail to detect any abnormalities or irregularities. At this point, high-resolution medical imaging (such as CT or MRI) and image processing technology are used to perform a detailed surface analysis of the core review area. This includes using three-dimensional reconstruction technology to visualize the surface of the area, and applying algorithms such as texture analysis and edge detection to identify subtle changes in the surface. At the same time, surface recognition involves preprocessing the image, such as denoising and contrast enhancement, to improve the accuracy of the analysis. In addition, the impact of image resolution and scanning conditions on the analysis results also needs to be considered.

[0079] Based on surface recognition, the first sub-abnormal feature related to the core review area is determined; at this time, according to the results of surface recognition, any features inconsistent with the normal anatomical structure, such as surface unevenness, nodules, lumps, ulcers, etc., are identified and recorded as the first sub-abnormal feature; determining the first sub-abnormal feature requires a combination of clinical experience and professional knowledge to distinguish between normal variations and true abnormalities; in addition, the size, shape, position and relationship of the abnormal feature to the surrounding tissue also need to be considered.

[0080] Analyze the internal structure of the core review area to detect any structural abnormalities or lesions. At this time, use medical imaging analysis techniques such as three-dimensional reconstruction, volume rendering, and morphological analysis to perform a detailed analysis of the internal structure of the core review area, which involves the evaluation of the bronchial wall, lumen, and branching structure. At the same time, structural recognition needs to consider the impact of factors such as image resolution, contrast, and noise on the analysis results. In addition, clinical guidelines and professional knowledge need to be combined to interpret the analysis results.

[0081] Based on structural recognition, the second sub-abnormality feature related to the core review area is determined. At this time, according to the results of structural recognition, any structural abnormalities such as wall thickening, lumen stenosis, abnormal branching structure, etc. are identified and recorded as the second sub-abnormality feature. The determination of the second sub-abnormality feature requires consideration of the range, degree, relationship with surrounding tissues, and cause of the abnormality.

[0082] The first sub-abnormality feature and the second sub-abnormality feature are combined to form a complete abnormality feature combination to more comprehensively describe the abnormal situation in the core review area; at this time, the first sub-abnormality feature and the second sub-abnormality feature are summarized and classified, and the abnormality feature combination is determined according to their nature, location and mutual relationship, which involves quantitative evaluation of abnormal features, etiology inference and risk assessment; determining the abnormality feature combination requires combining clinical experience and professional knowledge to ensure the accuracy and reliability of the analysis; in addition, the interaction and potential impact between abnormal features also need to be considered.

[0083] Specifically, assume that a bronchial CT scan image of a patient is being analyzed, and the core review area is determined to be a suspected nodule area located in the right main bronchus; high-resolution CT images and three-dimensional reconstruction technology are used to visualize the surface of the area; through careful observation and analysis, it is found that the surface of the area is uneven and there is a tiny nodular protrusion; based on the results of surface recognition, the first sub-abnormal feature of the area is determined to be "uneven surface and the presence of nodular protrusions", which suggests that there is some abnormality or lesion in the area.

[0084] Further analysis of the internal structure of the area revealed that the bronchial walls around the nodule were thickened and the lumen was slightly narrowed. These structural changes corroborated the first sub-abnormality feature, further supporting the hypothesis that an abnormality or lesion existed in the area. Based on the results of structural recognition, the second sub-abnormality feature of the area was determined to be "thickened walls and narrowed lumen," which provided more information about the nature of the abnormality or lesion.

[0085] Combining the first and second sub-abnormal features, a complete abnormal feature combination is formed: "uneven surface with nodular protrusions, thickened tube wall and narrow lumen". This abnormal feature combination provides a comprehensive description of the abnormal situation in the core review area and provides strong support for subsequent diagnosis and treatment.

[0086] refer to Figure 6 In step S15, an abnormal event is determined based on the abnormal feature combination and the core review area, and the bronchial status level is determined based on the abnormal event, the actual morphology of the bronchus, and the surrounding environment of the bronchus; In the specific implementation process of the present invention, the specific steps are: S151: Collecting abnormal feature combinations, determining a corresponding abnormal list based on the matching of the abnormal feature combinations with the actual morphology of the bronchus, wherein the abnormal list presents various parts of the bronchus; determining abnormal events based on the abnormal list and the core review area; S152: Collecting the surrounding environment of the bronchus, determining a first state coefficient based on the surrounding environment of the bronchus and the abnormal event, and determining a second state coefficient based on the surrounding environment of the bronchus and the actual morphology of the bronchus; S153: Match the corresponding state level mapping relationship according to the actual morphology of the bronchus and the previous detection information of the bronchus, and determine the state level of the bronchus according to the state level mapping relationship, the first state coefficient and the second state coefficient.

[0087] In an embodiment of the present application, abnormal feature combinations are collected, and a corresponding abnormal list is determined based on the matching of the abnormal feature combinations and the actual morphology of the bronchus, which presents various parts of the bronchus; abnormal events are determined based on the abnormal list and the core review area, which is compatible with the overall consideration of the abnormal list and the core review area, thereby ensuring the accuracy of abnormal events.

[0088] At this point, the abnormal feature combinations determined in the previous steps are collected. These feature combinations describe the abnormalities or lesions present in the bronchi. At this point, the abnormal feature combinations that have been determined in the previous analysis steps (such as S143) are obtained. These features include nodules, masses, wall thickening, lumen stenosis, etc.; ensure that the collected abnormal feature combinations are accurate and complete, and match the patient's medical imaging data.

[0089] The collected abnormal feature combination is matched with the actual morphology of the bronchus to determine the abnormalities in various parts of the bronchus; at this time, medical image analysis techniques such as three-dimensional reconstruction and volume rendering are used to locate the abnormal feature combination to the specific part of the bronchus. This requires combining the anatomical structure and branching of the bronchus to determine the specific location of the abnormality; the matching process needs to consider the size, shape, position and relationship of the abnormal features with the surrounding tissues to ensure the accuracy and completeness of the abnormality list.

[0090] After determining the abnormalities in various parts of the bronchus, the specific abnormal events are further determined in combination with the information from the core review area. At this time, the abnormality list is analyzed to find the abnormal features most relevant to the core review area. Combined with the patient's medical history, symptoms and other information, the specific abnormal events are determined. This requires considering factors such as the nature, size, and progression rate of the abnormal features. When determining abnormal events, multiple factors need to be considered comprehensively to ensure the accuracy and reliability of the diagnosis. In addition, the cause of the disease, pathophysiological mechanisms, etc. need to be considered to provide guidance for subsequent treatment.

[0091] Specifically, suppose a patient underwent a bronchial CT scan and the following combination of abnormal features was identified: a nodular protrusion was present in the middle segment of the right main bronchus, and the wall of the bronchus in this area was thickened; the two abnormal features of "nodular protrusion in the middle segment of the right main bronchus" and "thickening of the wall" were collected from the analysis results.

[0092] Using three-dimensional reconstruction technology, the two abnormal features were located in the middle segment of the right main bronchus; the abnormality list showed that there was a nodular protrusion in this area and the wall was thickened, and these abnormal features were related to a certain lesion; combined with the patient's medical history (such as a long-term smoking history), symptoms (such as cough, sputum, and difficulty breathing), and information from the abnormality list, the patient's abnormal event was determined to be "suspected lung cancer in the middle segment of the right main bronchus"; the core review area was the middle segment of the right main bronchus, and the nodular protrusions and thickening of the wall in this area were key evidence supporting this diagnosis.

[0093] Furthermore, the lateral environment of the bronchus is collected, and the first state coefficient is determined based on the lateral environment of the bronchus and abnormal events, and the second state coefficient is determined based on the lateral environment of the bronchus and the actual morphology of the bronchus. This takes into account the overall consideration of the lateral environment of the bronchus and the actual morphology of the bronchus, ensuring the accuracy of the second state coefficient.

[0094] At this time, the bronchial environment is collected to evaluate the impact of abnormal events on the bronchial environment, thereby determining the first state coefficient; at this time, the imaging data of the bronchial environment are analyzed to evaluate whether abnormal events (such as tumors, inflammation, etc.) cause compression, infiltration or destruction to surrounding tissues; based on the degree and scope of the impact, a predetermined scoring standard or model is used to determine the first state coefficient; when determining the first state coefficient, factors such as the nature, size, location and progression rate of the abnormal event need to be considered; the scoring standard or model is formulated based on clinical experience, research results or expert consensus.

[0095] Assess the coordination between the actual morphology of the bronchus and its surrounding environment to determine the second state coefficient; at this time, compare the actual morphology of the bronchus (such as lumen size, wall thickness, branching structure, etc.) with the characteristics of its surrounding environment (such as surrounding tissue density, vascular distribution, lymph node size, etc.); based on the coordination and consistency between the two, use a predetermined scoring standard or model to determine the second state coefficient; when determining the second state coefficient, it is necessary to consider the interaction and mutual influence between the bronchus and the surrounding environment; the scoring standard or model involves multiple aspects of evaluation, such as morphological matching, functional coordination, pathophysiological consistency, etc.

[0096] Specifically, suppose a patient underwent a bronchial CT scan and was diagnosed with "suspected lung cancer in the middle segment of the right main bronchus" as an abnormal event; imaging data of the bronchus and its surrounding environment were obtained through CT scan; analysis showed that there was a nodular protrusion in the middle segment of the right main bronchus, and the wall of the bronchus in this area was thickened; at the same time, the density of the surrounding lung tissue increased, some blood vessels were compressed, and there were signs of enlarged lymph nodes.

[0097] Assess the impact of abnormal events (suspected lung cancer) on the bronchial environment. Since the nodular protrusions compress the surrounding tissues and there are signs of enlarged lymph nodes, this indicates that the abnormal event has a significant impact on the bronchial environment. Based on the predetermined scoring criteria, the first state coefficient is determined to be 0.7 (indicating a moderate impact).

[0098] Compare the coordination between the actual morphology of the bronchus and its surrounding environment; analysis shows that the nodular protrusions and wall thickening in the middle segment of the right main bronchus are inconsistent with the surrounding features such as increased lung tissue density, vascular compression and lymphadenopathy, indicating that the coordination between the actual morphology of the bronchus and its surrounding environment is poor; according to the predetermined scoring criteria, the second state coefficient is determined to be 0.5 (indicating poor coordination); through this example, we can see how step S152 combines medical image analysis, abnormal events and information on the surrounding environment to determine the two key coefficients of the bronchial state (the first state coefficient and the second state coefficient), which provide an important basis for subsequent state level assessment and treatment plan.

[0099] Therefore, the corresponding state level mapping relationship is matched according to the actual morphology of the bronchus and the previous detection information of the bronchus, and the state level of the bronchus is determined according to the state level mapping relationship, the first state coefficient and the second state coefficient. This is compatible with the overall consideration of the state level mapping relationship, the first state coefficient and the second state coefficient, ensuring the accuracy of the state level of the bronchus. At the same time, it is compatible with the overall consideration of abnormal events, the actual morphology of the bronchus and the surrounding environment of the bronchus, further ensuring the accuracy of the identification of the state level of the bronchus.

[0100] At this time, a state-level mapping relationship that matches the current actual morphology of the bronchus and previous detection information is found; the state-level mapping relationship is usually a preset table or model used to associate the morphological characteristics and detection information of the bronchus with a specific state level; at this time, the current actual morphological information of the bronchus is collected, which includes lumen size, wall thickness, branching structure, etc.; then, review previous detection information, such as historical lesion progression, treatment response, imaging changes, etc.; compare this information with the preset state-level mapping relationship to find the most matching mapping relationship; at the same time, multiple factors need to be considered in the matching process, including the changing trend of bronchial morphology, the reliability and consistency of previous detection information, etc.; the state-level mapping relationship is based on clinical experience, research results or expert consensus, so its accuracy and applicability need to be ensured.

[0101] After determining the state-level mapping relationship that matches the actual morphology of the bronchus and previous detection information, the state level of the bronchus is further determined by combining the first state coefficient and the second state coefficient; at this time, the first state coefficient and the second state coefficient are substituted into the matched state-level mapping relationship, and the state level of the bronchus is calculated according to the rules or algorithms in the mapping relationship; the state level is a numerical value, classification label or descriptive term used to indicate the current state or degree of lesions of the bronchus; at the same time, when determining the state level, it is necessary to consider the weights and mutual relationships of the first state coefficient and the second state coefficient. These coefficients reflect different aspects of information, such as the impact of abnormal events on the surrounding environment, the coordination between the bronchial morphology and the surrounding environment, etc.; therefore, when calculating the state level, it is necessary to ensure that these coefficients are reasonably considered and balanced.

[0102] Specifically, suppose a patient undergoes a series of examinations and the following information is determined: The actual morphology of the bronchi: the lumen of the middle segment of the right main bronchus is narrow and the wall is thickened; Previous test information: Last year's CT scan showed mild wall thickening in this area, and this year the stenosis has worsened; First state coefficient: 0.6 (indicates that the abnormal event has a slight impact on the surrounding environment); Second state coefficient: 0.4 (indicates that the coordination between the bronchial morphology and the surrounding environment is poor); Collect actual bronchial morphological information: the middle segment of the right main bronchus has a narrow lumen and thickened walls; review previous detection information: there was slight wall thickening last year, and the degree of stenosis has worsened this year; compare the preset status level mapping relationship: find a mapping relationship that matches this information, which indicates that lumen stenosis and wall thickening are associated with specific status levels.

[0103] The first state coefficient (0.6) and the second state coefficient (0.4) are substituted into the matched state level mapping relationship; the state level of the bronchus is calculated according to the rules or algorithms in the mapping relationship; for example, if the mapping relationship stipulates that lumen stenosis and wall thickening are associated with the state level of "moderate lesions", and when the first state coefficient and the second state coefficient are in the range of 0.5-0.7, the state level is "moderate lesions", then the patient's bronchial state level is determined to be "moderate lesions"; through this example, we can see how step S153 combines the actual morphology of the bronchus, previous detection information, and the first state coefficient and the second state coefficient to determine the state level of the bronchus. This step provides an important basis for subsequent treatment planning and prognosis evaluation.

[0104] In one embodiment of the present application, a matching table of state level mapping relationships is collected, and the matching table of state level mapping relationships is shown in Table 3: Table 3. Matching table of state level mapping relationships

[0105] Now, there is a patient whose information is as follows: The actual morphology of the bronchi: narrow lumen and thickened wall; Previous test information: The lumen was slightly narrowed last year, but this year the narrowing worsened and the wall thickened; First state coefficient: 0.6; Second state coefficient: 0.4; According to the matching table of the state level mapping relationship, the row that best matches the patient's information is found: the lumen is narrow and the wall is thickened; the degree of stenosis gradually worsens and the wall is thickened; the first state coefficient range is 0.5-0.7; the second state coefficient range is 0.3-0.6; therefore, the patient's bronchial state level is "moderate lesion".

[0106] See also Figure 7 , Figure 7 : is a schematic diagram of the structural composition of a bronchial fluorescence image boundary recognition system based on visual detection in an embodiment of the present invention; the bronchial fluorescence image boundary recognition system based on visual detection includes: A fluorescence imaging module 21 is used to acquire the position of the bronchus and trigger corresponding visual detection based on the position of the bronchus to acquire a bronchial fluorescence image; An image detection module 22 is configured to determine a bronchial region and an environmental region based on detection of a bronchial fluorescence image, and to determine multiple edge portions of the bronchus based on edge detection of the bronchial region; a morphology detection module 23, configured to determine the boundary of the bronchus based on identification of multiple edge portions of the bronchus, and to determine the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image; An abnormal feature module 24 is used to determine a core review area based on the actual morphology of the bronchus and the age of the user, and to determine an abnormal feature combination based on the identification of the core review area; The status grade module 25 is used to determine abnormal events based on the abnormal feature combination and the core review area, and to determine the status grade of the bronchus based on the abnormal events, the actual morphology of the bronchus and the surrounding environment of the bronchus.

[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A boundary recognition method for bronchial fluorescence images based on visual detection, characterized in that: include: Acquiring the location of the bronchus, and triggering corresponding visual detection based on the location of the bronchus to acquire a bronchial fluorescence image; determining a bronchial region and an environmental region based on detection of a bronchial fluorescence image, and determining a plurality of bronchial edge portions based on edge detection of the bronchial region; determining the boundary of the bronchus based on identification of a plurality of edge portions of the bronchus, and determining the actual morphology of the bronchus based on the boundary of the bronchus and a bronchial fluorescence image; Determine a core review area based on the actual morphology of the bronchus and the user's age, and determine an abnormal feature combination based on the identification of the core review area; Abnormal events are determined based on the combination of abnormal features and the core review area, and the bronchial status grade is determined based on the abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus.

2. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 1, characterized in that: The collecting of the position of the bronchus and triggering corresponding visual detection based on the position of the bronchus to collect the bronchial fluorescence image include: Performing bronchial position detection and determining the bronchial position based on the bronchial position detection; and simultaneously collecting multiple posture parameters of the user and determining the user's current posture based on the multiple posture parameters and the user's height; The area to be detected is determined according to the position of the bronchus and the current posture of the user, and the visual detection part on the peripheral side is triggered according to the spatial position of the area to be detected, so that the visual detection part moves relative to the area to be detected to perform visual detection on the area to be detected and output the collected bronchial fluorescence image.

3. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 1, characterized in that: The determining of the bronchial region and the environmental region based on the detection of the bronchial fluorescence image, and determining the multiple edge portions of the bronchus based on the edge detection of the bronchial region, includes: Acquiring a bronchial fluorescence image, determining a plurality of branch features based on detection of the bronchial fluorescence image, determining a relative distance between two adjacent branch features based on positions of the plurality of branch features, and determining a bronchial region based on the relative distance between the two adjacent branch features and a connectivity state between the two adjacent branch features; In the bronchial region, the bronchial features are highlighted based on image preprocessing of the bronchial region. At the same time, the bronchial features are traversed, multiple edge contours of the bronchial features are marked according to the traversal of the bronchial features, and multiple edge portions of the bronchus are determined according to edge detection of the multiple edge contours. The non-bronchial area is determined based on the comparison between the bronchial area and the bronchial fluorescence image, the environmental area is determined according to the color areas of the non-bronchial area and the bronchial area, and the lateral environment of the bronchus is determined based on the identification of the environmental area.

4. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 1, characterized in that: The step of determining the boundary of the bronchus according to the identification of the plurality of edge portions of the bronchus, and determining the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image, comprises: Multiple edge portions of the bronchus are collected, and multiple edge segments are determined based on the synchronous detection of the multiple edge portions of the bronchus; a coherent overall segment is formed based on the synthesis of the multiple edge segments, and the coherent overall segment serves as the boundary of the bronchus.

5. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 4, characterized in that: The step of determining the boundary of the bronchus according to the identification of the plurality of edge portions of the bronchus, and determining the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image, further includes: Match the bronchial boundary to the bronchial fluorescence image and output the morphological region of the bronchus; The corners of the morphological region of the bronchus are marked, and the actual region of the bronchus is determined based on the optimization of the corners of the morphological region of the bronchus; and the actual morphology of the bronchus is determined based on the identification of the actual region of the bronchus.

6. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 1, characterized in that: The determining of the core review area based on the actual morphology of the bronchus and the age of the user, and determining the abnormal feature combination based on the identification of the core review area, include: Determining a plurality of branch tubes based on the actual morphology of the bronchus, and marking the actual morphologies of the plurality of branch tubes, wherein the plurality of branch tubes serve as various parts of the bronchus; In each branch tube body, the corresponding surface abnormality features are determined based on the detection of the branch tube body, multiple abnormal areas are determined based on the actual shapes of the multiple branch tube bodies and the corresponding surface abnormality features, and the core review area is determined based on the multiple abnormal areas, the user's age and the review mapping relationship.

7. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 6, characterized in that: The determining of the core review area based on the actual morphology of the bronchus and the age of the user, and determining the abnormal feature combination according to the identification of the core review area, further includes: Perform surface recognition and structural recognition on the core review area, determine a first sub-abnormal feature based on the surface recognition of the core review area, determine a second sub-abnormal feature based on the structural recognition of the core review area, and determine an abnormal feature combination based on the first sub-abnormal feature and the second sub-abnormal feature.

8. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 1, characterized in that: Determining abnormal events based on the abnormal feature combination and the core review area, and determining the bronchial status level based on the abnormal events, the actual morphology of the bronchus, and the surrounding environment of the bronchus, includes: Abnormal feature combinations are collected, and a corresponding abnormal list is determined based on the matching of the abnormal feature combinations with the actual morphology of the bronchus. The abnormal list presents various parts of the bronchus; abnormal events are determined based on the abnormal list and the core review area.

9. The method for boundary recognition of bronchial fluorescence images based on visual detection according to claim 8, characterized in that: The step of determining an abnormal event based on the abnormal feature combination and the core review area, and determining the bronchial status level based on the abnormal event, the actual morphology of the bronchus, and the surrounding environment of the bronchus, further includes: Collecting the surrounding environment of the bronchus, determining a first state coefficient based on the surrounding environment of the bronchus and abnormal events, and determining a second state coefficient based on the surrounding environment of the bronchus and the actual morphology of the bronchus; The state level mapping relationship corresponding to the actual morphology of the bronchus and previous detection information of the bronchus is matched, and the state level of the bronchus is determined according to the state level mapping relationship, the first state coefficient and the second state coefficient.

10. A bronchial fluorescence image boundary recognition system based on visual detection, characterized in that: The bronchial fluorescence image boundary recognition system based on visual detection is applied to the bronchial fluorescence image boundary recognition method based on visual detection according to any one of claims 1 to 9, and the bronchial fluorescence image boundary recognition system based on visual detection includes: A fluorescence imaging module is used to acquire the location of the bronchi and trigger corresponding visual detection based on the location of the bronchi to acquire bronchial fluorescence images; An image detection module, configured to determine a bronchial region and an environmental region based on detection of a bronchial fluorescence image, and to determine multiple edge portions of the bronchus based on edge detection of the bronchial region; a morphology detection module, configured to determine the boundary of the bronchus based on identification of multiple edge portions of the bronchus, and determine the actual morphology of the bronchus based on the boundary of the bronchus and the bronchial fluorescence image; An abnormal feature module is used to determine a core review area based on the actual morphology of the bronchus and the user's age, and to determine an abnormal feature combination based on the identification of the core review area; The status grade module is used to determine abnormal events based on the abnormal feature combination and the core review area, and to determine the status grade of the bronchus based on the abnormal events, the actual morphology of the bronchus and the surrounding environment of the bronchus.

Images