Method for positioning focus under bronchoscope based on reinforcement learning

Abstract

The invention relates to the technical field of reinforcement learning, in particular to a bronchoscope focus positioning method based on reinforcement learning, and the method comprises the steps: obtaining a current bronchoscope image, carrying out the connected domain extraction of the current bronchoscope image, and screening out a channel connected domain and a cartilage candidate edge from the obtained connected domain; screening out a real cartilage edge; constructing channel cartilage edge clusters, and determining a common ring coefficient between every two real cartilage edges in each channel cartilage edge cluster; screening out a common-ring edge cluster, and identifying a spacing line; determining an incomplete factor corresponding to each spacing line; determining a consistency index corresponding to each incomplete spacing line; and if the consistency index corresponding to the incomplete spacing line is greater than a preset consistency threshold value, determining the incomplete spacing line as a focus spacing line. According to the invention, the location of the focus area of the bronchoscope image is realized, and the location accuracy of the focus area of the bronchoscope image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of reinforcement learning, and particularly relates to a method for lesion localization under a bronchoscope based on reinforcement learning. Background Art

[0002] When there are lesions in a patient's bronchus, a doctor can observe the situation inside the airway to identify the lesions (such as tumors, inflammations, infections, etc.), and make corresponding treatments according to the shape and position of the lesions. Therefore, the accurate localization of the lesion area in bronchoscope images is a very important step in bronchoscopy, especially playing a crucial role in the diagnosis and treatment process. Currently, when positioning the target area, the commonly used method is to perform localization segmentation on the target area through the difference in gray values.

[0003] However, when localizing the lesion area in bronchoscope images through the difference in gray values, the following technical problems often exist: Since the gray difference between the lesion area and the normal tissue area in bronchoscope images is not large, directly localizing the lesion area in bronchoscope images through the difference in gray values may lead to misjudgment of lesion pixel points, resulting in poor accuracy of the localization of the lesion area in bronchoscope images. Summary of the Invention

[0004] To solve the technical problem of poor accuracy in localizing the lesion area in bronchoscope images, the present invention proposes a method for lesion localization under a bronchoscope based on reinforcement learning.

[0005] In a first aspect, the present invention provides a method for lesion localization under a bronchoscope based on reinforcement learning, and the method includes: Obtain the current bronchoscope image, perform connected component extraction on the current bronchoscope image, and screen out the channel connected components and cartilage candidate edges from the obtained connected components; Determine the connected component closest to the cartilage candidate edge as the representative connected component corresponding to the cartilage candidate edge, and screen out the true cartilage edges from all cartilage candidate edges whose corresponding representative connected components are the same connected component; Constitute the channel cartilage edge clusters with the true cartilage edges having the same corresponding representative connected component, and determine the co-loop coefficient between every two true cartilage edges in each channel cartilage edge cluster; According to the co-loop coefficients between different true cartilage edges in each channel cartilage edge cluster, screen out the co-loop edge clusters from each channel cartilage edge cluster, and identify the spacer lines from each co-loop edge cluster; According to the distribution of each spacer line, determine the corresponding incomplete factor of each spacer line. If the incomplete factor corresponding to the spacer line is greater than the preset incomplete threshold, then determine the spacer line as an incomplete spacer line; Based on the current bronchoscope image and its previous frame bronchoscope image, determine the consistency index corresponding to each incomplete septal line; If the consistency index corresponding to the incomplete septal line is greater than the preset consistency threshold, then determine the incomplete septal line as the lesion septal line.

[0006] Combined with the above first aspect, in a possible implementation manner, the screening of the channel connected domain and the cartilage candidate edge from the obtained connected domains includes: Perform threshold segmentation on all connected domains in the current bronchoscope image, and determine the connected domains in the category of connected domains with lower gray values among the two categories of connected domains obtained by the segmentation as the channel connected domain; Screen out reference pixel points from the connected domains in the current bronchoscope image; Screen out the reference pixel points whose corresponding first pixel points are also reference pixel points from all the reference pixel points as the selected pixel points; Determine the line formed by the continuous selected pixel points as the cartilage candidate edge.

[0007] Combined with the above first aspect, in a possible implementation manner, the screening of the reference pixel points from the connected domains in the current bronchoscope image includes: Determine any connected domain in the current bronchoscope image as the marked connected domain, and determine any pixel point in the marked connected domain as the marked pixel point; Screen out the pixel point with the smallest gradient value difference from the marked pixel point within the preset neighborhood corresponding to the marked pixel point as the first pixel point corresponding to the marked pixel point; Screen out the pixel point with the largest corresponding gradient value from the preset neighborhood corresponding to the marked pixel point as the second pixel point corresponding to the marked pixel point; If the first pixel point and the second pixel point corresponding to the marked pixel point are the same pixel point, then determine the marked pixel point as the reference pixel point.

[0008] Combined with the above first aspect, in a possible implementation manner, the screening of the real cartilage edge from all the cartilage candidate edges whose corresponding representative connected domains are the same connected domain includes: Determine the representative connected domain corresponding to any cartilage candidate edge as the reference connected domain; By means of random sampling, randomly extract different numbers of cartilage candidate edges from all the cartilage candidate edges whose corresponding representative connected domains are the reference connected domain, and form a cartilage candidate edge set with each extraction of the cartilage candidate edges to obtain multiple cartilage candidate edge sets under the reference connected domain; Partition the cartilage candidate edges in each set of cartilage candidate edges on the same side to obtain the respective cartilage candidate edge groups corresponding to each set of cartilage candidate edges; Determine the information entropy of the distances between the centroids of all cartilage candidate edges and the centroid of the corresponding representative connected domain in each cartilage candidate edge group as the distance information entropy corresponding to each cartilage candidate edge group; Determine the mean of the DTW distances between different cartilage candidate edges in each cartilage candidate edge group as the target difference factor corresponding to each cartilage candidate edge group; Determine the target loss value corresponding to each set of cartilage candidate edges according to all the distance information entropy and target difference factors corresponding to each set of cartilage candidate edges; Select the set of cartilage candidate edges with the smallest corresponding target loss value from all the sets of cartilage candidate edges under the reference connected domain as the temporary edge set, and determine each cartilage candidate edge in the temporary edge set as a true cartilage edge.

[0009] Combined with the first aspect above, in a possible implementation, the formula for the target loss value corresponding to the set of cartilage candidate edges is: ; where E is the target loss value corresponding to the set of cartilage candidate edges; is a hyperbolic function; is the variance of all the distance information entropy corresponding to the set of cartilage candidate edges; is the mean of all the target difference factors corresponding to the set of cartilage candidate edges.

[0010] Combined with the first aspect above, in a possible implementation, the determination of the co-ring coefficient between every two true cartilage edges in each channel cartilage edge cluster includes: Determine any true cartilage edge as a marked cartilage edge, and draw a ray passing through the centroid of the marked cartilage edge with the centroid of the representative connected domain corresponding to the marked cartilage edge as the endpoint, denoted as the target ray corresponding to the marked cartilage edge; Determine the intersection of the target ray corresponding to the marked cartilage edge and the representative connected domain as the target line segment corresponding to the marked cartilage edge; Select the true cartilage edge farthest from the centroid of the marked cartilage edge from all the true cartilage edges intersecting with the target line segment corresponding to the marked cartilage edge as the reference edge corresponding to the marked cartilage edge; Determine the distance between the centroid of the marked cartilage edge and the centroid of the corresponding representative connected domain as the first distance factor corresponding to the marked cartilage edge; Determine the distance between the centroid of the reference edge corresponding to the marked cartilage edge and the centroid of the representative connected domain corresponding to the marked cartilage edge as the second distance factor corresponding to the marked cartilage edge; Determine the ratio of the first distance factor and the second distance factor corresponding to the marked cartilage edge as the reference ratio corresponding to the marked cartilage edge; Determine any one of the channel cartilage edge clusters as the marked cartilage edge cluster, and determine the co-ring coefficient between every two true cartilage edges in the marked cartilage edge cluster according to the reference ratios corresponding to every two true cartilage edges in the marked cartilage edge cluster.

[0011] Combined with the above first aspect, in a possible implementation manner, the determining the co-ring coefficient between every two true cartilage edges in the marked cartilage edge cluster according to the reference ratios corresponding to every two true cartilage edges in the marked cartilage edge cluster includes: Determine any two true cartilage edges in the marked cartilage edge cluster as the first marked edge and the second marked edge respectively; Perform elliptical fitting on the union of the first marked edge and the second marked edge, and determine the fitting loss obtained by the elliptical fitting as the non-co-ring factor between the first marked edge and the second marked edge; Determine the co-ring coefficient between the first marked edge and the second marked edge according to the non-co-ring factor between the first marked edge and the second marked edge, and the difference between the reference ratio corresponding to the first marked edge and the reference ratio corresponding to the second marked edge.

[0012] Combined with the above first aspect, in a possible implementation manner, the formula corresponding to the co-ring coefficient between the first marked edge and the second marked edge is: ; where is the co-ring coefficient between the first marked edge and the second marked edge; is the exponential function with the natural constant as the base; is the non-co-ring factor between the first marked edge and the second marked edge; is the absolute value function; is the reference ratio corresponding to the first marked edge; is the reference ratio corresponding to the second marked edge.

[0013] Combined with the above first aspect, in a possible implementation manner, the screening out the co-ring edge clusters from each channel cartilage edge cluster according to the co-ring coefficients between different true cartilage edges in each channel cartilage edge cluster includes: Identify any one of the channel cartilage edge clusters as the marked cartilage edge cluster, and identify any one of the true cartilage edges in the marked cartilage edge cluster as the temporary cartilage edge; Select the true cartilage edge with the largest co-ring coefficient between the temporary cartilage edge from the marked cartilage edge cluster as the first edge corresponding to the temporary cartilage edge; The true cartilage edges with the same first edge in the marked cartilage edge cluster form a co-ring possible cluster; If the true cartilage edges in the co-ring possible cluster are on different sides, then determine the co-ring possible cluster as the co-ring edge cluster.

[0014] Combined with the above first aspect, in a possible implementation, the determining the incomplete factor corresponding to each spacer line according to the distribution of each spacer line includes: All co-ring edge clusters in each channel cartilage edge cluster form a co-ring edge cluster set; Identify any one spacer line as the marked spacer line, and select the spacer line closest to the marked spacer line from each co-ring edge cluster in the co-ring edge cluster set to which the marked spacer line belongs to form the first reference spacer line set corresponding to the marked spacer line; Sort the spacer lines in the first reference spacer line set in ascending order according to the lengths of the spacer lines in the first reference spacer line set to obtain the first reference spacer line sequence; Sort the spacer lines in the first reference spacer line set in the order from top to bottom and from left to right according to the positions of the spacer lines in the current bronchoscope image to obtain the first temporary spacer line sequence; Determine the sorting difference between the first reference spacer line sequence and the first temporary spacer line sequence as the first quantity difference corresponding to the marked spacer line; All spacer lines in the first reference spacer line set corresponding to the marked spacer line except the marked spacer line form the second reference spacer line set corresponding to the marked spacer line; Sort the spacer lines in the second reference spacer line set in ascending order according to the lengths of the spacer lines in the second reference spacer line set to obtain the second reference spacer line sequence; Sort the spacer lines in the second reference spacer line set in the order from top to bottom and from left to right according to the positions of the spacer lines in the current bronchoscope image to obtain the second temporary spacer line sequence; Determine the sorting difference between the second reference spacer line sequence and the second temporary spacer line sequence as the second quantity difference corresponding to the marked spacer line; Determine the ray from the centroid of the connected domain of the channel to which the marked interval line belongs to the centroid of the marked interval line as the reference ray corresponding to the marked interval line; Determine the distance between the centroid of each interval line in the second reference interval line set and the reference ray as the reference distance coefficient corresponding to each interval line in the second reference interval line set; Determine the mean value of the reference distance coefficients corresponding to all interval lines in the second reference interval line set as the average reference distance corresponding to the marked interval line; Determine the incomplete factor corresponding to the marked interval line according to the first quantity difference, the second quantity difference and the average reference distance corresponding to the marked interval line.

[0015] In a second aspect, the present invention provides a system for accurately positioning a lesion area in a bronchoscope image, the system comprising: An acquisition extraction and screening module, configured to acquire a current bronchoscope image, perform connected domain extraction on the current bronchoscope image, and screen out a channel connected domain and a cartilage candidate edge from the obtained connected domains; A determination and screening module, configured to determine the channel connected domain closest to the cartilage candidate edge as the representative connected domain corresponding to the cartilage candidate edge, and screen out the true cartilage edge from all cartilage candidate edges corresponding to the same representative connected domain; A construction and determination module, configured to form a channel cartilage edge cluster with the true cartilage edges corresponding to the same representative connected domain, and determine the co-ring coefficient between every two true cartilage edges in each channel cartilage edge cluster; A screening and recognition module, configured to screen out a co-ring edge cluster from each channel cartilage edge cluster according to the co-ring coefficients between different true cartilage edges in each channel cartilage edge cluster, and recognize an interval line from each co-ring edge cluster; A double determination module, configured to determine the incomplete factor corresponding to each interval line according to the distribution of each interval line. If the incomplete factor corresponding to the interval line is greater than a preset incomplete threshold, then determine the interval line as an incomplete interval line; A consistency index determination module, configured to determine the consistency index corresponding to each incomplete interval line according to the current bronchoscope image and its previous frame of bronchoscope image; A judgment and determination module, configured to, if the consistency index corresponding to the incomplete interval line is greater than a preset consistency threshold, then determine the incomplete interval line as a lesion interval line.

[0016] In a third aspect, a server is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the method in the first aspect or any possible implementation manner of the first aspect.

[0017] In a fourth aspect, a computer program product is provided, which includes computer program code that, when running on a computer, causes the computer to execute the method in the above first aspect or any possible implementation manner of the first aspect.

[0018] In a fifth aspect, a computer-readable storage medium is provided, which stores computer program code that, when running on a computer, causes the computer to execute the method in the above first aspect or any possible implementation manner of the first aspect.

[0019] The present invention has the following beneficial effects: A method for lesion localization under a bronchoscope based on reinforcement learning according to the present invention realizes the localization of the lesion area in a bronchoscope image, solves the technical problem of poor accuracy in localizing the lesion area in a bronchoscope image, and improves the accuracy of localizing the lesion area in a bronchoscope image. Compared with localizing the lesion area in a bronchoscope image by different gray values, the present invention comprehensively considers multiple features related to the lesion situation in a bronchoscope image, such as the co-ring coefficient, the incomplete factor, and the consistency index, etc., thereby improving the accuracy of quantifying the lesion spacing line, and further improving the accuracy of localizing the lesion area in a bronchoscope image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for describing the embodiments or the prior art. Obviously, the following described drawings are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of a method for lesion localization under a bronchoscope based on reinforcement learning according to the present invention; Figure 2 It is a schematic diagram of the composition structure of a system for accurately localizing the lesion area in a bronchoscope image according to the present invention; Figure 3 It is a schematic diagram of the structure of a computer device according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes the implementation manners, structures, features, and effects of the technical solutions proposed according to the present invention in detail in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0024] In order to facilitate doctors to identify the morphology and location of lesions in the bronchus, machine vision is used to accurately locate the lesions in the bronchoscope images. Since there are many cartilaginous protrusions on the bronchial wall in the bronchoscope images, and the changes in the cartilaginous morphology and channels will be confused with the influence of the lesions on the bronchial wall, it is difficult to locate the lesions.

[0025] The advantages of the present invention over the prior art are as follows: In order to accurately locate the lesion area in the bronchoscope image, this method uses a bronchoscope to collect images of the patient's bronchus, preprocesses them, and segments the bronchial connectivity domain. The edges of the bronchial cartilage are extracted through the gray-scale changes in the image, and the lesions are detected according to the integrity of the hose edges, providing good data support for subsequent lesion location.

[0026] Reference Figure 1 , shows the flow of some embodiments of a method for lesion location under a bronchoscope based on reinforcement learning according to the present invention. The method for lesion location under a bronchoscope based on reinforcement learning includes the following steps: Step S1, obtain the current bronchoscope image, perform connectivity domain extraction on the current bronchoscope image, and screen out the channel connectivity domain and cartilage candidate edges from the obtained connectivity domains.

[0027] Among them, the current bronchoscope image can be the bronchoscope image collected at the current moment.

[0028] As an example, this step may include the following steps: The first step is to obtain the current bronchoscope image.

[0029] It should be noted that before bronchoscopy, a series of imaging examinations are usually carried out, such as chest X-ray or CT (Computed Tomography) scan, to help doctors understand the overall condition of the airway and preliminarily locate the possible lesion areas. After preliminarily determining the possible lesion areas, in order to reduce the discomfort of the patient, local anesthesia or general anesthesia will be used on the patient. Local anesthesia will make the airway less sensitive and reduce discomfort. After the anesthesia is completed, the doctor inserts the bronchoscope through the mouth or nose and enters the bronchus along the trachea. The camera will capture the images in the airway in real time. The images collected at this time are bronchoscope images. The real-time collected bronchoscope images of the patient can be recorded and saved for subsequent diagnosis and treatment.

[0030] The second step is to extract the connected components of the current bronchoscope image.

[0031] For example, the connected components of the current bronchoscope image can be extracted through a connected component extraction algorithm to obtain multiple connected components.

[0032] It should be noted that canny edge detection can also be performed on the bronchial images of the patient. Since there are significant differences between the acquisition platform interface and the bronchial images, canny edge detection can well detect the edges of the patient's bronchial images. Therefore, some connected components of the bronchus and several edges inside can be extracted.

[0033] The third step is to perform threshold segmentation on all the connected components in the above current bronchoscope image, and determine the connected components in the category of connected components with lower gray values among the two categories of connected components obtained by the segmentation as the channel connected components.

[0034] It should be noted that Otsu threshold segmentation can be used to perform threshold segmentation on the obtained bronchial connected components. Since the between-class difference obtained by Otsu threshold segmentation is the largest, the bronchial connected components can be segmented into the bronchial wall with higher gray values and the channel part with lower gray values. Among them, the channel part with lower gray values is the channel connected component.

[0035] It should be noted that for the bronchial connected component image of the patient obtained by the above operations, since the structure of the bronchus is formed by the combination of incomplete C-shaped cartilage rings, the cartilage part can keep the trachea unobstructed. Therefore, in the bronchoscope image, the bronchus is ring-shaped; and the appearance of lesions in the bronchus will cause changes in the ring structure of the bronchus. Therefore, lesion recognition can be performed by analyzing the ring structure of the bronchus in the follow-up.

[0036] The fourth step of screening reference pixel points from the connected components in the above current bronchoscope image may include the following sub-steps: First sub-step: Determine any connected region in the current bronchoscope image as the marked connected region, and determine any pixel point in the marked connected region as the marked pixel point.

[0037] Second sub-step: Screen out the pixel point with the smallest gradient value difference from the marked pixel point within the corresponding preset neighborhood of the marked pixel point as the first pixel point corresponding to the marked pixel point.

[0038] Among them, the preset neighborhood can be a pre-set neighborhood, which can be an eight-neighborhood.

[0039] Third sub-step: Screen out the pixel point with the largest corresponding gradient value from the preset neighborhood corresponding to the marked pixel point as the second pixel point corresponding to the marked pixel point.

[0040] Fourth sub-step: If the first pixel point and the second pixel point corresponding to the marked pixel point are the same pixel point, then determine the marked pixel point as the reference pixel point.

[0041] Fifth step: Screen out the reference pixel points whose corresponding first pixel points are also reference pixel points from all the reference pixel points as the selected pixel points.

[0042] Among them, the selected pixel points can be reference pixel points, and the corresponding first pixel points of the selected pixel points are also reference pixel points.

[0043] Sixth step: Determine the line formed by consecutive selected pixel points as the cartilage candidate edge.

[0044] Among them, the number of selected pixel points in the cartilage candidate edge is at least greater than 1. The cartilage candidate edge is also called the selected line. For the obtained selected line, it contains the cartilage edge and the edges of some other tissues in the bronchoscope image. Therefore, it is necessary to screen out the cartilage edge according to the circular arrangement of the cartilage edge.

[0045] It should be noted that since the C-shaped cartilage combination of the bronchus will cause a tubular structure with a circular protrusion inside the bronchus, the cartilage edge in a normal bronchoscope image will show a C-shaped superimposed arrangement. Therefore, it is necessary to obtain the cartilage edge in the image according to the arrangement of the partial cartilage edge.

[0046] Step S2: Determine the channel connected region closest to the cartilage candidate edge as the representative connected region corresponding to the cartilage candidate edge, and screen out the true cartilage edge from all the cartilage candidate edges whose corresponding representative connected regions are the same connected region.

[0047] Among them, the channel connected region closest to the cartilage candidate edge can be the channel connected region whose corresponding centroid is closest to the centroid of the cartilage candidate edge.

[0048] As an example, this step may include the following steps: In the first step, the representative connected region corresponding to any cartilage candidate edge is determined as the reference connected region.

[0049] In the second step, by means of random sampling, different numbers of cartilage candidate edges are randomly selected from all cartilage candidate edges whose corresponding representative connected region is the above-mentioned reference connected region, and the cartilage candidate edges selected each time are formed into a cartilage candidate edge set, obtaining multiple cartilage candidate edge sets under the above-mentioned reference connected region.

[0050] For example, if the cartilage candidate edges whose corresponding representative connected region is the reference connected region successively include: the first cartilage candidate edge, the second cartilage candidate edge, and the third cartilage candidate edge, and the number of random selections is 2 and 3, then 4 cartilage candidate edge sets under the reference connected region can be obtained, which can be successively {the first cartilage candidate edge, the second cartilage candidate edge}, {the first cartilage candidate edge, the third cartilage candidate edge}, {the second cartilage candidate edge, the third cartilage candidate edge}, and {the first cartilage candidate edge, the second cartilage candidate edge, the third cartilage candidate edge}.

[0051] In the third step, the cartilage candidate edges in each cartilage candidate edge set are divided into the same side, obtaining each cartilage candidate edge group corresponding to each cartilage candidate edge set.

[0052] It should be noted that the other cartilage candidate edges on the line connecting the centroid of the cartilage candidate edge and the centroid of the channel connected region are denoted as the cartilage edges on the same side.

[0053] In the fourth step, the information entropy of the distance between the centroid of all cartilage candidate edges in each cartilage candidate edge group and the centroid of its corresponding representative connected region is determined as the distance information entropy corresponding to each cartilage candidate edge group.

[0054] In the fifth step, the mean value of the DTW distances between different cartilage candidate edges in each cartilage candidate edge group is determined as the target difference factor corresponding to each cartilage candidate edge group.

[0055] Among them, the DTW distance is the corresponding shortest path obtained by using the DTW algorithm.

[0056] In the sixth step, according to all the distance information entropies and target difference factors corresponding to each cartilage candidate edge set, the target loss value corresponding to each cartilage candidate edge set is determined.

[0057] For example, the formula for determining the target loss value corresponding to the cartilage candidate edge set can be: ; where E is the target loss value corresponding to the cartilage candidate edge set. is a hyperbolic function. is the variance of all distance information entropies corresponding to the cartilage candidate edge set. is the mean of all target difference factors corresponding to the cartilage candidate edge set.

[0058] It should be noted that when is smaller, it often indicates that the edge distance information entropy values between the cartilage candidate edge groups are relatively adjacent, and it often indicates that the edges in the cartilage candidate edge groups are arranged in a ring-like pattern. When is smaller, it often indicates that the shapes between the edges in the cartilage edge cluster are more likely to be similar.

[0059] Step 7: Screen out the cartilage candidate edge set with the smallest corresponding target loss value from all cartilage candidate edge sets under the above reference connected domain as the temporary edge set, and determine each cartilage candidate edge in the temporary edge set as a real cartilage edge.

[0060] Step S3: Construct a channel cartilage edge cluster from the real cartilage edges corresponding to the same representative connected domain, and determine the co-ring coefficient between every two real cartilage edges in each channel cartilage edge cluster.

[0061] It should be noted that for several cartilage edges in the bronchoscope image obtained by the above operations, there will be cases where the circular edge is disconnected due to lesions on the bronchial surface. However, since the cartilage in the bronchus is in a C-shaped structure, the cartilage edge in the bronchus itself is not a closed circular structure, and due to the shooting angle of the bronchoscope, the tracheal circular structure near the bottom is blocked by the circular structure above. Therefore, it is necessary to analyze the actual integrity of the cartilage edge.

[0062] As an example, this step may include the following steps: Step 1: Determine any real cartilage edge as the marked cartilage edge. Taking the centroid of the representative connected domain corresponding to the above marked cartilage edge as an endpoint, draw a ray passing through the centroid of the above marked cartilage edge, which is denoted as the target ray corresponding to the above marked cartilage edge.

[0063] Step 2: Determine the intersection of the target ray corresponding to the above marked cartilage edge and the representative connected domain as the target line segment corresponding to the above marked cartilage edge.

[0064] Step 3: Screen out the real cartilage edge that is farthest from the centroid of the above marked cartilage edge from all real cartilage edges intersecting with the target line segment corresponding to the above marked cartilage edge as the reference edge corresponding to the above marked cartilage edge.

[0065] In the fourth step, the distance between the centroid of the marked cartilage edge and the centroid of the corresponding connected domain is determined as the first distance factor corresponding to the marked cartilage edge.

[0066] In the fifth step, the distance between the centroid of the reference edge corresponding to the marked cartilage edge and the centroid of the corresponding connected domain is determined as the second distance factor corresponding to the marked cartilage edge.

[0067] In the sixth step, the ratio of the first distance factor and the second distance factor corresponding to the marked cartilage edge is determined as the reference ratio corresponding to the marked cartilage edge.

[0068] In the seventh step, any cluster of cartilage edge channels is determined as the marked cartilage edge cluster, and the co - ring coefficient between every two true cartilage edges in the marked cartilage edge cluster can be determined according to the reference ratios corresponding to every two true cartilage edges in the marked cartilage edge cluster, which may include the following sub - steps: In the first sub - step, any two true cartilage edges in the marked cartilage edge cluster are respectively determined as the first marked edge and the second marked edge.

[0069] In the second sub - step, an ellipse fitting is performed on the union of the first marked edge and the second marked edge, and the fitting loss obtained from the ellipse fitting is determined as the non - co - ring factor between the first marked edge and the second marked edge.

[0070] In the third sub - step, according to the non - co - ring factor between the first marked edge and the second marked edge, and the difference between the reference ratio corresponding to the first marked edge and the reference ratio corresponding to the second marked edge, the co - ring coefficient between the first marked edge and the second marked edge is determined.

[0071] For example, the formula for determining the co - ring coefficient between the first marked edge and the second marked edge can be: ; where is the co - ring coefficient between the first marked edge and the second marked edge. is the exponential function with the natural constant as the base. is the non - co - ring factor between the first marked edge and the second marked edge. is the absolute - value function. is the reference ratio corresponding to the first marked edge. is the reference ratio corresponding to the second marked edge.

[0072] It should be noted that when is larger, it often indicates that the edges belong to the same ellipse and are in the same layer in the channel, so the co - ring coefficient of the edges is higher.

[0073] Step S4. According to the co-ring coefficients between different true cartilage edges in each channel cartilage edge cluster, filter out the co-ring edge clusters from each channel cartilage edge cluster, and identify the spacer lines from each co-ring edge cluster.

[0074] It should be noted that since there are regularly arranged C-shaped annular cartilages in the normal bronchus, there will normally be regularly arranged notches in the cartilage edges. Therefore, the integrity of the cartilage edges can be analyzed based on the notches in the co-ring edges.

[0075] As an example, this step may include the following steps: First step, determine any one channel cartilage edge cluster as the marked cartilage edge cluster, and determine any one true cartilage edge in the above marked cartilage edge cluster as the temporary cartilage edge.

[0076] Second step, screen out the true cartilage edge with the largest co-ring coefficient with the above temporary cartilage edge from the above marked cartilage edge cluster as the first edge corresponding to the above temporary cartilage edge.

[0077] Third step, form the co-ring possible clusters with the true cartilage edges having the same corresponding first edge in the above marked cartilage edge cluster.

[0078] Fourth step, if the true cartilage edges in the co-ring possible clusters are on different sides, determine the co-ring possible clusters as the co-ring edge clusters.

[0079] Among them, the true cartilage edges on different sides often indicate that these true cartilage edges are not on the same side of the channel.

[0080] Fifth step, identify the spacer lines from each co-ring edge cluster.

[0081] For example, the endpoints between adjacent edges in each co-ring edge cluster can be obtained, and the optimal fitting line between the endpoints can be obtained by fitting an ellipse according to the least squares method, and several spacer lines of the co-ring edge clusters are obtained in total.

[0082] Step S5. According to the distribution of each spacer line, determine the incomplete factor corresponding to each spacer line. If the incomplete factor corresponding to the spacer line is greater than the preset incomplete threshold, determine the spacer line as the incomplete spacer line.

[0083] Among them, the preset incomplete threshold can be a threshold set in advance, which can be 0.7.

[0084] As an example, this step may include the following steps: First step, form the co-ring edge cluster set with all the co-ring edge clusters in each channel cartilage edge cluster.

[0085] In the second step, any one of the spaced lines is determined as the marked spaced line, and the spaced line closest to the marked spaced line is selected from each co-ring edge cluster in the co-ring edge cluster set to which the marked spaced line belongs, forming the first reference spaced line set corresponding to the marked spaced line.

[0086] In the third step, according to the lengths of the spaced lines in the first reference spaced line set, the spaced lines in the first reference spaced line set are sorted in ascending order to obtain the first reference spaced line sequence.

[0087] In the fourth step, according to the positions of the spaced lines in the first reference spaced line set in the current bronchoscope image, the spaced lines in the first reference spaced line set are sorted in the order from top to bottom and from left to right to obtain the first temporary spaced line sequence.

[0088] In the fifth step, the sorting difference between the first reference spaced line sequence and the first temporary spaced line sequence is determined as the first quantity difference corresponding to the marked spaced line.

[0089] For example, if the first reference spaced line sequence is {the first spaced line, the second spaced line, the third spaced line, the fourth spaced line, the fifth spaced line}, and the first temporary spaced line sequence is {the first spaced line, the third spaced line, the fifth spaced line, the fourth spaced line, the second spaced line}, then the sorting difference between the two sequences can be 3, that is, the first quantity difference corresponding to the marked spaced line can be 3 at this time.

[0090] In the sixth step, all the spaced lines in the first reference spaced line set corresponding to the marked spaced line except the marked spaced line form the second reference spaced line set corresponding to the marked spaced line.

[0091] Among them, the spaced lines in the second reference spaced line set can be called reference spaced lines.

[0092] In the seventh step, according to the lengths of the spaced lines in the second reference spaced line set, the spaced lines in the second reference spaced line set are sorted in ascending order to obtain the second reference spaced line sequence.

[0093] In the eighth step, according to the positions of the spaced lines in the second reference spaced line set in the current bronchoscope image, the spaced lines in the second reference spaced line set are sorted in the order from top to bottom and from left to right to obtain the second temporary spaced line sequence.

[0094] In the ninth step, the sorting difference between the second reference spaced line sequence and the second temporary spaced line sequence is determined as the second quantity difference corresponding to the marked spaced line.

[0095] Step 10: Determine the ray from the centroid of the connected region of the channel to which the above-mentioned marked spacer line belongs towards the centroid of the above-mentioned marked spacer line as the reference ray corresponding to the above-mentioned marked spacer line.

[0096] Step 11: Determine the distance between the centroid of each spacer line in the second reference spacer line set and the reference ray as the reference distance coefficient corresponding to each spacer line in the second reference spacer line set.

[0097] Step 12: Determine the mean value of the reference distance coefficients corresponding to all the spacer lines in the second reference spacer line set as the average reference distance corresponding to the above-mentioned marked spacer line.

[0098] Step 13: Determine the incomplete factor corresponding to the above-mentioned marked spacer line according to the first quantity difference, the second quantity difference and the average reference distance corresponding to the above-mentioned marked spacer line.

[0099] For example, the formula for determining the incomplete factor corresponding to the marked spacer line can be: ; where is the incomplete factor corresponding to the marked spacer line. is the hyperbolic function. is the average reference distance corresponding to the marked spacer line. is the first quantity difference corresponding to the marked spacer line. is the second quantity difference corresponding to the marked spacer line.

[0100] It should be noted that when is smaller, it often indicates that the spacer line and the mean value of its reference spacer line extend outward from the centroid of the channel. When is smaller, it often indicates that the arrangement of the spacer lines is closer to the normal arrangement, so it indicates that the incomplete factor of the spacer lines is smaller.

[0101] Step 14: If the incomplete factor corresponding to the spacer line is greater than the preset incomplete threshold, determine the spacer line as an incomplete spacer line.

[0102] Among them, the preset incomplete threshold can be a threshold set in advance, and it can be 0.7.

[0103] For example, the incomplete factor of the spacer line can also be compared with the preset threshold of 0.7 to obtain the spacer lines with incomplete factors greater than the threshold, and mark them, so as to obtain the incomplete cartilage edge.

[0104] It should be noted that for the marked spacer lines obtained in the incomplete cartilage edge, the marked spacer lines may be caused by the light spot reflected by the light source on the surface of the bronchoscope on the bronchial surface, or may be caused by the morphological change of the cartilage surface tissue due to the lesion of the bronchial wall.

[0105] Step S6: Determine the consistency index corresponding to each incomplete dividing line according to the current bronchoscope image and its previous-frame bronchoscope image.

[0106] As an example, this step may include the following steps: First step: Through the optical flow detection method, screen out the region corresponding to each incomplete dividing line from the previous-frame bronchoscope image as the historical dividing line corresponding to each incomplete dividing line, and screen out the region corresponding to the co-ring edge cluster to which each incomplete dividing line belongs from the previous-frame bronchoscope image as the historical co-ring edge cluster corresponding to each incomplete dividing line.

[0107] For example, corner detection can be used to obtain the key points on the edges in each co-ring edge cluster, and optical flow detection is used to obtain the optical flow vectors of the key points in the current-frame image and the previous-frame image; according to the optical flow vectors and the above operations, the position correspondence in different-frame images is achieved.

[0108] Second step: Determine the consistency index corresponding to each incomplete dividing line according to the DTW distance between each incomplete dividing line and its corresponding historical dividing line, and the area ratio between the historical co-ring edge cluster corresponding to each incomplete dividing line and the co-ring edge cluster to which it belongs.

[0109] It should be noted that since the incomplete dividing lines generated by the lesions will change equally with the movement of the lens and the surrounding cartilage, the incomplete dividing lines will change in the same way as the co-ring edge clusters. Therefore, the consistency of each incomplete dividing line is obtained.

[0110] For example, the formula for determining the consistency index corresponding to the incomplete dividing line can be: ; where is the consistency index corresponding to the incomplete dividing line. is the exponential function with the natural constant as the base. is the DTW distance between the incomplete dividing line and its corresponding historical dividing line. is the absolute value function. is the area ratio between the historical co-ring edge cluster corresponding to the incomplete dividing line and the co-ring edge cluster to which it belongs. is the length ratio between the incomplete dividing line and its corresponding historical dividing line.

[0111] It should be noted that when is smaller, it often means that the incomplete dividing line is more similar to its corresponding historical dividing line. When is smaller, it often indicates that the change of the incomplete dividing line is more similar to the change of the cartilage edge, so its consistency is greater.

[0112] Step S7, if the consistency index corresponding to the incomplete septum line is greater than the preset consistency threshold, then the incomplete septum line is determined as the lesion septum line.

[0113] Among them, the preset consistency threshold can be a threshold set in advance, and it can be 0.5.

[0114] It should be noted that for the consistency of the incomplete septum lines obtained by the above operations, it can be compared with the preset threshold of 0.5 to mark the incomplete septum lines with consistency greater than the threshold; the incomplete septum lines are subjected to region growing, where adjacent incomplete septum lines grow into the same region and the original cartilage edge cannot be damaged, thereby obtaining several lesion regions. Accurately positioning the lesion region in the bronchoscope image can include: marking the lesion region in the bronchoscope image obtained by the above operations in the image to obtain coordinates, and continuously changing the coordinates of the lesion for positioning as the lens moves; after the lesion is located, performing a lesion biopsy is a common operation, aiming to obtain tissue samples for pathological examination to determine the nature of the lesion; the pathological results are usually fed back to the patient by outpatient or inpatient doctors, and the subsequent treatment or observation plan is discussed.

[0115] Reference Figure 2 , based on the same inventive concept as the above method embodiment, the present invention provides a system for accurately positioning the lesion region in a bronchoscope image. The system includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of a method for lesion positioning under a bronchoscope based on reinforcement learning, which may specifically include: An acquisition, extraction, and screening module 201, configured to acquire the current bronchoscope image, perform connected component extraction on the current bronchoscope image, and screen out the channel connected components and cartilage candidate edges from the obtained connected components; A determination and screening module 202, configured to determine the channel connected component closest to the cartilage candidate edge as the representative connected component corresponding to the cartilage candidate edge, and screen out the true cartilage edge from all cartilage candidate edges whose corresponding representative connected components are the same connected component; A construction and determination module 203, configured to form a channel cartilage edge cluster with the true cartilage edges having the same corresponding representative connected component, and determine the co-loop coefficient between every two true cartilage edges in each channel cartilage edge cluster; A screening and recognition module 204, configured to screen out the co-loop edge clusters from each channel cartilage edge cluster according to the co-loop coefficients between different true cartilage edges in each channel cartilage edge cluster, and identify the septum lines from each co-loop edge cluster; A double determination module 205, configured to determine an incomplete factor corresponding to each spacing line according to the distribution of each spacing line, and if the incomplete factor corresponding to the spacing line is greater than a preset incomplete threshold, determine the spacing line as an incomplete spacing line; A consistency index determination module 206, configured to determine a consistency index corresponding to each incomplete spacing line according to the current bronchoscope image and the previous frame of bronchoscope image; A judgment determination module 207, configured to, if the consistency index corresponding to the incomplete spacing line is greater than a preset consistency threshold, determine the incomplete spacing line as a lesion spacing line.

[0116] Figure 3 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Exemplarily, as Figure 3 shown, the computer device 300 includes: a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and running on the processor 302. Wherein, when the processor 302 executes the computer program 303, the computer device can execute any one of the aforementioned bronchoscope-based lesion localization methods based on reinforcement learning.

[0117] Based on the same inventive concept as the above method embodiment, the present invention provides a server, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes any one of the aforementioned bronchoscope-based lesion localization methods based on reinforcement learning.

[0118] Based on the same inventive concept as the above method embodiment, the present invention provides a computer program product, which includes: computer program code, when the computer program code runs on a computer, the computer executes any one of the aforementioned bronchoscope-based lesion localization methods based on reinforcement learning.

[0119] Based on the same inventive concept as the above method embodiment, the present invention provides a computer-readable storage medium, which stores computer program code, and when the computer program code runs on a computer, the computer executes any one of the aforementioned bronchoscope-based lesion localization methods based on reinforcement learning.

[0120] In summary, compared with localizing the lesion area of the bronchoscope image by different gray values, the present invention comprehensively considers multiple features related to the lesion situation of the bronchoscope image, such as the co-ring coefficient, incomplete factor, and consistency index, etc., thereby improving the accuracy of quantifying the lesion spacing line, and further improving the accuracy of localizing the lesion area of the bronchoscope image.

[0121] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A bronchoscopic lesion localization method based on reinforcement learning, characterized in that, Including the following steps: Obtain the current bronchoscope image, perform connected component extraction on the current bronchoscope image, and screen out the channel connected components and cartilage candidate edges from the obtained connected components; Determine the connected component closest to the cartilage candidate edge among the channel connected components as the representative connected component corresponding to the cartilage candidate edge, and screen out the true cartilage edges from all cartilage candidate edges whose corresponding representative connected components are the same connected component; Form the channel cartilage edge clusters with the true cartilage edges having the same corresponding representative connected component, and determine the co-ring coefficient between every two true cartilage edges in each channel cartilage edge cluster; According to the co-ring coefficients between different true cartilage edges in each channel cartilage edge cluster, screen out the co-ring edge clusters from each channel cartilage edge cluster, and identify the spacer lines from each co-ring edge cluster; Determine the corresponding incomplete factor for each spacer line according to the distribution of each spacer line. If the incomplete factor corresponding to the spacer line is greater than the preset incomplete threshold, then determine the spacer line as an incomplete spacer line; Determine the corresponding consistency index for each incomplete spacer line according to the current bronchoscope image and its previous frame bronchoscope image; If the consistency index corresponding to the incomplete spacer line is greater than the preset consistency threshold, then determine the incomplete spacer line as a lesion spacer line.

2. The method for locating lesions under a bronchoscope based on reinforcement learning according to claim 1, wherein The screening out of the channel connected components and cartilage candidate edges from the obtained connected components includes: Perform threshold segmentation on all connected components in the current bronchoscope image, and determine the connected components in the category of connected components with lower gray values among the two categories of connected components obtained by the segmentation as the channel connected components; Screen out reference pixel points from the connected components in the current bronchoscope image; Screen out the reference pixel points among all reference pixel points whose corresponding first pixel points are also reference pixel points as the selected pixel points; Determine the line formed by the continuous selected pixel points as the cartilage candidate edge.

3. The method for bronchoscopic lesion localization based on reinforcement learning according to claim 2, wherein, The screening out of the reference pixel points from the connected components in the current bronchoscope image includes: Determine any one connected component in the current bronchoscope image as the marked connected component, and determine any one pixel point in the marked connected component as the marked pixel point; Screen out the pixel point with the smallest gradient value difference from the marked pixel point within the preset neighborhood corresponding to the marked pixel point as the first pixel point corresponding to the marked pixel point; Screen out the pixel point with the largest corresponding gradient value from the preset neighborhood corresponding to the marked pixel point as the second pixel point corresponding to the marked pixel point; If the first pixel point and the second pixel point corresponding to the marked pixel point are the same pixel point, then determine the marked pixel point as the reference pixel point.

4. The method for locating lesions under a bronchoscope based on reinforcement learning according to claim 1, wherein The screening out of the true cartilage edges from all cartilage candidate edges whose corresponding representative connected components are the same connected component includes: Determine the representative connected component corresponding to any one cartilage candidate edge as the reference connected component; By means of random sampling, different numbers of cartilage candidate edges are randomly selected from all cartilage candidate edges corresponding to the representative connected domain being the reference connected domain, and the cartilage candidate edges selected each time are used to form a cartilage candidate edge set, thereby obtaining multiple cartilage candidate edge sets under the reference connected domain; Perform ipsilateral partitioning on the cartilage candidate edges in each cartilage candidate edge set to obtain respective cartilage candidate edge groups corresponding to each cartilage candidate edge set; Determine the information entropy of the distance between the centroid of all cartilage candidate edges in each cartilage candidate edge group and the centroid of its corresponding representative connected domain as the distance information entropy corresponding to each cartilage candidate edge group; Determine the mean value of the DTW distances between different cartilage candidate edges in each cartilage candidate edge group as the target difference factor corresponding to each cartilage candidate edge group; Determine the target loss value corresponding to each cartilage candidate edge set according to all the distance information entropy and target difference factors corresponding to each cartilage candidate edge set; Screen out the cartilage candidate edge set with the smallest corresponding target loss value from all cartilage candidate edge sets under the reference connected domain as the temporary edge set, and determine each cartilage candidate edge in the temporary edge set as a true cartilage edge.

5. The method for locating lesions under a bronchoscope based on reinforcement learning according to claim 4, wherein, The formula for the target loss value corresponding to the cartilage candidate edge set is: ; where E is the target loss value corresponding to the set of cartilage candidate edges; is a hyperbolic function; is the variance of all distance information entropies corresponding to the set of cartilage candidate edges; is the mean of all target difference factors corresponding to the set of cartilage candidate edges.

6. The method for locating lesions under a bronchoscope based on reinforcement learning according to claim 1, wherein, The determination of the co-ring coefficient between every two true cartilage edges in each channel cartilage edge cluster includes: Determine any true cartilage edge as the marked cartilage edge, and take the centroid of the representative connected domain corresponding to the marked cartilage edge as an endpoint to draw a ray passing through the centroid of the marked cartilage edge, which is denoted as the target ray corresponding to the marked cartilage edge; Determine the intersection of the target ray corresponding to the marked cartilage edge and the representative connected domain as the target line segment corresponding to the marked cartilage edge; From all true cartilage edges intersecting with the target line segment corresponding to the marked cartilage edge, screen out the true cartilage edge farthest from the centroid of the marked cartilage edge as the reference edge corresponding to the marked cartilage edge; Determine the distance between the centroid of the marked cartilage edge and the centroid of its corresponding representative connected domain as the first distance factor corresponding to the marked cartilage edge; Determine the distance between the centroid of the reference edge corresponding to the marked cartilage edge and the centroid of the representative connected domain corresponding to the marked cartilage edge as the second distance factor corresponding to the marked cartilage edge; Determine the ratio of the first distance factor and the second distance factor corresponding to the marked cartilage edge as the reference ratio corresponding to the marked cartilage edge; Determine any channel cartilage edge cluster as the marked cartilage edge cluster, and determine the co-ring coefficient between every two true cartilage edges in the marked cartilage edge cluster according to the reference ratios corresponding to every two true cartilage edges in the marked cartilage edge cluster.

7. A bronchoscopic lesion localization method based on reinforcement learning according to claim 6, characterized in that, The determination of the co-ring coefficient between every two true cartilage edges in the marked cartilage edge cluster according to the reference ratios corresponding to every two true cartilage edges in the marked cartilage edge cluster includes: Determine any two true cartilage edges in the marked cartilage edge cluster as the first marked edge and the second marked edge respectively; Perform elliptical fitting on the union of the first marked edge and the second marked edge, and determine the fitting loss obtained by the elliptical fitting as the non-cocyclic factor between the first marked edge and the second marked edge; Determine the cocyclic coefficient between the first marked edge and the second marked edge according to the non-cocyclic factor between the first marked edge and the second marked edge, and the difference between the reference ratio corresponding to the first marked edge and the reference ratio corresponding to the second marked edge; 8. The method for bronchoscopic lesion localization based on reinforcement learning according to claim 7, wherein The formula for the cocyclic coefficient between the first marked edge and the second marked edge is: ; wherein, is the co-ring coefficient between the first marker edge and the second marker edge; is the exponential function with the natural constant as the base; is the non-co-ring factor between the first marker edge and the second marker edge; is the absolute value function; is the reference ratio corresponding to the first marker edge; is the reference ratio corresponding to the second marker edge.

9. The method for bronchoscopic lesion localization based on reinforcement learning according to claim 1, wherein The screening of the cocyclic edge cluster from each channel cartilage edge cluster according to the cocyclic coefficient between different true cartilage edges in each channel cartilage edge cluster includes: Determine any one channel cartilage edge cluster as the marked cartilage edge cluster, and determine any one true cartilage edge in the marked cartilage edge cluster as the temporary cartilage edge; Screen out the true cartilage edge with the largest cocyclic coefficient between the temporary cartilage edge from the marked cartilage edge cluster as the first edge corresponding to the temporary cartilage edge; Constitute a possible cocyclic cluster with the true cartilage edges having the same first edge in the marked cartilage edge cluster; If the true cartilage edges in the possible cocyclic cluster are on different sides, determine the possible cocyclic cluster as the cocyclic edge cluster.

10. The method for locating a lesion under a bronchoscope based on reinforcement learning according to claim 1, wherein The determination of the incomplete factor corresponding to each spacer according to the distribution of each spacer includes: Constitute a cocyclic edge cluster set with all the cocyclic edge clusters in each channel cartilage edge cluster; Determine any one spacer as the marked spacer, and screen out the spacers closest to the marked spacer from each cocyclic edge cluster in the cocyclic edge cluster set to which the marked spacer belongs to constitute the first reference spacer set corresponding to the marked spacer; Sort the spacers in the first reference spacer set in ascending order according to the lengths of the spacers in the first reference spacer set to obtain the first reference spacer sequence; Sort the spacers in the first reference spacer set in the order from top to bottom and from left to right according to the positions of the spacers in the current bronchoscope image to obtain the first temporary spacer sequence; Determine the sorting difference between the first reference spacer sequence and the first temporary spacer sequence as the first quantity difference corresponding to the marked spacer; Constitute the second reference spacer set corresponding to the marked spacer with all the spacers except the marked spacer in the first reference spacer set corresponding to the marked spacer; Sort the spacers in the second reference spacer set in ascending order according to the lengths of the spacers in the second reference spacer set to obtain the second reference spacer sequence; Sort the lines in the second reference line set according to their positions in the current bronchoscope image in the order from top to bottom and from left to right to obtain a second temporary line sequence; Determine the sorting difference between the second reference line sequence and the second temporary line sequence as the second quantity difference corresponding to the marked line; Determine the ray from the centroid of the channel connected domain to which the marked line belongs to the centroid of the marked line as the reference ray corresponding to the marked line; Determine the distance between the centroid of each line in the second reference line set and the reference ray as the reference distance coefficient corresponding to each line in the second reference line set; Determine the mean value of the reference distance coefficients corresponding to all lines in the second reference line set as the average reference distance corresponding to the marked line; Determine the incomplete factor corresponding to the marked line according to the first quantity difference, the second quantity difference and the average reference distance corresponding to the marked line.

Images