A lung endoscope image navigation recognition method and system

Abstract

The present invention discloses a pulmonary endoscope image navigation and recognition method and system, which relates to the field of pulmonary endoscope technology and solves the problem that the physical signs associated with areas of different brightness within the original image cannot be strongly reflected. The present invention partitions the corresponding images, and then fuses some images with the same feature partitions. Different weight coefficients are assigned in the fusion process to obtain a fused image with the strongest physical signs. The corresponding areas in different images of three same-position areas are integrated to obtain a fused area with the best physical signs after integration, thereby obtaining a corresponding fused image, thereby achieving the best physical sign expression effect and facilitating subsequent lesion analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of pulmonary endoscopes, and in particular to a pulmonary endoscope image navigation and recognition method and system. Background Art

[0002] A pulmonary endoscope is a medical device used to examine the internal conditions of the lungs. To illuminate the dark environment inside the lungs, a cold light source conducted by optical fiber is usually used. The cold light source can prevent damage to lung tissue caused by high temperatures, and its light intensity is adjustable to ensure appropriate lighting under different observation conditions.

[0003] The application with publication number CN113096109A discloses a lung medical image analysis method, device and system. The lung medical image analysis method of the present invention is to segment the lung field and the lung lesion area from the acquired lung medical image, calculate the region of interest ROI containing the lung field and the lung lesion area, and crop the input picture for image recognition in the lung medical image according to the region of interest ROI. The input picture is input into the second-stage detection model, and the feature parameters can be further extracted to obtain the final lesion analysis result output by the preset semantic segmentation model. Through the lung medical image analysis method, device and system of the invention, multiple lung lesions can be identified and classified at the same time. The device of the invention has high efficiency and high accuracy in analyzing lung medical images and has good application prospects.

[0004] When conducting navigation analysis on lung endoscopy images, the acquired images are optimized after denoising, smoothing, and median filtering. However, the original optimization method still cannot strongly reflect the signs associated with areas of different brightness within the image, which creates difficulties in the subsequent confirmation of the lesion area. Summary of the Invention

[0005] In response to the deficiencies of the prior art, the present invention provides a lung endoscope image navigation and recognition method and system, which solves the problem that the signs associated with areas of different brightness within the original image cannot be strongly reflected.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a pulmonary endoscope image navigation and recognition method, whose path planning includes:

[0007] The process also includes the following steps: using specialized medical image processing software to construct a 3D lung model from the CT image to confirm the 3D lung model; then, based on the spatial characteristics of the actual cavity inside the 3D lung model, calibrating the feature points and determining the feature coefficients of the corresponding feature points; and then, based on the different feature coefficients associated with different navigation paths, selecting the optimal planning path; the specific method for selecting the optimal planning path is as follows:

[0008] Based on the acquired lung CT images, a corresponding lung 3D model is generated, and the relevant feature points are calibrated based on the actual cavity within the lung 3D model: the actual cavity is divided into several equal-dividing planes, and the distance characteristics between each equal-dividing plane are consistent. The edge contours around the equal-dividing planes are identified to see whether they intersect with the calibrated bronchial bifurcation points or vascular branches. If they do, the points on the corresponding edge contours are calibrated as feature points, and a weight coefficient of 0.8 is assigned to these feature points. If they do not intersect, no calibration is performed, and the weight coefficient assigned to the uncalibrated points is 1.

[0009] Based on the different weight coefficients assigned to different points on the edge contour of the corresponding averaging surface, the characteristic coefficient of the corresponding center point of the averaging surface is determined. The characteristic coefficient is the average of the weight coefficients associated with several points on the edge contour of the averaging surface, and the center point of the averaging surface is marked as the characteristic point;

[0010] Based on the current location of the endoscope and the endoscope point to be reached, the endoscope point is calibrated in advance by relevant personnel. Multiple navigation paths are generated with the current location as the starting point and the endoscope point as the end point.

[0011] Based on the characteristic points passed by different navigation paths, the characteristic coefficients of several characteristic points are summed up to determine the total characteristic coefficient. The group of navigation paths with the largest total characteristic coefficient is selected as the planning path, and the endoscope is controlled to reach the endoscope point.

[0012] The distance feature is the straight-line distance between the center points inside the corresponding averaging surface, and its navigation path is generated by the center of the system, that is, the center line of the corresponding cavity at the middle position;

[0013] The image acquisition, processing and recognition process includes the following steps:

[0014] Step 1: When the lung endoscope reaches the endoscope point, image acquisition is performed. Different aperture coefficients are used to confirm the standard image, strong exposure image, and weak exposure image associated with the same endoscope point, and an image set belonging to the corresponding endoscope point is generated. The specific method is as follows:

[0015] S11. Based on the acquisition instruction, perform image acquisition at the designated endoscopic point, preferably using an aperture factor of f / 8 for image acquisition, and calibrate the acquired image as a standard image of the endoscopic point. Then, perform image acquisition at an aperture factor of f / 2.8, and calibrate the acquired image as a weakly exposed image of the endoscopic point. Then, perform image acquisition at an aperture factor of f / 16, and calibrate the acquired image as a strongly exposed image of the endoscopic point.

[0016] S12, integrating the three sets of images associated with the same endoscopic point to determine the image set belonging to this endoscopic point;

[0017] Step 2: Based on the image set identified at the corresponding endoscopic point, a standard image is selected from the image set and grayscale value analysis is performed on the standard image. Dark, intermediate, and bright areas are selected from the standard image. Based on the region boundaries, associated regions are selected from the strong-exposure image and the weak-exposure image. Different weight coefficients are assigned to different associated regions belonging to different images. The image set is then fused to generate a fused image of the endoscopic point. The specific sub-steps are as follows:

[0018] S21, select a standard image from the image set, and confirm the grayscale value HD associated with different points in the standard image i , where i represents different points, and the confirmed gray value HD i Compare with the preset value Y1 and the preset value Y2, and Y1<Y2, if HD i <Y1, mark this point as a dark area point, if Y1≤HD i ≤Y2, then mark this point as the middle point. If HD i > Y2, then this point is marked as a bright area point. Based on the different types of points identified, the standard image is divided into a dark area, an intermediate area, and a bright area. The dark area is composed of dark area points, the intermediate area is composed of intermediate points, and the bright area is composed of bright area points.

[0019] S22. Based on the dark areas, intermediate areas, and bright areas identified in the standard image, and based on the consistency of the contour edges of the corresponding areas, associated areas with identical positional features are selected from the strongly exposed image and the weakly exposed image. The contour edge features of the associated areas are consistent with those of the corresponding areas. The corresponding dark areas and the identified associated areas are calibrated as a dark area set, and the intermediate area set and the bright area set are simultaneously identified.

[0020] S23. For the dark area set: the dark areas of the standard image are calibrated as standard dark areas, the dark areas of the strongly exposed image are calibrated as strongly exposed dark areas, and the dark areas of the weakly exposed image are calibrated as weakly exposed dark areas. A weight coefficient of 0.3 is assigned to the standard dark areas, a weight coefficient of 0.1 is assigned to the strongly exposed dark areas, and a weight coefficient of 0.6 is assigned to the weakly exposed dark areas. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the dark areas, pixel adjustment is performed before fusion to determine the dark area fused image.

[0021] For the intermediate area set: the intermediate area of ​​the standard image is calibrated as the standard intermediate area, the intermediate area of ​​the strongly exposed image is calibrated as the strongly exposed intermediate area, and the intermediate area of ​​the weakly exposed image is calibrated as the weakly exposed intermediate area. The weight coefficient assigned to the standard intermediate area is 0.6, and the weight coefficients assigned to the strongly exposed intermediate area and the weakly exposed intermediate area are both 0.2. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding intermediate areas, they are pixel-adjusted and then fused to determine the intermediate area fused image;

[0022] For the bright area set: the bright areas of the standard image are calibrated as standard bright areas, the bright areas of the strongly exposed image are calibrated as strongly exposed bright areas, and the bright areas of the weakly exposed image are calibrated as weakly exposed bright areas. The weight coefficient assigned to the standard bright area is 0.3, the weight coefficient assigned to the strongly exposed bright area is 0.6, and the weight coefficient assigned to the weakly exposed bright area is 0.1. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding bright areas, pixel adjustment is performed before fusion to confirm the bright area fused image;

[0023] S24, combining the determined dark area fusion image, intermediate area fusion image, and bright area fusion image to confirm a fusion image;

[0024] Step 3: Based on the determined fused image and the grayscale value characteristics of the points within the fused image, determine the grayscale value differences between adjacent points, and based on the determination results, divide the fused image into multiple different grayscale areas. The specific method is as follows:

[0025] S31, based on the different grayscale values ​​associated with different points in the fused image, identify the absolute value ZZ of the grayscale value difference between adjacent points, compare ZZ with a preset value Y3, and if ZZ ≤ Y3, calibrate the corresponding two adjacent points as points of the same type; if ZZ > Y3, calibrate the corresponding two adjacent points as points of different types;

[0026] S32. Based on the confirmed plurality of similar points, confirm similar grayscale areas, where adjacent points between boundaries of different grayscale areas all belong to different types of points;

[0027] Step 4: Based on the different grayscale areas identified in the fused image, determine the center point of the corresponding grayscale area, then generate data based on the preset lesion model, and perform real-time verification on the actual real-time generated data to determine whether the corresponding grayscale area is the lesion area. The specific method is as follows:

[0028] S41, determining the center point of the corresponding grayscale area based on the overall edge contour of the corresponding grayscale area;

[0029] S42. Based on the determined center point, the center point is used as the lesion point in the lesion model. The lesion model performs lesion processing on the surrounding area according to the calibrated lesion point. The grayscale values ​​of the points around the lesion point gradually change as the lesion processing progresses. The grayscale value of the real-time change is subtracted from the actual grayscale value of the corresponding point in the grayscale area, and the real-time grayscale value of the corresponding point is calibrated as Hz. i , where i represents different points, and the grayscale value corresponding to the point in the grayscale area is calibrated as HQ i , using C i =|Hz i -HQ i |Confirm the corresponding difference C i ;

[0030] Based on the real-time lesion processing process, the gray value of each point will change synchronously, resulting in the confirmed difference C i Synchronously change, and the several groups of difference values ​​C generated by each change process i Perform variance processing and determine the variance value F k , where k represents different change processes. If there is F k ≤Y4, then the grayscale area is marked as the lesion area, where Y4 is the preset value. If there is no F k If the change process of ≤Y4, no calibration is performed;

[0031] S43, perform steps S41-S42 in sequence to calibrate the lesion area in other grayscale areas.

[0032] Preferably, a pulmonary endoscope image navigation and recognition system comprises:

[0033] The image set generation end collects images when the lung endoscope reaches the endoscope point. It uses different aperture coefficients to confirm the standard image, strong exposure image, and weak exposure image associated with the same endoscope point, and generates an image set belonging to the corresponding endoscope point.

[0034] The fused image generation end selects a standard image from the image set based on the image set confirmed by the corresponding endoscope point, performs grayscale value analysis on the standard image, selects dark areas, intermediate areas, and bright areas from the standard image, and selects related areas from the strong-exposure image and the weak-exposure image based on the region boundaries. Different weight coefficients are assigned to different related areas belonging to different images, and the image set is fused to generate a fused image of the endoscope point.

[0035] A grayscale region division end determines grayscale value differences between adjacent points based on the determined fused image and grayscale value features of points within the fused image, and divides the fused image into a plurality of different grayscale regions based on the determination result;

[0036] The lesion area confirmation end determines the center point of the corresponding grayscale area based on the different grayscale areas confirmed in this fused image, and then generates data based on the preset lesion model. The actual real-time generated data is checked in real time to determine whether the corresponding grayscale area is the lesion area.

[0037] The present invention provides a method and system for pulmonary endoscopy image navigation and recognition. Compared with the existing technology, it has the following advantages:

[0038] The present invention uses specialized medical image processing software to construct a three-dimensional lung model from CT images to confirm the three-dimensional lung model. Based on the spatial characteristics of the actual cavity inside the three-dimensional lung model, the present invention calibrates the feature points and determines the feature coefficients of the corresponding feature points. Based on the different feature coefficients associated with different navigation paths, the present invention selects the optimal planning path, minimizing the cost and risk. This method can fully avoid damage to the lung cavity environment and effectively reduce the corresponding risk.

[0039] Image acquisition is performed on the same endoscopic point. Three groups of related images with different aperture coefficients are confirmed for the same group of endoscopic points. Based on the grayscale value performance of the related images, the corresponding images are partitioned. Then, some images with the same characteristic partitions are fused. Different weight coefficients are assigned in the fusion process to obtain a fused image with the strongest physical signs. The areas within the corresponding three different images of the same location area are integrated to obtain the fused area with the best physical signs after integration, thereby obtaining the corresponding fused image, thereby achieving the best physical sign performance effect and facilitating subsequent lesion analysis;

[0040] Based on the confirmed fusion image and the performance of the grayscale values ​​of adjacent internal points, the fusion image is divided into multiple groups of different grayscale areas, and then numerical analysis is performed on each group of different grayscale areas. Lesion processing data is generated according to the preset model, and the data generated in real time at the same point is difference-confirmed. The corresponding variance is locked based on the confirmed difference, and the lesion area is locked based on the recognition result of the variance. In this way, the lesion area can be determined quickly and effectively, and the determination accuracy can be ensured at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the process of the present invention;

[0042] Figure 2 It is a schematic diagram of the principle framework of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] First embodiment

[0045] See also Figure 1 , the present application provides a lung endoscope image navigation and recognition method, comprising the following steps:

[0046] CT images are analyzed using specialized medical image processing software. The bronchial branches at all levels, the location of the lesion, and important structures such as surrounding blood vessels are identified. The optimal insertion path for the bronchoscope is planned based on the location of the lesion, with the goal of selecting straight and unobstructed bronchial branches while avoiding important structures such as blood vessels to reduce operational risks. For example, if the lesion is located in the dorsal segment of the right lower lobe, the planned path would need to enter the right main bronchus from the main bronchus, then pass through the bronchus intermedius and the lower lobe bronchus to reach the target dorsal segment bronchus.

[0047] Under local or general anesthesia, a bronchoscope is inserted through the patient's mouth or nose. During the insertion process, real-time images are obtained through the camera at the front end of the bronchoscope.

[0048] The position and orientation of the bronchoscope within the airway are determined using sensors on the bronchoscope (e.g., electromagnetic or optical sensors). These sensors are then registered with preoperative CT images to achieve real-time positioning. For example, in an electromagnetic navigation bronchoscope system, a device that generates an electromagnetic field is placed around the patient. Sensors within the bronchoscope generate signals in the magnetic field, and computer processing aligns the bronchoscope's position with the preoperatively planned path in three dimensions.

[0049] As the bronchoscope is advanced, the real-time endobronchial images are compared with the preoperative CT images. The morphology and branching of the bronchi are observed, as well as any abnormal lesions. If the current image deviates from the planned path, such as encountering unexpected stenosis or branches, the direction of the bronchoscope is adjusted based on the image information and the navigation system's prompts.

[0050] When approaching the lesion area, the magnification function of the bronchoscope can be used to more clearly observe the details of the lesion, such as the color, texture, and vascular distribution of the lesion, and at the same time confirm whether the target position planned before surgery has been reached.

[0051] The specific methods for navigation path planning related to this part are as follows:

[0052] Based on the collected lung CT images, a corresponding lung 3D model is generated, and the relevant feature points are calibrated based on the actual cavity in the lung 3D model: the actual cavity is divided into several equal-dividing planes, and the distance features between each equal-dividing plane are consistent (the distance feature is the straight-line distance between the center points inside the corresponding equal-dividing plane). It is identified whether the edge contours around the equal-dividing plane intersect with the calibrated bronchial bifurcation points or vascular branches. If they intersect, the points on the corresponding edge contours are calibrated as feature points, and a weight coefficient of 0.8 is assigned to these feature points. If they do not intersect, no calibration is performed, and the weight coefficient assigned to the points that have not been calibrated is 1 (these points are all points on the edge contours of the equal-dividing planes);

[0053] Based on the different weight coefficients assigned to different points on the edge contour of the corresponding averaging surface, the characteristic coefficient of the corresponding center point of the averaging surface is determined. The characteristic coefficient is the average of the weight coefficients associated with several points on the edge contour of the averaging surface, and the center point of the averaging surface is marked as the characteristic point;

[0054] Based on the current location of the endoscope and the endoscope point to be reached (calibrated in advance by relevant personnel), multiple navigation paths are generated with the current location as the starting point and the endoscope point as the end point (the navigation path is generated by the center of the system, that is, the center line of the corresponding cavity in the middle position);

[0055] Based on the characteristic points passed by different navigation paths, the characteristic coefficients of several characteristic points are summed up to determine the total characteristic coefficient. The group of navigation paths with the largest total characteristic coefficient is selected as the planning path, and the endoscope is controlled to reach the endoscope point.

[0056] Specifically, this part of the content can effectively ensure that the cost and risk of the corresponding path when moving are minimized, because it can fully avoid damage to the lung cavity environment and effectively reduce the corresponding risk.

[0057] As a further embodiment of this application

[0058] The specific steps when the lung endoscope reaches the endoscopy point are as follows:

[0059] Step 1: When the lung endoscope reaches the endoscopic point, image acquisition is performed. Different aperture coefficients are used to confirm the standard image, strong exposure image, and weak exposure image associated with the same endoscopic point to generate an image set belonging to the corresponding endoscopic point. Specifically, when performing image acquisition confirmation, the corresponding operator performs endoscopic point image acquisition. For the same group of endoscopic points, three groups of images associated with different aperture coefficients are confirmed. The specific method of confirmation is as follows:

[0060] S11. Based on the acquisition instruction, perform image acquisition at the designated endoscopic point, preferably using an aperture factor of f / 8 for image acquisition, and calibrate the acquired image as a standard image of the endoscopic point. Then, perform image acquisition at an aperture factor of f / 2.8, and calibrate the acquired image as a weakly exposed image of the endoscopic point. Then, perform image acquisition at an aperture factor of f / 16, and calibrate the acquired image as a strongly exposed image of the endoscopic point.

[0061] S12: Integrate the three sets of images associated with the same endoscopic point to determine the image set belonging to the endoscopic point. Specifically, there are three sets of images associated with the same endoscopic point. The reason for collecting multiple sets of images for the same endoscopic point is to achieve better image performance. It is necessary to generate the best overall image for different exposure conditions.

[0062] Step 2: Based on the image set confirmed by the corresponding endoscopic point, a standard image is selected from the image set, and grayscale value analysis is performed on the standard image. Dark areas, intermediate areas, and bright areas are selected from the standard image. Based on the area boundaries, associated areas are selected from the strong exposure image and the weak exposure image. Different weight coefficients are assigned to different associated areas belonging to different images, and the image set is fused to generate a fused image of the endoscopic point. Specifically, in order to achieve the best performance of the image taken by the corresponding endoscopic point, three different groups of images are associated and fused, and different weight coefficients are assigned to different areas, so that each area of ​​different brightness in the corresponding standard image can achieve a better performance, and the fused image can achieve the best performance signs, which is convenient for subsequent specific confirmation of the lesion area. The specific sub-steps of generating the fused image are as follows:

[0063] S21, select a standard image from the image set, and confirm the grayscale value HD associated with different points in the standard image i , where i represents different points, and the confirmed gray value HD i Compare with the preset values ​​Y1 and Y2, where the specific values ​​of Y1 and Y2 are set in advance, and Y1<Y2, if HD i <Y1, mark this point as a dark area point, if Y1≤HD i ≤Y2, then mark this point as the middle point. If HD i > Y2, then this point is marked as a bright area point. Based on the different types of points identified, the standard image is divided into a dark area, an intermediate area, and a bright area. The dark area is composed of dark area points, the intermediate area is composed of intermediate points, and the bright area is composed of bright area points.

[0064] S22. Based on the dark areas, intermediate areas, and bright areas identified in the standard image, and based on the consistency of the contour edges of the corresponding areas, associated areas with the same positional features are selected from the strongly exposed image and the weakly exposed image, where the contour edge features of the associated areas are consistent with those of the corresponding areas (that is, if the images are considered as corresponding planes, all points within the planes have coordinate features. Then, based on these coordinate features, the same areas with the same positional features can be identified, and the associated areas are also simultaneously divided into dark areas, intermediate areas, or bright areas). The corresponding dark areas and the identified associated areas are marked as a dark area set, and the intermediate area set and the bright area set are simultaneously confirmed.

[0065] S23. For the dark area set: the dark areas belonging to the standard image are calibrated as standard dark areas, the dark areas belonging to the strongly exposed image are calibrated as strongly exposed dark areas, and the dark areas belonging to the weakly exposed image are calibrated as weakly exposed dark areas. A weight coefficient of 0.3 is assigned to the standard dark areas, a weight coefficient of 0.1 is assigned to the strongly exposed dark areas, and a weight coefficient of 0.6 is assigned to the weakly exposed dark areas. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding dark areas, pixel adjustment is performed before fusion to confirm the dark area fused image (only dark areas with the same positional features are fused. The pixel value adjustment is to assign corresponding weight coefficients on the basis of the original pixel values, change the pixel values, and thus adjust the pixel values ​​in an associated manner, that is, adjusted pixel value = original pixel value × weight coefficient);

[0066] For the intermediate area set: the intermediate area of ​​the standard image is calibrated as the standard intermediate area, the intermediate area of ​​the strongly exposed image is calibrated as the strongly exposed intermediate area, and the intermediate area of ​​the weakly exposed image is calibrated as the weakly exposed intermediate area. The weight coefficient assigned to the standard intermediate area is 0.6, and the weight coefficients assigned to the strongly exposed intermediate area and the weakly exposed intermediate area are both 0.2. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding intermediate areas, they are pixel-adjusted and then fused to determine the intermediate area fused image;

[0067] For the bright area set: the bright areas of the standard image are calibrated as standard bright areas, the bright areas of the strongly exposed image are calibrated as strongly exposed bright areas, and the bright areas of the weakly exposed image are calibrated as weakly exposed bright areas. The weight coefficient assigned to the standard bright area is 0.3, the weight coefficient assigned to the strongly exposed bright area is 0.6, and the weight coefficient assigned to the weakly exposed bright area is 0.1. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding bright areas, pixel adjustment is performed before fusion to confirm the bright area fused image;

[0068] S24 , combining the determined dark area fusion image, middle area fusion image, and bright area fusion image (when combining, they can be recombined in the reverse manner of partitioning the standard image) to confirm the fusion image.

[0069] Specifically, the fusion image confirmed here can more clearly and accurately express the image characteristics in terms of the image's manifestation signs. For the dark area, its weakly exposed associated image occupies a dominant position and can better display the dark area characteristics. For the bright area, its strongly exposed associated image occupies a dominant position and can also better display the bright area characteristics. After the weight coefficients are assigned in sequence, the corresponding areas in the three different images of the same position area can be integrated to obtain the fusion area with the best manifestation signs after integration, thereby obtaining the corresponding fusion image, so as to achieve the best sign manifestation effect and facilitate subsequent lesion analysis.

[0070] Step 3: Based on the determined fused image and the grayscale value characteristics of the points within the fused image, determine the grayscale value differences between adjacent points, and based on the determination results, divide the fused image into multiple different grayscale areas. Specifically, the division here is different from the above-mentioned area confirmation method. The threshold used in the above-mentioned area confirmation is relatively large, while the threshold set here is relatively small. Here, the main purpose is to perform boundary division and confirm different grayscale areas. The specific sub-steps of the division are:

[0071] S31. Based on the different grayscale values ​​associated with different points in the fused image, identify the absolute value ZZ of the grayscale value difference between adjacent points, and compare ZZ with a preset value Y3, where the specific value of Y3 is determined by the operator based on experience. If ZZ ≤ Y3, the corresponding two adjacent points are calibrated as points of the same type; if ZZ > Y3, the corresponding two adjacent points are calibrated as points of different types.

[0072] S32: Based on the identified similar points, similar grayscale regions are identified. Adjacent points between the boundaries of different grayscale regions belong to different types of points. Specifically, the grayscale regions are divided to facilitate the determination, calibration, and display of the lesion region within the corresponding grayscale region.

[0073] Step 4: Based on the different grayscale areas identified in the fused image, determine the center point of the corresponding grayscale area, then generate data based on the preset lesion model, and perform real-time verification on the actual real-time generated data to determine whether the corresponding grayscale area is the lesion area. The specific method for determining the lesion area is as follows:

[0074] S41. Based on the overall edge contour of the corresponding grayscale region, determine the center point of the corresponding grayscale region (the center point can be determined using a two-dimensional coordinate system. The two-dimensional coordinates of the points associated with the overall edge contour are determined in the two-dimensional coordinate system. Then, the determined sets of two-dimensional coordinates are averaged to determine the average coordinates. The point where the average coordinates are located is the center point of the corresponding grayscale region);

[0075] S42. Based on the determined center point, the center point is used as the lesion point in the lesion model. The lesion model performs lesion processing on the surrounding area according to the calibrated lesion point. The grayscale values ​​of the points around the lesion point gradually change as the lesion processing progresses. The grayscale value of the real-time change is subtracted from the actual grayscale value of the corresponding point in the grayscale area, and the real-time grayscale value of the corresponding point is calibrated as Hz. i , where i represents different points, and the grayscale value corresponding to the point in the grayscale area is calibrated as HQ i , using C i =|Hz i -HQ i |Confirm the corresponding difference C i ;

[0076] Based on the real-time lesion processing process, the gray value of each point will change synchronously, resulting in the confirmed difference C i Synchronously change, and the several groups of difference values ​​C generated by each change process i Perform variance processing and determine the variance value F k , where k represents different change processes. If there is F k ≤Y4, then the grayscale area is marked as the lesion area, where Y4 is a preset value, and its specific value is determined by the operator based on experience. If there is no F k If the change process of ≤Y4, no calibration is performed;

[0077] S43, performing steps S41-S42 in sequence to calibrate the lesion area on other grayscale areas;

[0078] Specifically, the grayscale value of the grayscale area has been confirmed, that is, it does not change. There is a corresponding lesion center point in the lesion area. Based on this center point, the grayscale value can be changed. According to the corresponding pathological process and the set program, the grayscale value of the points around the corresponding center point is changed. Then, in the gradual change process, the changed grayscale value will gradually approach the grayscale value associated with the grayscale area, which means that this type of grayscale area basically belongs to the lesion area, and then the relevant calibration of the lesion area can be performed directly.

[0079] Specifically, the code content for running the lesion model (there are spaces inside it originally, but the corresponding spaces have been deleted to avoid formatting problems) is as follows:

[0080] import numpy asnp

[0081] importmatplotlib.pyplotasplt

[0082] #Image size

[0083] image_height=100

[0084] image_width=100

[0085] #Create a simple grayscale image (the initial grayscale value is between 0-255)

[0086] image=np.random.randint(0,256,(image_height,image_width))

[0087] #Define the coordinates of the lesion point (here it is assumed to be near the center of the image)

[0088] lesion_center_x=50

[0089] lesion_center_y=50

[0090] #Grayscale value of the lesion

[0091] lesion_center_gray_value=image[lesion_center_x,lesion_center_y]

[0092] #Simulated lesion treatment radius (here simply set to 10 pixels)

[0093] radius=10

[0094] #Perform lesion processing (simulate the grayscale value close to the lesion point)

[0095] forxinrange(max(0,lesion_center_x-radius),min(image_height,lesion_center_x+radius+1)):

[0096] foryinrange(max(0,lesion_center_y-radius),min(image_width,lesion_center_y+radius+1)):

[0097] distance=np.sqrt((x-lesion_center_x)**2+(y-lesion_center_y)**2)

[0098] ifdistance<=radius:

[0099] #Here we simply let the grayscale values ​​of the surrounding points approach the grayscale value of the lesion point linearly

[0100] #Adjust the weight according to the distance. The closer the distance, the greater the weight.

[0101] weight = 1-(distance / radius)

[0102] image[x,y]=int((1-weight)*image[x,y]+weight*lesion_center_gray_value)

[0103] # Display the processed image

[0104] plt.imshow(image,cmap='gray')

[0105] plt.title("ImagewithLesionProcessing")

[0106] plt.show().

[0107] Second embodiment

[0108] Regarding a corresponding pulmonary endoscope image navigation and recognition method, when the pulmonary endoscope image navigation and recognition method is executed, there is an image navigation and recognition system adapted thereto;

[0109] A pulmonary endoscope image navigation and recognition system, comprising:

[0110] The image set generation end collects images when the lung endoscope reaches the endoscope point. It uses different aperture coefficients to confirm the standard image, strong exposure image, and weak exposure image associated with the same endoscope point, and generates an image set belonging to the corresponding endoscope point.

[0111] The fused image generation end selects a standard image from the image set based on the image set confirmed by the corresponding endoscope point, performs grayscale value analysis on the standard image, selects dark areas, intermediate areas, and bright areas from the standard image, and selects related areas from the strong-exposure image and the weak-exposure image based on the region boundaries. Different weight coefficients are assigned to different related areas belonging to different images, and the image set is fused to generate a fused image of the endoscope point.

[0112] A grayscale region division end determines grayscale value differences between adjacent points based on the determined fused image and grayscale value features of points within the fused image, and divides the fused image into a plurality of different grayscale regions based on the determination result;

[0113] The lesion area confirmation end determines the center point of the corresponding grayscale area based on the different grayscale areas confirmed in this fused image, and then generates data based on the preset lesion model. The actual real-time generated data is checked in real time to determine whether the corresponding grayscale area is the lesion area.

[0114] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0115] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A lung endoscope image navigation and recognition method, characterized in that: The following steps are involved: Step 1: When the lung endoscope reaches the endoscope point, image acquisition is performed. Different aperture coefficients are used to confirm the standard image, strong exposure image, and weak exposure image associated with the same endoscope point to generate an image set belonging to the corresponding endoscope point. Step 2: Based on the image set identified at the corresponding endoscopic point, a standard image is selected from the image set and grayscale value analysis is performed on the standard image. Dark, intermediate, and bright areas are selected from the standard image. Based on the region boundaries, associated regions are selected from the strongly exposed and weakly exposed images. Different associated regions belonging to different images are assigned different weight coefficients, and the image set is fused to generate a fused image of the endoscopic point. Step 3: Based on the determined fused image and the grayscale value characteristics of the points within the fused image, determine the grayscale value differences between adjacent points, and based on the determination result, divide the fused image into a plurality of different grayscale regions; Step 4: Based on the different grayscale areas confirmed in this fused image, determine the center point of the corresponding grayscale area, then generate data based on the preset lesion model, and verify the real-time generated data in real time to determine whether the corresponding grayscale area is the lesion area.

2. A lung endoscope image navigation and recognition method according to claim 1, characterized in that: It also includes the preliminary steps: using specialized medical image processing software to construct a three-dimensional lung model from CT images to confirm the three-dimensional lung model, and then calibrating the feature points based on the spatial characteristics of the actual cavity inside the three-dimensional lung model, and determining the feature coefficients of the corresponding feature points, and then selecting the best planning path based on the different feature coefficients associated with different navigation paths.

3. A lung endoscope image navigation and recognition method according to claim 2, characterized in that: In the preceding steps, the specific method of selecting the best planning path is: Based on the acquired lung CT images, a corresponding lung 3D model is generated, and the relevant feature points are calibrated based on the actual cavity within the lung 3D model: the actual cavity is divided into several equal-dividing planes, and the distance characteristics between each equal-dividing plane are consistent. The edge contours around the equal-dividing planes are identified to see whether they intersect with the calibrated bronchial bifurcation points or vascular branches. If they do, the points on the corresponding edge contours are calibrated as feature points, and a weight coefficient of 0.8 is assigned to these feature points. If they do not intersect, no calibration is performed, and the weight coefficient assigned to the uncalibrated points is 1. Based on the different weight coefficients assigned to different points on the edge contour of the corresponding averaging surface, the characteristic coefficient of the corresponding center point of the averaging surface is determined. The characteristic coefficient is the average of the weight coefficients associated with several points on the edge contour of the averaging surface, and the center point of the averaging surface is marked as the characteristic point; Based on the current location of the endoscope and the endoscope point to be reached, the endoscope point is calibrated in advance by relevant personnel. Multiple navigation paths are generated with the current location as the starting point and the endoscope point as the end point. According to the feature points passed by different navigation paths, the feature coefficients of several feature points are summed up to determine the total feature coefficient. The set of navigation paths with the largest total feature coefficient is selected as the planning path, and the endoscope is controlled to reach the endoscope point.

4. A lung endoscope image navigation and recognition method according to claim 3, characterized in that: The distance feature is the straight-line distance between the center points inside the corresponding equidivision surface, and its navigation path is generated by the center of the system, that is, the center line of the corresponding cavity at the middle position.

5. A lung endoscope image navigation and recognition method according to claim 1, characterized in that: In step 1, the specific method of determining the image set corresponding to the endoscopy point is: S11. Based on the acquisition instruction, perform image acquisition at the designated endoscopic point, preferably using an aperture factor of f / 8 for image acquisition, and calibrate the acquired image as a standard image of the endoscopic point. Then, perform image acquisition at an aperture factor of f / 2.8, and calibrate the acquired image as a weakly exposed image of the endoscopic point. Then, perform image acquisition at an aperture factor of f / 16, and calibrate the acquired image as a strongly exposed image of the endoscopic point. S12: Integrate the three groups of images associated with the same endoscopy point to confirm the image set belonging to this endoscopy point.

6. A lung endoscope image navigation and recognition method according to claim 1, characterized in that: In step 2, the specific sub-steps of generating the fused image are: S21, select a standard image from the image set, and confirm the gray value HD associated with different points in the standard image i , where i represents different points, and the confirmed gray value HD i Compare with the preset value Y1 and the preset value Y2, and Y1<Y2, if HD i <Y1, mark this point as a dark area point, if Y1≤HD i ≤Y2, then mark this point as the middle point. If HD i > Y2, then this point is marked as a bright area point. Based on the different types of points identified, the standard image is divided into a dark area, an intermediate area, and a bright area. The dark area is composed of dark area points, the intermediate area is composed of intermediate points, and the bright area is composed of bright area points. S22. Based on the dark areas, intermediate areas, and bright areas identified in the standard image, and based on the consistency of the contour edges of the corresponding areas, associated areas with identical positional features are selected from the strongly exposed image and the weakly exposed image. The contour edge features of the associated areas are consistent with those of the corresponding areas. The corresponding dark areas and the identified associated areas are calibrated as a dark area set, and the intermediate area set and the bright area set are simultaneously identified. S23. For the dark area set: the dark areas of the standard image are calibrated as standard dark areas, the dark areas of the strongly exposed image are calibrated as strongly exposed dark areas, and the dark areas of the weakly exposed image are calibrated as weakly exposed dark areas. A weight coefficient of 0.3 is assigned to the standard dark areas, a weight coefficient of 0.1 is assigned to the strongly exposed dark areas, and a weight coefficient of 0.6 is assigned to the weakly exposed dark areas. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the dark areas, pixel adjustment is performed before fusion to determine the dark area fused image. For the intermediate area set: the intermediate area of ​​the standard image is calibrated as the standard intermediate area, the intermediate area of ​​the strongly exposed image is calibrated as the strongly exposed intermediate area, and the intermediate area of ​​the weakly exposed image is calibrated as the weakly exposed intermediate area. The weight coefficient assigned to the standard intermediate area is 0.6, and the weight coefficients assigned to the strongly exposed intermediate area and the weakly exposed intermediate area are both 0.

2. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding intermediate areas, they are pixel-adjusted and then fused to determine the intermediate area fused image; For the bright area set: the bright areas of the standard image are calibrated as standard bright areas, the bright areas of the strongly exposed image are calibrated as strongly exposed bright areas, and the bright areas of the weakly exposed image are calibrated as weakly exposed bright areas. The weight coefficient assigned to the standard bright area is 0.3, the weight coefficient assigned to the strongly exposed bright area is 0.6, and the weight coefficient assigned to the weakly exposed bright area is 0.

1. Based on the weight coefficients and the original pixel values ​​associated with the corresponding points in the corresponding bright areas, pixel adjustment is performed before fusion to confirm the bright area fused image; S24: Combine the determined dark area fusion image, middle area fusion image, and bright area fusion image to confirm a fusion image.

7. A lung endoscope image navigation and recognition method according to claim 1, characterized in that: In step 3, the specific method of dividing the fused image into multiple different grayscale areas is: S31, based on the different grayscale values ​​associated with different points in the fused image, identify the absolute value ZZ of the grayscale value difference between adjacent points, compare ZZ with a preset value Y3, and if ZZ ≤ Y3, calibrate the corresponding two adjacent points as points of the same type; if ZZ > Y3, calibrate the corresponding two adjacent points as points of different types; S32. Based on the confirmed plurality of similar points, grayscale regions of the same type are confirmed, and adjacent points between boundaries of different grayscale regions all belong to different types of points.

8. The lung endoscope image navigation and recognition method according to claim 1, characterized in that: In step 4, the specific method of locking the lesion area is: S41, determining the center point of the corresponding grayscale area based on the overall edge contour of the corresponding grayscale area; S42. Based on the determined center point, the center point is used as the lesion point in the lesion model. The lesion model performs lesion processing on the surrounding area according to the calibrated lesion point. The grayscale values ​​of the points around the lesion point gradually change as the lesion processing progresses. The grayscale value of the real-time change is subtracted from the actual grayscale value of the corresponding point in the grayscale area, and the real-time grayscale value of the corresponding point is calibrated as Hz. i , where i represents different points, and the grayscale value corresponding to the point in the grayscale area is calibrated as HQ i , using C i =|Hz i -HQ i |Confirm the corresponding difference C i ; Based on the real-time lesion processing process, the gray value of each point will change synchronously, resulting in the confirmed difference C i Synchronously change, and the several groups of difference values ​​C generated by each change process i Perform variance processing and determine the variance value F k , where k represents different change processes. If there is F k ≤Y4, then this grayscale area is marked as the lesion area, where Y4 is the preset value; S43, perform steps S41-S42 in sequence to calibrate the lesion area in other grayscale areas.

9. A lung endoscope image navigation and recognition method according to claim 8, characterized in that: In step S42, if there is no F k If the change process of Y4 is less than or equal to 4, no calibration is performed.

10. A pulmonary endoscope image navigation and recognition system, the recognition system operates based on a pulmonary endoscope image navigation and recognition method according to any one of claims 1 to 9, characterized in that: include: The image set generation end collects images when the lung endoscope reaches the endoscope point. It uses different aperture coefficients to confirm the standard image, strong exposure image, and weak exposure image associated with the same endoscope point, and generates an image set belonging to the corresponding endoscope point. The fused image generation end selects a standard image from the image set based on the image set confirmed by the corresponding endoscope point, performs grayscale value analysis on the standard image, selects dark areas, intermediate areas, and bright areas from the standard image, and selects related areas from the strong-exposure image and the weak-exposure image based on the region boundaries. Different weight coefficients are assigned to different related areas belonging to different images, and the image set is fused to generate a fused image of the endoscope point. A grayscale region division end determines grayscale value differences between adjacent points based on the determined fused image and grayscale value features of points within the fused image, and divides the fused image into a plurality of different grayscale regions based on the determination result; The lesion area confirmation end determines the center point of the corresponding grayscale area based on the different grayscale areas confirmed in this fused image, then generates data based on the preset lesion model, and verifies the real-time generated data in real time to lock in whether the corresponding grayscale area is the lesion area.

Images