A method and system for hysteroscopic and laparoscopic tracking control based on image processing

Abstract

The present invention discloses a laparoscope and hysteroscope tracking and control method and system based on image processing. The present invention relates to the technical field of laparoscopes and hysteroscopes, and solves the problem that a new control method is not adopted to adjust the shooting posture of the lens in real time, so that the angles of the pictures taken by the lens are all in a consistent state. The present invention processes the spatial positioning points of the shooting device and the generated shooting pictures, determines the relevant spatial signs of the corresponding shooting device when shooting the first picture, identifies the associated feature contours, selects feature points from the corresponding feature contours to lock the feature vectors. Subsequently, when performing associated shooting on the shooting pictures at other positions, according to the determined feature vectors, the device postures at other positions are adjusted in real time, so that the pictures at other positions are consistent with the originally taken pictures in terms of shooting angles, which is convenient for medical staff to make diagnoses and does not require the corresponding medical staff to adjust the angles.

Description

Technical Field

[0001] The present invention relates to the technical field of hysteroscopy and laparoscopy, and particularly to a hysteroscopy and laparoscopy tracking control method and system based on image processing. Background Technique

[0002] Hysteroscopy and laparoscopy is a commonly used minimally invasive surgical technique. By making several small holes in the abdomen, inserting a laparoscope and surgical instruments, and observing the internal abdominal cavity situation on a monitor with the help of a high-definition camera system, surgical operations can be performed. Hysteroscopy and laparoscopy can be used for the diagnosis and treatment of various gynecological diseases, such as endometriosis, uterine fibroids, ovarian cysts, infertility, etc.

[0003] The application with the publication number CN101862216A discloses a laparoscopic self-help uterine lifter, and the technical problem to be solved is to facilitate the use by doctors, with simple operation and improved work efficiency. The present invention adopts the following technical solutions: A laparoscopic self-help uterine lifter includes a base and a uterine lifter. The uterine lifter is connected to the base. The uterine lifter is composed of a vertical rod, a main screw rod, and a uterine lifting screw rod. One end of the vertical rod is connected to the base, and the other end is connected to the main screw rod. The front end of the main screw rod is connected to the uterine lifting screw rod through a universal rotating shaft. A cervical plug or a cervical cup is sleeved on the uterine lifting screw rod. Compared with the prior art, the base and the first and second screw rods are used to fix the uterine lifter on the patient's uterus, and the position of the uterus is adjusted by adjusting the fixing sleeve or the steering rod to expose the field of view, which is beneficial to the operation of pelvic, uterine, or accessory surgeries. It is operated by the endoscope holder in laparoscopic surgery to control the position of the uterus, with simple operation, no need for special personnel control, capable of saving manpower and surgical time, and improving work efficiency.

[0004] During the actual operation and control process of the hysteroscopy and laparoscopy, generally based on the actual detection images of the corresponding hysteroscopy and laparoscopy, relevant personnel make real-time adjustments and changes to the lens to determine the corresponding detection position and ensure that the detection images are comprehensive enough. However, in the original such control and processing method: the range of adjustment and change is large, and the angles of the images taken before and after are inconsistent. There is a certain difficulty coefficient in the actual observation and evaluation process. A new control method is not adopted to make real-time adjustments to the shooting posture of the lens so that the angles of the images taken by the lens are in a consistent state, thereby ensuring the overall correlation effect of the shooting. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides a hysteroscopy and laparoscopy tracking control method and system based on image processing, which solves the problem that a new control method is not adopted to make real-time adjustments to the shooting posture of the lens so that the angles of the images taken by the lens are in a consistent state.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: A hysteroscopy and laparoscopy tracking control method based on image processing includes the following steps:

[0007] Step 1: After the micro shooting device enters the inside of the hysteroscope and laparoscope body, based on the micro-positioning sensor inside it, after the device travels a certain distance, it performs associated shooting to determine the shooting image, and adjusts the shooting posture of the device for subsequent shooting images in real time with the shooting posture of the first group of shooting images;

[0008] Step 2: For several groups of shooting images taken, sort the shooting images according to the associated relationship before and after shooting, and then integrate and splice adjacent shooting images according to the different spatial characteristics of the shooting device to determine and display the spliced image.

[0009] A hysteroscope and laparoscope tracking control system based on image processing, comprising:

[0010] A micro shooting device for high-definition shooting of the picture image inside the hysteroscope and laparoscope and generating a shooting image;

[0011] A micro-positioning sensor is arranged inside the micro shooting device for real-time positioning of the position information of the micro shooting device to generate positioning information;

[0012] An attitude adjustment end, based on the micro-positioning sensor inside it, after the device travels a certain distance, it performs associated shooting to determine the shooting image, and adjusts the shooting posture of the device for subsequent shooting images in real time with the shooting posture of the first group of shooting images;

[0013] A picture splicing end, for several groups of shooting images taken, sorts the shooting images according to the associated relationship before and after shooting, and then integrates and splices adjacent shooting images according to the different spatial characteristics of the shooting device to determine and display the spliced image.

[0014] Among them, in Step 1, the micro-positioning sensor sends the positioning information to the control end in real time. Based on the real-time positioning information, the traveling distance of the micro shooting device is identified. When the traveling distance reaches Y1, where Y1 is a preset value, a shooting instruction is executed, and the first group of shooting images is generated.

[0015] Furthermore, in the said Step 1, the specific sub-steps for real-time adjustment of the shooting posture are:

[0016] S11: Optimize the first group of shooting images. The image optimization includes: denoising, image quality enhancement, and grayscale processing. Identify the internal contour from the shooting images after image optimization. If there is only one group of internal contours, mark this internal contour as the feature contour. If there are multiple groups of internal contours, select a group of internal contours with the largest area as the feature contour;

[0017] Identify its spatial positioning points from the positioning information, determine a set of associated points on this feature contour that are farthest from the spatial positioning points, calibrate these associated points as feature points, and determine a set of spatial feature vectors XL1 from the spatial positioning points to the feature points;

[0018] S12. Based on the micro-positioning sensor inside its micro camera device, when the micro camera device travels another distance of Y1, execute the second set of shooting instructions, and calibrate the spatial positioning point determined at the current moment as D k , at this moment k = 2, representing the spatial positioning point corresponding to the second set of shooting instructions. Based on the trajectory and direction of the micro camera device during this stage, perform a translation lock on the feature contour determined from the first set of shooting images along the same trajectory and in the same direction to obtain the translated contour. Identify the spatial positioning point D2 and connect it to the feature points calibrated within the translated contour to determine the spatial feature vector XL2 of this time. Identify whether XL1 and XL2 are consistent:

[0019] If they are consistent, directly perform shooting to determine the second set of shooting images;

[0020] If they are inconsistent, adjust XL2. During the adjustment process, the corresponding micro camera device will also be adjusted accordingly. After making XL2 consistent with XL1, execute the shooting instructions to determine the second set of shooting images;

[0021] If after multiple adjustments, XL1 and XL2 still cannot be in a consistent state, then adjust the angle of XL2 to be consistent with XL1, and then the shooting instructions can be executed to determine the second set of shooting images;

[0022] S13. During the relevant shooting processes of each subsequent set of different shooting images, according to the same processing method as in step S13 above, perform real-time adjustment on the attitude of the micro camera device, and after the adjustment is completed, determine the shooting images.

[0023] Furthermore, in step two, the specific sub-steps for integrating and splicing adjacent shooting frames include:

[0024] S21. Based on the shooting order, sort several sets of shooting frames, perform an association process on the first set of shooting frames and the second set of shooting frames in adjacent frames, and identify and lock their splicing contours:

[0025] Based on the movement routes of the first group of captured images and the second group of captured images, determine the movement trajectory of their spatial positioning points. The end point of the movement trajectory is the spatial positioning point of the second group of captured images, and the starting point of the movement trajectory is the spatial positioning point of the first group of captured images. Lock the previous point of the movement trajectory and calibrate this point as the standard point. Construct the horizontal base plane of this standard point, which is perpendicular to the movement trajectory, and its perpendicular point is the standard point. Determine the horizontal base plane and identify several intersection points where this horizontal base plane intersects with the first group of captured images. Based on the several intersection points, determine its first group of splicing contours. This perpendicular point belongs to the spatial positioning point of the first group of splicing contours. Then construct a horizontal base plane perpendicular to the end point of the movement trajectory, and based on several intersection points between this horizontal base plane and the second group of captured images, determine its second group of splicing contours. This end point belongs to the spatial positioning point of the second group of splicing contours;

[0026] S22. Based on the spatial positioning points corresponding to the two groups of splicing contours, overlap the two groups of spatial positioning points and analyze whether the associated splicing contours overlap during the overlapping stage:

[0027] If they overlap, directly integrate and splice the first group of captured images and the second group of captured images based on the positions of their splicing contours;

[0028] If they do not overlap, after overlapping the spatial positioning points of the two splicing contours, identify the associated differences between the two splicing contours on the two-dimensional plane, and calibrate the relevant area generated between the two splicing contours as the area to be adjusted;

[0029] Based on the two side lines of the area to be adjusted, lock the midline inside the area to be adjusted. The midline divides the area to be adjusted into two left and right areas, and the areas of the left and right areas are equal. The two end points of the midline are the intersection points generated between the two splicing contours. Based on the determined midline, decompose the two side lines of the area to be adjusted into several decomposition points, and construct perpendicular lines passing through the decomposition points and perpendicular to the midline. Based on the several groups of perpendicular lines constructed, calibrate the two groups of decomposition points associated with the same perpendicular point as the alignment decomposition points;

[0030] Move the alignment decomposition points so that they all move to the position of the associated perpendicular point, so as to gradually overlap the two splicing contours and complete the overall splicing between adjacent captured images;

[0031] S23. For other adjacent images, process them in the same way as steps S21 - S22 in sequence to obtain the splicing image belonging to the overall several groups of captured images.

[0032] The present invention provides a laparoscope and hysteroscope tracking control method and system based on image processing. Compared with the prior art, it has the following beneficial effects:

[0033] The present invention determines the relevant spatial signs of the corresponding imaging device when capturing the first image by performing relevant processing on the spatial positioning points of the imaging device and the captured images generated, and based on the specific captured images, identifies the associated feature contours, selects feature points from the corresponding feature contours, and thus locks the feature vectors based on the spatial positioning points and the feature points. When performing associated imaging on the captured images at other positions subsequently, according to the determined feature vectors, the device postures at other positions are adjusted in real time, so that the images at other positions are consistent with the originally captured images in terms of imaging angles, facilitating medical staff to make diagnoses and eliminating the need for corresponding medical staff to adjust the angles;

[0034] For several groups of captured images generated, in adjacent captured images, the spatial features between adjacent captured images and the corresponding splicing contours are locked. Based on the splicing coincidence situation between the corresponding splicing contours, it is identified whether there is an abnormal splicing between the splicing contours. For relevant contours with incomplete splicing, by determining the area to be adjusted generated between the relevant contours, the corresponding center line is locked, and then the contour is adjusted through the corresponding center line, so that multiple adjacent captured images are associated and spliced, thereby achieving a better display effect and facilitating the overall diagnosis of relevant medical staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic flowchart of the method of the present invention;

[0036] Figure 2 is a schematic diagram for determining the area to be adjusted of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] Embodiment 1

[0039] Please refer to Figure 1 , this application provides a laparoscope and hysteroscope tracking control method based on image processing. For the relevant acquisition process of the skin and muscle tissue images associated inside the laparoscope and hysteroscope, relevant professional instruments need to be inserted for acquisition. The instrument includes a corresponding micro-positioning sensor and a micro-imaging device. The micro-imaging device transmits the acquired tissue images to an external terminal, including the following steps:

[0040] Step 1: After the micro shooting device enters the inside of the hysteroscope and laparoscope, based on the micro positioning sensor inside it, after the device travels a certain distance, it performs associated shooting to determine the shooting image, and in the shooting posture of the first group of shooting images, the shooting posture of the device for subsequent shooting images is adjusted in real time to ensure that the captured images are in a state of consistent angle shooting, which is convenient for subsequent associated analysis. Its posture can be controlled by itself, and a corresponding rotation mechanism is set inside the micro device to facilitate posture adjustment. Because the amplitude of manual posture adjustment is too large, it is very difficult to ensure that the shooting posture of the corresponding shooting device is relatively consistent with the shooting posture of the previous group of images, resulting in the inability to integrate and splice the shooting images of different regions in the later stage. When performing associated analysis on such shooting images in the later stage, it is necessary to compare the images that appear before and after one by one, which is not conducive to the relevant medical work of the corresponding medical staff;

[0041] Among them, the specific sub-steps for real-time adjustment are as follows:

[0042] S11: The micro positioning sensor sends positioning information to the control end in real time. Based on the real-time positioning information, the traveling distance of the micro shooting device is identified. When the traveling distance reaches Y1, where Y1 is a preset value, and its specific value is determined by relevant operators according to experience, generally with a very small value, the shooting instruction is executed, and the first group of shooting images is generated;

[0043] S12. Optimize the first group of captured images. The image optimization includes: denoising, image quality enhancement, and grayscale processing. Since the relevant processing methods associated with image optimization are relatively common in the prior art, they will not be elaborated here. Median filtering is used to denoise the image. For each pixel point in the image, the median of the pixel values in its neighborhood is taken as the new pixel value of this point. Median filtering has a good effect on removing salt-and-pepper noise. For example, for a 3x3 neighborhood, the 9 pixel values are sorted from smallest to largest, and the middle value is taken as the new value of the central pixel point. The image quality enhancement and grayscale processing include: by adjusting the grayscale histogram of the image, making the grayscale distribution of the image more uniform. This can enhance the contrast of the image and make the details in the image more obvious. For example, for a grayscale image, the distribution of its grayscale values is statistically analyzed, and then the grayscale values are remapped so that the probability of each grayscale level appearing in the image is roughly equal. The sharpness of the image is improved by enhancing the edge and detail information of the image. Common sharpening methods include the Laplacian operator, the Sobel operator, etc. These operators detect edges by calculating the gradient of the image and then enhance the edges. The color effect of the image is enhanced by adjusting parameters such as the color saturation and hue of the image. For example, the color saturation can be increased to make the colors of the image more vivid, or the hue can be adjusted to make the colors of the image more in line with human visual perception. Identify the internal contour from the captured images after completing the image optimization (that is, the edge contour line corresponding to the image. Each group of edge contour lines forms a closed loop and then is the corresponding internal contour). If there is only one group of internal contours, this internal contour is marked as the feature contour. If there are multiple groups of internal contours, then select a group of internal contours with the largest area as the feature contour;

[0044] Identify the spatial positioning point from the positioning information. Determine a group of associated points on this feature contour that are farthest from the spatial positioning point, and mark this group of associated points as the feature points. Starting from the spatial positioning point to the feature points, determine a group of spatial feature vectors XL1 (the vector has an angle and a length. The reference of its angle can be determined based on the set horizontal plane or other horizontal planes in different directions, so there is no associated limitation here and it can be understood as a group of feature vectors in space);

[0045] S13. Based on the micro-positioning sensor inside the micro-capturing device, when the micro-capturing device travels another distance of Y1, execute the second group of shooting instructions and mark the spatial positioning point determined at the current moment as D k, at this moment k = 2, representing the spatial positioning point corresponding to the second group of shooting instructions. Based on the trajectory and direction of the micro shooting device during this stage, the feature contour determined from the first group of shooting images is translated and locked in the same trajectory and direction to obtain a translated contour (the trajectory and direction here do not include the rotation direction. Since the positioning sensor cannot identify whether rotation has occurred, it can only identify its spatial movement points and the corresponding movement directions. When the contour is translated, according to the corresponding spatial direction, its posture in space will also change accordingly. For example, when moving in an arc, the corresponding contour will change from a side view direction to a top view direction, etc. This is just a simple example and not specifically for this case), identify the spatial feature vector XL2 determined by connecting the spatial positioning point D2 with the feature points calibrated within the translated contour, and identify whether XL1 and XL2 are consistent:

[0046] If they are consistent, directly perform shooting to determine the second group of shooting images;

[0047] If they are not consistent, adjust XL2. During the adjustment process, the corresponding micro shooting device will also be adjusted accordingly. After making XL2 consistent with XL1, execute the shooting instruction to determine the second group of shooting images;

[0048] There is also a situation: if after multiple adjustments, it is still impossible to be in a consistent state (that is, when the corresponding micro device is moving, there are specific relevant obstacles, resulting in the inability to make the lengths of the vectors consistent. Then, as long as the angles are consistent, it is okay), after adjusting the angle of XL2 to be consistent with XL1, the shooting instruction can be executed to determine the second group of shooting images;

[0049] S14. During the shooting process of each subsequent group of different shooting images, according to the same processing method as in step S13 above, the posture of the micro shooting device is adjusted in real time, and after the adjustment is completed, the shooting images are determined;

[0050] Specifically, for the shooting scenes with different travel distances each time, in order to ensure that the corresponding shooting device has high-intensity consideration when shooting the scene, first of all, the shooting angles of the corresponding shooting device should be as consistent as possible. Then, there is no need for external medical staff to adjust the shooting angles in real time. Because when medical staff make adjustments, the adjustment range will be too large. Then, the adjustment range will not only have a large deviation, but also cause discomfort to the detected person. Therefore, based on the corresponding travel trajectory and direction, the generated feature contour is adjusted and changed in real time to lock the corresponding feature vector, and real-time adjustment and attitude control are performed based on the corresponding feature vector to ensure that the images are relatively consistent during shooting. Since most of the structures in the uterine cavity and abdominal cavity are channel-shaped, if the angles are not consistent, it is not conducive to the specific diagnosis of the corresponding medical staff.

[0051] Step 2: For several groups of captured images, sort the captured images according to the correlation before and after shooting, and then integrate and splice adjacent captured images according to the different spatial characteristics of their shooting devices to determine and display the spliced image. Specifically, the spliced image can better display the correlation of internal tissue characteristics, facilitating better diagnosis by medical staff;

[0052] Among them, the specific sub-steps for integration and splicing include:

[0053] S21: Sort several groups of captured images according to their shooting order, perform correlation processing on the first group of captured images and the second group of captured images in adjacent images, and identify and lock their splicing contours:

[0054] Based on the movement routes of the first group of captured images and the second group of captured images, determine the movement trajectory of their spatial positioning points (the spatial positioning points are the real-time positioning routes of the corresponding micro-positioning sensors, and the real-time positioning routes are the driving routes from the first group of captured images to the second group of captured images). The end point of the movement trajectory is the spatial positioning point of the second group of captured images, and the starting point of the movement trajectory is the spatial positioning point of the first group of captured images. Lock the previous point of the movement trajectory and calibrate this point as the standard point (the unit length between its adjacent points is a preset value, determined by relevant operators according to experience, or can also be considered as the interval frequency value of the corresponding positioning sensor. For example, if this end point is the point sent at the corresponding time point, then the previous group of points is the point sent at the previous time point of the corresponding time point. The time between the two time intervals is the corresponding interval frequency, and the distance length between two adjacent spatial points can be basically ignored). Construct the horizontal base plane of this standard point. The horizontal base plane is perpendicular to the movement trajectory, and the perpendicular point is the standard point. Determine the horizontal base plane and identify several intersection points where this horizontal base plane intersects with the first group of captured images. Based on the several intersection points, determine its first group of splicing contours. This perpendicular point belongs to the spatial positioning point of the first group of splicing contours (the captured images can be understood as the associated images of the channel, and the horizontal base plane can be understood as the horizontal plane perpendicular to this channel, so there will be intersection points, and the contour line generated by the intersection points is the associated contour). Then construct a horizontal base plane perpendicular to the end point of the movement trajectory, and based on several intersection points between this horizontal base plane and the second group of captured images, determine its second group of splicing contours. This end point belongs to the spatial positioning point of the second group of splicing contours;

[0055] S22: Based on the spatial positioning points of the corresponding two groups of splicing contours, overlap the two groups of spatial positioning points and analyze whether the associated splicing contours overlap during the overlapping stage:

[0056] If they coincide, directly integrate and splice the first set of captured images and the second set of captured images based on the position of their splicing contours;

[0057] If they do not coincide (the reason for non - coincidence is that when the corresponding detection device moves, the muscle tissue in the corresponding abdominal cavity channel will change or move due to extrusion, resulting in associated changes in the corresponding captured images, which will cause the image situations corresponding to the front and rear nodes to be different, which can be understood as inconsistent spanning lengths), after making the spatial positioning points of the two splicing contours coincide, identify the associated differences between the two splicing contours on the two - dimensional plane, and label the relevant area generated between the two splicing contours as the area to be adjusted (that is, the area where the two splicing contours do not cross - coincide, such as Figure 2 the filled area in, and the filled relevant area is the corresponding area to be adjusted);

[0058] Based on the two side lines of the area to be adjusted, lock the mid - line inside the area to be adjusted. The mid - line divides the area to be adjusted into two left - and - right areas with equal areas (since the determination method of the mid - line is relatively common, it will not be elaborated here too much. Generally, the triangle method is used to determine two left - and - right areas with the same area), and the two endpoints of the mid - line are the intersection points generated between the two splicing contours. Based on the determined mid - line, decompose the two side lines of the area to be adjusted into several decomposition points, and construct perpendicular lines passing through the decomposition points and perpendicular to the mid - line. Based on the constructed several groups of perpendicular lines, label the two groups of decomposition points associated with the same perpendicular point as the alignment decomposition points;

[0059] Move the alignment decomposition points so that they all move to the position of the associated perpendicular point, so as to gradually make the two splicing contours coincide and complete the overall splicing between adjacent captured images. Specifically, in the process of image splicing here, there is no need to consider the relevant problems of the image texture or stretching inside the image after splicing. First of all, the area to be adjusted between the two contours is very small. Since the distance between the two determined spatial positioning points can be ignored, even if there are relevant changes in the organizational structure, the specific contour differences of the changes are also small. Then, by making associated adjustments to the edges of the image, the impact on the captured image can be ignored. Therefore, there is no need to consider the impact of the splicing process on the captured images;

[0060] S23. For other adjacent images, process them in the same way as steps S21 - S22 in sequence to obtain the spliced image of the overall several groups of captured images;

[0061] Step 3: For the completed stitched image, the stitching trace is retained and displayed at the stitching location. The reason for retaining the trace for associated display is to facilitate external medical staff to view that stitching processing has been performed here. If the original associated captured images need to be viewed, they can be directly selected and viewed from the several transmitted captured images.

[0062] Embodiment 2

[0063] A laparoscope and hysteroscope tracking control system based on image processing, comprising:

[0064] A micro shooting device for performing high-definition shooting on the internal image of the laparoscope and hysteroscope to generate captured images;

[0065] A micro-positioning sensor is arranged in the micro shooting device for real-time positioning of the position information of the micro shooting device to generate positioning information;

[0066] An attitude adjustment end, based on the micro-positioning sensor inside it, after this device travels a certain distance, associated shooting is performed to determine the captured images, and the shooting attitude of this device for subsequent captured images is adjusted in real time with the shooting attitude of the first group of captured images.

[0067] An image stitching end, for several groups of captured images taken, sorts the captured images according to the associated relationship before and after shooting, and then integrates and stitches adjacent captured images according to the different spatial characteristics of the shooting devices to determine and display the stitched image.

[0068] Some data in the above formula are numerically calculated after removing their dimensions, and the content not described in detail in this specification belongs to the prior art well known to those skilled in the art.

[0069] The above embodiments are only used to illustrate the technical method of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A laparoscope and hysteroscope tracking control system based on image processing, characterized in that, Comprising: A micro shooting device for performing high-definition shooting on the picture image inside the hysteroscope and laparoscope and generating a shooting image; A micro positioning sensor disposed inside the micro shooting device for real-time positioning of the position information of the micro shooting device to generate positioning information; An attitude adjustment end, based on the micro positioning sensor, after the micro shooting device travels a certain distance each time, performs associated shooting to determine the shooting image, and in the shooting attitude of the first group of shooting images, adjusts the shooting attitude of the subsequent shooting pictures of the micro shooting device in real time; A picture splicing end, for a number of groups of shooting pictures taken, sorts the shooting pictures according to the associated relationship before and after shooting, and then integrates and splices adjacent shooting pictures according to the different spatial characteristics of the micro shooting device to determine and display the spliced picture. The specific method of integration and splicing is: S21. Sort a number of groups of shooting pictures according to the shooting order, perform associated processing on the first group of shooting pictures and the second group of shooting pictures in adjacent pictures, and identify and lock the splicing contour: Based on the traveling routes of the first group of shooting pictures and the second group of shooting pictures, determine the moving trajectory of the spatial positioning points. The end point of the moving trajectory is the spatial positioning point of the second group of shooting pictures, and the starting point of the moving trajectory is the spatial positioning point of the first group of shooting pictures. Lock the previous point of the moving trajectory and calibrate this point as the standard point. Construct the horizontal base plane of this standard point. The horizontal base plane is perpendicular to the moving trajectory, and the perpendicular point is the standard point. Determine the horizontal base plane and identify several intersection points where this horizontal base plane intersects with the first group of shooting pictures. Based on the several intersection points, determine the first group of splicing contours. This perpendicular point belongs to the spatial positioning point of the first group of splicing contours. Then construct a horizontal base plane perpendicular to the end point of the moving trajectory, and based on several intersection points between this horizontal base plane and the second group of shooting pictures, determine the second group of splicing contours. This end point belongs to the spatial positioning point of the second group of splicing contours; S22. Based on the spatial positioning points of the corresponding two groups of splicing contours, overlap the two groups of spatial positioning points and analyze whether the associated splicing contours overlap during the overlapping stage: If they overlap, directly integrate and splice the first group of shooting pictures and the second group of shooting pictures based on the positions of the splicing contours; If they do not overlap, after overlapping the spatial positioning points of the two splicing contours, identify the associated differences between the two splicing contours on the two-dimensional plane, and label the relevant area generated between the two splicing contours as the area to be adjusted; Based on the two side lines of the area to be adjusted, lock the midline inside the area to be adjusted. The midline divides the area to be adjusted into two left and right areas, and the areas of the left and right areas are equal. The two end points of the midline are the intersection points generated between the two splicing contours. Based on the determined midline, decompose the two side lines of the area to be adjusted into several decomposition points, and construct perpendicular lines passing through the decomposition points and perpendicular to the midline. Based on the several groups of perpendicular lines constructed, label the two groups of decomposition points associated with the same perpendicular point as the paired decomposition points; Move the alignment decomposition points so that they are all moved to the associated vertical point positions, so that the two splicing contours are gradually coincident, and the overall splicing between adjacent captured images is completed.

2. The laparoscopic tracking control system based on image processing according to claim 1, wherein The micro-positioning sensor sends positioning information to the control terminal in real time. Based on the real-time positioning information, the traveling distance of the micro-capture device is identified. When the traveling distance reaches Y1, where Y1 is a preset value, a shooting instruction is executed, and a first set of captured images is generated.

3. A laparoscopic and hysteroscopic tracking control system based on image processing according to claim 2, wherein, The specific sub-steps for the attitude adjustment end to adjust the shooting attitude in real time are as follows: S11. Optimize the first set of captured images. Image optimization includes: denoising, image quality enhancement, and grayscale processing. Identify the internal contours from the captured images after image optimization. If there is only one set of internal contours, label this internal contour as the feature contour. If there are multiple sets of internal contours, select a set of internal contours with the largest area as the feature contour; Identify the spatial positioning points from the positioning information, determine a set of associated points on this feature contour that are farthest from the spatial positioning points, label this associated point as the feature point, and determine a set of spatial feature vectors XL1 from the spatial positioning points to the feature points; S12. Based on the micro-positioning sensor inside the micro shooting device, when the micro shooting device travels another distance of Y1, execute the second set of shooting instructions and calibrate the spatial positioning point determined at the current moment as D k , at this moment k = 2, representing the spatial positioning point corresponding to the second set of shooting instructions. Based on the trajectory and direction of the micro shooting device during this stage, perform a same-trajectory and same-direction translation lock on the feature contour determined from the first set of shooting images to obtain the translation contour, identify the spatial positioning point D2, and then connect it with the feature points calibrated within the translation contour to determine the spatial feature vector XL2 for this time. Identify whether the spatial feature vector XL1 is consistent with the spatial feature vector XL2: If they are consistent, directly perform shooting and determine the second set of captured images; If they are inconsistent, adjust the spatial feature vector XL2. During the adjustment process, the corresponding micro-capture device will also follow the adjustment. After the spatial feature vector XL2 is made consistent with the spatial feature vector XL1, execute the shooting instruction and determine the second set of captured images; S13. During the relevant shooting process of each subsequent set of different captured images, according to the same processing method as in step S12 above, adjust the attitude of the micro-capture device in real time, and after the adjustment is completed, determine the captured images.

4. A laparoscope and hysteroscope tracking control system based on image processing according to claim 3, wherein, The step S12 also includes: If after multiple adjustments, the spatial feature vector XL1 and the spatial feature vector XL2 still cannot be in a consistent state, then after adjusting the angle of the spatial feature vector XL2 to be consistent with the spatial feature vector XL1, the shooting instruction can be executed to determine the second set of captured images.

5. A laparoscope and hysteroscope tracking control system based on image processing according to claim 1, characterized in that, In the picture splicing end, the method for determining the splicing pictures of the overall several sets of captured pictures is as follows: S23. For other adjacent pictures, process them in the same way as steps S21 - S22 in sequence to obtain the splicing pictures belonging to the overall several sets of captured pictures.

Images