3D endoscope image processing system based on artificial intelligence assistance

Abstract

The invention relates to the technical field of image processing and intelligent endoscopic imaging, in particular to a 3D endoscope image processing system based on artificial intelligence assistance, which comprises an image and object distance acquisition module, an AI parallax dynamic generation module, a physiological parameter adaptation module, a macro optimization module and a stereo imaging output module, wherein the image and object distance acquisition module is used for acquiring an original image and real-time object distance data; the AI parallax dynamic generation module is used for generating an initial parallax parameter associated with an object distance; the physiological parameter adaptation module is used for outputting personalized parallax parameters; the macro optimization module is used for applying asymmetric compensation to the personalized parallax parameters; and the stereo imaging output module is used for carrying out space-time segmentation, sub-pixel displacement and channel synthesis on the original image. According to the invention, through fusion of high-precision object distance acquisition, personalized parallax adjustment and dynamic region fusion processing, three-dimensional high-fidelity restoration of a scene in a complex cavity under a single optical path structure is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing and intelligent endoscopic imaging technology, and in particular to an artificial intelligence-assisted 3D endoscopic image processing system. Background Art

[0002] In laparoscopic minimally invasive surgery, nasopharyngeal endoscopy and intraoperative precision operation scenarios, 3D endoscopes have gradually replaced traditional two-dimensional imaging systems with their stereoscopic vision advantages; however, existing 3D endoscopes mostly rely on dual-optical path structures, and have problems such as large size, high hardware cost, and complex optical axis alignment, making them difficult to be widely used in highly constrained spaces.

[0003] To overcome hardware bottlenecks, some systems attempt to generate stereoscopic images using single-optical-path image acquisition combined with post-processing. However, due to the lack of high-precision object distance acquisition and depth modeling mechanisms, the generated images often exhibit weak stereoscopic perception, blurred tissue boundaries, and sluggish response to instrument movement, making it difficult to meet the dual requirements of real-time performance and spatial fidelity in actual surgical scenarios. Therefore, there is an urgent need to build an AI-assisted 3D endoscopic image processing system to improve the accuracy and universality of 3D endoscopic images. Summary of the Invention

[0004] Based on the above objectives, the present invention provides a 3D endoscopic image processing system based on artificial intelligence assistance.

[0005] An artificial intelligence-assisted 3D endoscope image processing system includes an image and object distance acquisition module, an AI parallax dynamic generation module, a physiological parameter adaptation module, a macro optimization module, and a stereoscopic imaging output module; wherein:

[0006] Image and object distance acquisition module: used to obtain the original image through a single optical path focusable lens and synchronously measure real-time object distance data;

[0007] AI disparity dynamic generation module: used to receive the original image and object distance data, and generate the initial disparity parameters associated with the object distance through the convolutional neural network;

[0008] Physiological parameter adaptation module: used to receive the physiological parameters collected by the operator's pupil tracker and output personalized disparity parameters based on the initial disparity parameters;

[0009] Macro Optimization Module: When the object distance data is less than the set threshold, it applies asymmetric compensation to the personalized parallax parameters and outputs the optimized parallax parameters;

[0010] Stereoscopic imaging output module: Based on the optimized disparity parameters, the original image is subjected to spatiotemporal segmentation, sub-pixel shifting, and channel synthesis, and finally outputs a 3D endoscopic image stream.

[0011] Optionally, the image and object distance acquisition module includes an optical imaging unit, a composite distance measurement unit and a synchronization control unit; wherein:

[0012] Optical imaging unit: including a focus-adjustable liquid lens and a CMOS image sensor. The focus-adjustable liquid lens is used to respond to focus adjustment instructions to dynamically adjust the focus range. The CMOS image sensor is used to collect single-light-path original images.

[0013] Composite ranging unit: used to output object distance data in real time by fusing ToF infrared delay ranging and three-wavelength laser speckle distortion analysis;

[0014] Synchronous control unit: Used to generate frame trigger signals through FPGA, synchronously control the focus switching of the adjustable focus liquid lens, the exposure timing of the CMOS image sensor, and the laser emission cycle of the composite ranging unit, ensuring the spatiotemporal alignment of the original image and the real-time object distance data.

[0015] Optionally, the composite ranging unit includes:

[0016] The ToF sensor subunit is used to transmit modulated infrared light waves to the target area, receive the reflected echo, and calculate the first object distance data D1 based on the propagation delay;

[0017] Three-wavelength laser speckle generator subunit: used to synchronously emit visible spectrum speckle and near-infrared speckle patterns, and obtain the second object distance data D2 based on speckle distortion analysis;

[0018] Object distance calculation subunit: used to fuse the first object distance data D1 and the second object distance data D2, and output the final real-time object distance data D using a weighted average method.

[0019] Optionally, the AI ​​disparity dynamic generation module includes a data fusion unit, a dual-branch convolutional network unit, and a dynamic association unit; wherein:

[0020] Data fusion unit: used to divide the original image into grid areas, bind the corresponding object distance label to each image block, and construct a space-depth integrated image block input set;

[0021] Dual-branch convolutional network unit: It is used to extract the spatial texture features of the image and the semantic features of the object distance label in parallel, and fuse the two types of information in the decoding stage to generate a structured initial disparity map;

[0022] Dynamic association unit: extracts multiple disparity regions based on the gradient distribution of the initial disparity map and object distance data, and generates a disparity reference parameter set associated with the object distance change.

[0023] Optionally, the physiological parameter adaptation module includes a physiological parameter interface unit, a parameter mapping unit, and a personalized correction unit; wherein:

[0024] Physiological parameter interface unit: used to receive pupil distance D in real time via near-field wireless link pupil , Dominant Eye Marker S eye and dynamic focusing frequency f focus Three types of physiological data are synchronously stored in the buffer;

[0025] Parameter mapping unit: used to scale the initial disparity parameters based on the pupil distance, and perform asymmetric weight adjustment on the left and right channels according to the dominant eye identifier to form a basic personalized parameter set;

[0026] Personalized correction unit: used to perform motion state discrimination and differentiated correction on the basic personalized parameter set according to the dynamic focusing frequency.

[0027] Optionally, the parameter mapping unit includes:

[0028] Scaling correction subunit: used to adjust the pupil distance D pupil Scaling and correcting the initial parallax parameter set to generate a basic personalized parameter set;

[0029] Asymmetric weighted subunit: used to identify S according to eye dominance eye Perform asymmetric weighted adjustment on the left and right channel parameters. Specifically, when S eye = left eye, then the left channel parameter is multiplied by 0.9, and the right channel parameter is multiplied by 1.1; when S eye =right eye, then the left channel parameters are multiplied by 1.1 and the right channel parameters are multiplied by 0.9.

[0030] Optionally, the personalized correction unit includes:

[0031] When f focus When the frequency is >2Hz, it indicates that the operator is frequently adjusting focus, and time domain smoothing filtering is applied to the basic personalized parameter set;

[0032] When f focus When the value is ≤2Hz, it indicates that the operator's focusing state is stable, and the current personalized parallax parameters are kept unchanged.

[0033] Optionally, the macro optimization module includes an object distance monitoring unit and a layered compensation unit; wherein:

[0034] Object distance monitoring unit: used to continuously receive real-time object distance data output by the image and object distance acquisition module. When the real-time object distance is detected to be less than 50mm, the compensation channel is automatically activated to enter the macro parallax adjustment process.

[0035] Layered compensation unit: used to perform spatially variable weighted asymmetric compensation on the personalized disparity parameter set in the macro state, specifically including basic compensation and tissue adaptation compensation;

[0036] The basic compensation is used to apply a uniform compensation amount δ to all parallax areas, and its calculation expression is: δ = 0.23·D 2 , where: δ is the basic compensation amount; D is the real-time object distance data;

[0037] The tissue adaptation compensation is used to adjust the compensation weight according to the reflectivity characteristics of the intracavitary area. Specifically, if it is a high-reflection area, the compensation weight is 1.2; if it is a low-reflection area, the compensation weight is 0.8.

[0038] Optionally, the stereoscopic imaging output module includes a time-space segmentation unit, a displacement engine unit, and a dynamic synthesis unit; wherein:

[0039] The spatiotemporal segmentation unit receives the original image and optimized disparity parameters, samples the single-frame original image alternately in a checkerboard pattern, and divides it into left-channel and right-channel subframes. It generates a displacement control matrix based on the optimized disparity parameters and marks the displacement priority of each pixel.

[0040] Shift engine unit: used to perform differential sub-pixel shift on the left and right channel subframes: specifically, applying positive shift to the left channel image and negative shift to the right channel image;

[0041] Dynamic synthesis unit: used to perform dual-channel image fusion processing after completing sub-pixel displacement, output 3D endoscopic image stream, and dynamic delay ≤ 2ms.

[0042] Optionally, the dynamic synthesis unit includes:

[0043] Tissue recognition subunit: Extract inter-frame motion features from left and right channel image frames that have undergone sub-pixel shift processing, and form a consistency distribution map by comparing the motion vector direction and amplitude of corresponding pixels;

[0044] Region separation subunit: used to divide the consistency distribution map into motion consistency regions and inconsistency regions, and generate a spatial mask for each type of region;

[0045] Fusion processing subunit: used for performing a weighted fusion operation based on consistency weight on the left channel and right channel images in the tissue area;

[0046] Enhancement control subunit: used to perform local sharpening processing on the image within the device area and enhance the clarity of the device boundary by constructing a Laplace edge enhancement operator.

[0047] Beneficial effects of the present invention:

[0048] The present invention breaks through the technical limitations of traditional binocular 3D endoscopy, which has complex structures and limited space, by constructing a single-optical path 3D image processing system that integrates image acquisition, AI disparity generation, physiological parameter adaptation, macro compensation and dynamic fusion processing. It achieves high-precision restoration of the depth of the intracavitary space in a compact device structure. The system builds a highly robust real-time object distance input foundation by fusing ToF and speckle ranging, and cooperates with dual-branch disparity modeling based on convolutional neural networks to effectively improve the spatial consistency and depth recognition accuracy of the initial disparity generation.

[0049] The present invention introduces a parallax adjustment mechanism tailored to individual surgeon differences and a macro compensation strategy driven by object distance. The system can achieve adaptive responses to different visual behaviors and tissue reflection characteristics, enhancing the adaptability of images in macro and dynamic operation scenarios. At the same time, the combination of spatiotemporal segmentation and sub-pixel displacement engine, combined with tissue-instrument partition recognition and fusion enhancement processing, significantly improves the dynamic clarity and spatial stability of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 Schematic diagram of a 3D endoscope image processing system according to an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of a physiological parameter adaptation module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It is also noted that, to provide a more detailed description, the following embodiments are best and preferred embodiments, and those skilled in the art may employ alternative methods for implementing certain known technologies. Furthermore, the accompanying drawings are intended only to provide a more detailed description of the embodiments and are not intended to limit the present invention.

[0054] It should be noted that references in the specification to "one embodiment," "an embodiment," "an exemplary embodiment," "some embodiments," etc. indicate that the described embodiments may include specific features, structures, or characteristics, but not every embodiment necessarily includes such specific features, structures, or characteristics. In addition, when specific features, structures, or characteristics are described in conjunction with an embodiment, it is within the knowledge of persons skilled in the relevant art to implement such features, structures, or characteristics in conjunction with other embodiments (whether or not explicitly described).

[0055] In general, terms can be understood, at least in part, from their use in context. For example, depending at least in part on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending at least in part on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0056] like Figure 1-Figure 2 As shown, an artificial intelligence-assisted 3D endoscope image processing system includes an image and object distance acquisition module, an AI parallax dynamic generation module, a physiological parameter adaptation module, a macro optimization module, and a stereoscopic imaging output module; wherein:

[0057] Image and object distance acquisition module: used to obtain the original image through a single optical path focusable lens and synchronously measure real-time object distance data;

[0058] AI disparity dynamic generation module: used to receive the original image and object distance data, and generate the initial disparity parameters associated with the object distance through the convolutional neural network;

[0059] Physiological parameter adaptation module: used to receive the physiological parameters collected by the operator's pupil tracker and output personalized disparity parameters based on the initial disparity parameters;

[0060] Macro Optimization Module: When the object distance data is less than the set threshold, it applies asymmetric compensation to the personalized parallax parameters and outputs the optimized parallax parameters;

[0061] Stereoscopic imaging output module: Based on the optimized disparity parameters, the original image is subjected to spatiotemporal segmentation, sub-pixel shifting, and channel synthesis, and finally outputs a 3D endoscopic image stream.

[0062] The image and object distance acquisition module includes an optical imaging unit, a composite distance measurement unit and a synchronization control unit; wherein:

[0063] Optical imaging unit: includes a focus-adjustable liquid lens and a CMOS image sensor. The focus-adjustable liquid lens is used to respond to focus adjustment commands to dynamically adjust the focus range from 4.5 to 60 mm. The CMOS image sensor is used to capture single-light-path original images.

[0064] Composite ranging unit: This unit is used to output high-precision object distance data in real time by fusing ToF infrared delay ranging and three-wavelength laser speckle distortion analysis, ensuring high resolution and high robustness of depth information acquired during single-optical path imaging.

[0065] Synchronous control unit: Used to generate frame trigger signals through FPGA, synchronously control the focus switching of the adjustable-focus liquid lens, the exposure timing of the CMOS image sensor, and the laser emission cycle of the composite ranging unit, ensuring the spatiotemporal alignment of the original image and the real-time object distance data. The above steps, by constructing an image and object distance acquisition module that integrates optical imaging, dual-channel ranging, and high-precision timing control, can not only obtain original image data with consistent resolution and controllable distortion, but also achieve precise positioning of target objects at any viewpoint, providing highly consistent input data for subsequent AI parallax dynamic generation and three-dimensional image synthesis, significantly improving the imaging accuracy and real-time response capabilities of the entire system.

[0066] The composite ranging unit includes:

[0067] The ToF sensor subunit is used to transmit modulated infrared light waves to the target area, receive the reflected echo, and calculate the first object distance data D1 based on the propagation delay. The calculation expression is: Where D1 is the first object distance data; c is the speed of light, which is 3×10 8 m / s; Δt is the round-trip time delay;

[0068] The three-wavelength laser speckle generator subunit is used to synchronously emit visible spectrum speckle patterns with a wavelength of 400-650nm and near-infrared speckle patterns at 830nm and 1310nm. The second object distance data D2 is obtained based on speckle distortion analysis. The formula is: D2 = k·δ -1 , where D2 is the second object distance data; k is the distortion constant obtained by calibration; δ is the speckle pattern distortion displacement;

[0069] The object distance calculation subunit is used to fuse the first object distance data D1 and the second object distance data D2, and output the final real-time object distance data D using the weighted average method. The fusion calculation method is: Where D is the real-time object distance data; w1 and w2 are the trust weights of D1 and D2 respectively, and it is preset to satisfy w1+w2=1.

[0070] The AI ​​disparity dynamic generation module includes a data fusion unit, a dual-branch convolutional network unit, and a dynamic association unit; among them:

[0071] Data fusion unit: This unit divides the original image into grid regions and associates a corresponding object distance label with each image block, thereby constructing a spatial-depth integrated image block input set and providing a linkage feature foundation for subsequent neural networks. The specific operations are as follows:

[0072] 1) Divide a single frame image I into M×N grid cells;

[0073] 2) Bind the object distance value d of the corresponding spatial position to each grid cell i,j, generates a set of image blocks containing spatial position and object distance labels, expressed as: Among them, B i,j is the image block in row i and column j; d i,j is the object distance label corresponding to the image block;

[0074] Dual-branch convolutional network unit: It is used to extract the spatial texture features of the image and the semantic features of the object distance label in parallel, and fuse the two types of information in the decoding stage to generate a structured initial disparity map;

[0075] Let the spatial texture feature vector be F s , the object distance embedding vector is F d , then the initial disparity map P0 is expressed as: Among them, Ψ is the fused decoding function; It is a feature fusion operation (splicing plus attention weighting);

[0076] Dynamic association unit: Extracts multiple disparity regions based on the gradient distribution of the initial disparity map and object distance data, generates a disparity reference parameter set associated with object distance changes, and constructs a basic disparity model for subsequent image control; the above unit constructs a dual-branch convolutional network structure that fuses image spatial features and object distance labels, and combines gradient distribution analysis to achieve initial disparity regionalization modeling, which can effectively improve the spatial resolution and depth accuracy of disparity estimation, and provide a stable and precision-controlled basic disparity parameter set for subsequent personalized adjustment and stereo image generation.

[0077] The physiological parameter adaptation module includes a physiological parameter interface unit, a parameter mapping unit, and a personalized correction unit; wherein:

[0078] Physiological parameter interface unit: used to receive pupil distance D in real time via near-field wireless link pupil , Dominant Eye Marker S eye and dynamic focusing frequency f focus The three types of physiological data are synchronously stored in the buffer to ensure the timing consistency of subsequent parallax processing, thereby improving the accuracy of data fusion;

[0079] Parameter mapping unit: used to scale the initial disparity parameters based on the pupil distance and asymmetrically adjust the weights of the left and right channels according to the dominant eye identifier to form a basic personalized parameter set, thereby enhancing the accuracy of stereo depth matching;

[0080] Personalized correction unit: used to perform motion state discrimination and differentiated correction on the basic personalized parameter set according to the dynamic focusing frequency.

[0081] The parameter mapping unit includes:

[0082] Scaling correction subunit: used to adjust the pupil distance D pupilThe initial parallax parameter set is scaled and corrected to generate a basic personalized parameter set. The scaling correction expression is: Where: ΔP i is the reference disparity value of the i-th initial disparity area; ΔP i ′ is the personalized parallax parameter after scaling; 54 is the reference value of the standard pupil distance of the human eye;

[0083] Asymmetric weighted subunit: used to identify S according to eye dominance eye Perform asymmetric weighted adjustment on the left and right channel parameters. Specifically, when S eye = left eye, then the left channel parameter is multiplied by 0.9, and the right channel parameter is multiplied by 1.1; when S eye =right eye, the left channel parameters are multiplied by 1.1 and the right channel parameters are multiplied by 0.9. By applying asymmetric weighted adjustment to the generated left and right disparity parameters, the disparity signal of the channel corresponding to the operator's dominant eye is preferentially enhanced to strengthen the depth recognition ability of the dominant visual channel.

[0084] The personalized correction unit includes:

[0085] When f focus When the frequency is >2Hz, it indicates that the operator is in a state of frequent focusing. In this case, time domain smoothing filtering is applied to the basic personalized parameter set. The expression is:

[0086] ΔP i ″(t)=α·ΔP i ′(t)+(1-α)·ΔP i ″(t-1), where ΔP i ″(t) is the final personalized disparity value at time t; α is the smoothing coefficient, ranging from 0.1 to 0.3; t is the time index of the current frame;

[0087] When f focus When ≤2Hz, it indicates that the operator’s focus state is stable, and the current personalized parallax parameter remains unchanged, that is: ΔP i ″=ΔP i '; This physiological parameter adaptation module accesses individualized physiological feature data and introduces a dynamic adjustment mechanism in the parallax parameter generation process, making the three-dimensional image output more consistent with the operator's subjective perception characteristics in terms of spatial stereoscopic sense and visual comfort, thereby significantly improving the visual guidance efficiency and long-term observation stability in endoscopic operations.

[0088] The macro optimization module includes an object distance monitoring unit and a layered compensation unit; wherein:

[0089] Object distance monitoring unit: used to continuously receive real-time object distance data output by the image and object distance acquisition module. When the real-time object distance is detected to be less than 50mm, the compensation channel is automatically activated to enter the macro parallax adjustment process.

[0090] Layered compensation unit: used to perform spatially variable weighted asymmetric compensation on the personalized disparity parameter set in the macro state, specifically including basic compensation and tissue adaptation compensation;

[0091] The basic compensation is used to apply a uniform compensation amount δ to all parallax areas, and its calculation expression is: δ = 0.23·D 2 , where: δ is the basic compensation amount; D is the real-time object distance data; the parallax parameter after basic compensation is updated as:

[0092] Tissue adaptation compensation is used to adjust the compensation weight based on the reflectivity characteristics of the intracavitary area. Specifically, if it is a high-reflection area (such as the mucosal surface), the compensation weight is 1.2; if it is a low-reflection area (such as the vascular plexus), the compensation weight is 0.8. The disparity parameters output after adjustment are: where w i is the reflection weight factor corresponding to the area; the above-mentioned unit automatically activates the parallax compensation channel when the real-time object distance data is less than the set threshold, and dynamically adjusts the local weight based on the tissue reflection characteristics of the target area, thereby effectively compensating for the depth perception error under macro imaging conditions and improving the hierarchical expression and spatial consistency of three-dimensional images in short-distance scenes.

[0093] The stereoscopic imaging output module includes a time-space segmentation unit, a displacement engine unit, and a dynamic synthesis unit; wherein:

[0094] The spatiotemporal segmentation unit receives the original image and optimized disparity parameters, samples the single-frame original image alternately in a checkerboard pattern, and divides it into left-channel and right-channel subframes. It generates a displacement control matrix based on the optimized disparity parameters and marks the displacement priority of each pixel.

[0095] Displacement engine unit: used to perform differential sub-pixel displacement on the left and right channel subframes: Specifically, a positive displacement is applied to the left channel image, and the expression is: I L ′(x, y)=I L (x+ΔP i , y); apply reverse displacement to the right channel image, the expression is: I R ′(x, y)=I R (x+ΔP i , y), where: I L ′(x, y), I R ′(x, y) is the left and right channel images after parallax displacement processing; ΔP iis the optimized parallax parameter; (x, y) is the image pixel coordinate position; in order to achieve high-precision displacement, the bicubic interpolation method can be used to resample the displacement result, and the motion compensation algorithm can be combined to perform dynamic interpolation correction on the fast-moving area, with an overall displacement accuracy of 1 / 16 pixel;

[0096] Dynamic synthesis unit: used to perform dual-channel image fusion processing after completing sub-pixel displacement, and output a 3D endoscopic image stream that complies with the ISO13485 standard with a dynamic delay of ≤2ms; the above unit constructs a three-dimensional image output mechanism based on checkerboard framing, sub-pixel level difference displacement and biological motion decoupling. It not only realizes spatial depth reconstruction under parallax control, but also ensures image clarity and timeliness in the dual dynamic scenes of tissues and instruments, effectively improving the clinical usability and imaging safety of 3D endoscopic operations.

[0097] The dynamic synthesis unit includes:

[0098] Tissue recognition subunit: Inter-frame motion feature extraction is performed on the left and right channel image frames that have completed sub-pixel displacement processing. By comparing the motion vector direction and amplitude of the corresponding pixels, a consistency distribution map is formed. The consistency calculation formula is: Where C(x, y) is the motion consistency of pixel (x, y); is the motion vector of the pixel (x, y) in the left channel image; is the motion vector of the corresponding pixel in the right channel image;

[0099] Region separation subunit: used to divide the consistency distribution map into motion consistency regions (tissue regions) and inconsistency regions (device regions), and generate a spatial mask for each type of region to guide subsequent image fusion strategies;

[0100] Fusion processing subunit: used to perform weighted fusion operation based on consistency weight on the left channel and right channel images in the tissue area to preserve the sense of spatial depth while suppressing noise interference and improving the smoothness and structural continuity of the fused image; the calculation formula of the fused image is:

[0101] Among them I F (x, y) is the fused image value of pixel (x, y); I L ′(x, y) is the grayscale value of the left channel image at pixel (x, y) after displacement; I R ′(x, y) is the grayscale value of the right channel image at pixel (x, y) after displacement; w L (x, y), w R (x, y) are the weight coefficients of the left and right channels calculated according to the degree of motion consistency, satisfying wL (x, y) + w R (x, y) = 1;

[0102] Enhancement control subunit: It is used to perform local sharpening processing on the image within the instrument area. By constructing the Laplace edge enhancement operator, it can enhance the clarity of the instrument boundary, suppress the blurring and smearing caused by rapid displacement, and improve the imaging recognizability of key operating tools. The edge response value of the sharpened image is calculated as follows: Where L(x, y) is the edge response value of the pixel point (x, y); is the Laplacian operator (i.e., the second-order derivative) of the image; I(x, y) is the original pixel value of the fused image or single-channel image; by refining the dynamic synthesis process into four structured processing sub-units and combining multi-scale motion vector fields, spatial region masks and response enhancement algorithms, the system can implement differentiated fusion strategies according to the characteristics of different regions, thereby ensuring the clarity, stability and real-time response accuracy of three-dimensional images in the dynamic interference environment during surgery.

[0103] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0104] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A 3D endoscopic image processing system based on artificial intelligence assistance, characterized in that: It includes an image and object distance acquisition module, an AI parallax dynamic generation module, a physiological parameter adaptation module, a macro optimization module, and a stereo imaging output module; among which: Image and object distance acquisition module: used to obtain the original image through a single optical path focusable lens and synchronously measure real-time object distance data; AI disparity dynamic generation module: used to receive the original image and object distance data, and generate the initial disparity parameters associated with the object distance through the convolutional neural network; Physiological parameter adaptation module: used to receive the physiological parameters collected by the operator's pupil tracker and output personalized disparity parameters based on the initial disparity parameters; Macro Optimization Module: When the object distance data is less than the set threshold, it applies asymmetric compensation to the personalized parallax parameters and outputs the optimized parallax parameters; Stereoscopic imaging output module: Based on the optimized disparity parameters, the original image is subjected to spatiotemporal segmentation, sub-pixel shifting, and channel synthesis, and finally outputs a 3D endoscopic image stream.

2. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 1, characterized in that: The image and object distance acquisition module includes an optical imaging unit, a composite distance measurement unit and a synchronization control unit; wherein: Optical imaging unit: including a focus-adjustable liquid lens and a CMOS image sensor. The focus-adjustable liquid lens is used to respond to focus adjustment instructions to dynamically adjust the focus range. The CMOS image sensor is used to collect single-light-path original images. Composite ranging unit: used to output object distance data in real time by fusing ToF infrared delay ranging and three-wavelength laser speckle distortion analysis; Synchronous control unit: Used to generate frame trigger signals through FPGA, synchronously control the focus switching of the adjustable focus liquid lens, the exposure timing of the CMOS image sensor, and the laser emission cycle of the composite ranging unit, ensuring the spatiotemporal alignment of the original image and the real-time object distance data.

3. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 2, characterized in that: The composite distance measuring unit comprises: The ToF sensor subunit is used to transmit modulated infrared light waves to the target area, receive the reflected echo, and calculate the first object distance data D1 based on the propagation delay; Three-wavelength laser speckle generator subunit: used to synchronously emit visible spectrum speckle and near-infrared speckle patterns, and obtain the second object distance data D2 based on speckle distortion analysis; Object distance calculation subunit: used to fuse the first object distance data D1 and the second object distance data D2, and output the final real-time object distance data D using a weighted average method.

4. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 1, characterized in that: The AI ​​disparity dynamic generation module includes a data fusion unit, a dual-branch convolutional network unit, and a dynamic association unit; wherein: Data fusion unit: used to divide the original image into grid areas, bind the corresponding object distance label to each image block, and construct a space-depth integrated image block input set; Dual-branch convolutional network unit: It is used to extract the spatial texture features of the image and the semantic features of the object distance label in parallel, and fuse the two types of information in the decoding stage to generate a structured initial disparity map; Dynamic association unit: extracts multiple disparity regions based on the gradient distribution of the initial disparity map and object distance data, and generates a disparity reference parameter set associated with the object distance change.

5. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 1, characterized in that: The physiological parameter adaptation module includes a physiological parameter interface unit, a parameter mapping unit and a personalized correction unit; wherein: Physiological parameter interface unit: used to receive pupil distance D in real time via near-field wireless link pupil , Dominant Eye Marker S eye and dynamic focusing frequency f focus Three types of physiological data are synchronously stored in the buffer; Parameter mapping unit: used to scale the initial disparity parameters based on the pupil distance, and perform asymmetric weight adjustment on the left and right channels according to the dominant eye identifier to form a basic personalized parameter set; Personalized correction unit: used to perform motion state discrimination and differentiated correction on the basic personalized parameter set according to the dynamic focusing frequency.

6. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 5, characterized in that: The parameter mapping unit includes: Scaling correction subunit: used to adjust the pupil distance D pupil Scaling and correcting the initial parallax parameter set to generate a basic personalized parameter set; Asymmetric weighted subunit: used to identify S according to eye dominance eye Perform asymmetric weighted adjustment on the left and right channel parameters. Specifically, when S eye = left eye, then the left channel parameter is multiplied by 0.9, and the right channel parameter is multiplied by 1.1; when S eye =right eye, then the left channel parameters are multiplied by 1.1 and the right channel parameters are multiplied by 0.

9.

7. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 6, characterized in that: The personalized correction unit includes: When f focus When the frequency is >2Hz, it indicates that the operator is frequently adjusting focus, and time domain smoothing filtering is applied to the basic personalized parameter set; When f focus When the value is ≤2Hz, it indicates that the operator's focusing state is stable, and the current personalized parallax parameters are kept unchanged.

8. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 1, characterized in that: The macro optimization module includes an object distance monitoring unit and a layered compensation unit; wherein: Object distance monitoring unit: used to continuously receive real-time object distance data output by the image and object distance acquisition module. When the real-time object distance is detected to be less than 50mm, the compensation channel is automatically activated to enter the macro parallax adjustment process. Layered compensation unit: used to perform spatially variable weighted asymmetric compensation on the personalized disparity parameter set in the macro state, specifically including basic compensation and tissue adaptation compensation; The basic compensation is used to apply a uniform compensation amount δ to all parallax areas, and its calculation expression is: δ = 0.23·D 2 , where: δ is the basic compensation amount; D is the real-time object distance data; The tissue adaptation compensation is used to adjust the compensation weight according to the reflectivity characteristics of the intracavitary area. Specifically, if it is a high-reflection area, the compensation weight is 1.2; if it is a low-reflection area, the compensation weight is 0.

8.

9. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 1, characterized in that: The stereoscopic imaging output module includes a time-space segmentation unit, a displacement engine unit and a dynamic synthesis unit; wherein: The spatiotemporal segmentation unit receives the original image and optimized disparity parameters, samples the single-frame original image alternately in a checkerboard pattern, and divides it into left-channel and right-channel subframes. It generates a displacement control matrix based on the optimized disparity parameters and marks the displacement priority of each pixel. Shift engine unit: used to perform differential sub-pixel shift on the left and right channel subframes: specifically, applying positive shift to the left channel image and negative shift to the right channel image; Dynamic synthesis unit: used to perform dual-channel image fusion processing after completing sub-pixel displacement, output 3D endoscopic image stream, and dynamic delay ≤ 2ms.

10. The artificial intelligence-assisted 3D endoscopic image processing system according to claim 1, characterized in that: The dynamic synthesis unit comprises: Tissue recognition subunit: Extract inter-frame motion features from left and right channel image frames that have undergone sub-pixel shift processing, and form a consistency distribution map by comparing the motion vector direction and amplitude of corresponding pixels; Region separation subunit: used to divide the consistency distribution map into motion consistency regions and inconsistency regions, and generate a spatial mask for each type of region; Fusion processing subunit: used for performing a weighted fusion operation based on consistency weight on the left channel and right channel images in the tissue area; Enhancement control subunit: used to perform local sharpening processing on the image within the device area and enhance the clarity of the device boundary by constructing a Laplace edge enhancement operator.

Images