An image back orientation method and device of an endoscope, a storage medium and a terminal

Abstract

The application discloses an image righting method and device of an endoscope, a storage medium and a terminal, and comprises the following steps: acquiring an initial image and a three-axis angular velocity signal of the endoscope; obtaining a posture quaternion by using a quaternion differential equation according to the three-axis angular velocity signal for attitude solution; updating the posture quaternion in real time; interpolating the posture quaternion of adjacent time points to obtain a line-level quaternion and generate a line-level rotation matrix within one frame exposure time; and righting and compensating the initial image to obtain a righted image by using reverse mapping and bilinear interpolation based on the line-level rotation matrix. The application avoids the problems of gimbal lock and functional failure caused by the gimbal lock in Euler angle attitude solution, eliminates the jelly effect by using an interpolation algorithm for line-by-line compensation, avoids the problems of large delay, insufficient real-time performance and difficulty in integration in a limited space caused by traditional software algorithms or external micro control unit post-processing, and better meets the imaging requirements of medical endoscope clinical operation.

Description

Technical Field

[0001] This invention relates to the field of endoscopy technology, and more specifically to an image alignment method, apparatus, storage medium, and terminal for an endoscope. Background Technology

[0002] In the field of medical endoscopy, changes in the angle and posture of the endoscope during clinical operations often lead to tilting, rotation, or jitter in the acquired images, severely affecting doctors' observation and diagnosis of lesions. Therefore, posture calculation and image stabilization are crucial. From the perspective of posture calculation technology, traditional solutions often use Euler angles to represent spatial posture. The core principle is to describe the posture through three successive rotations around a fixed coordinate axis. However, three-dimensional rotation space is a three-dimensional manifold, while Euler angles only use three angles. When the pitch angle θ approaches ±90°, the two rotation axes coincide, causing the system to lose a degree of freedom, resulting in gimbal lock. This makes it impossible to track rapid flipping or rotation of the endoscope, and can also lead to posture calculation interruptions and image alignment failures due to the inability to recognize angle information. Furthermore, the Euler angle algorithm requires multiple trigonometric function calculations, which is difficult to achieve efficiently on hardware platforms, resulting in insufficient real-time performance.

[0003] From the perspective of image stabilization technology, existing technologies mainly rely on software algorithms or external microcontroller units for post-processing. This involves acquiring the rotation angle of a gyroscope and using it as a basis to rotate and acquire images, ultimately displaying the rotated image. However, software processing has significant latency, which cannot meet the real-time operation requirements of endoscopes. Integrating external microcontroller units increases system power consumption, and in confined spaces such as disposable electronic endoscopes, integrating high-performance processors is extremely difficult, making it hard to guarantee image stability and accuracy in clinical applications.

[0004] Therefore, how to overcome the shortcomings of existing technologies using effective methods has become an urgent technical challenge. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by providing an image alignment method, apparatus, storage medium, and terminal for an endoscope.

[0006] The technical solution of the present invention is: an image alignment method for an endoscope, comprising the following steps:

[0007] Acquire initial images and triaxial angular velocity signals from the endoscope;

[0008] The attitude quaternion is obtained by using quaternion differential equations to solve the attitude based on the three-axis angular velocity signal, and the attitude quaternion is updated in real time.

[0009] Within one frame of exposure time, the pose quaternions at adjacent time points are interpolated to obtain row-level quaternions. A row-level rotation matrix is ​​generated based on the row-level quaternions. Based on the row-level rotation matrix, the initial image is corrected and compensated using inverse mapping and bilinear interpolation to obtain a corrected image.

[0010] As an improvement to this embodiment of the invention, the "acquiring the initial image and triaxial angular velocity signal of the endoscope" specifically includes: acquiring the original image stream as the initial image through the image sensor of the endoscope; and acquiring the triaxial angular velocity signal of the endoscope during its movement through the gyroscope integrated with the endoscope.

[0011] As an improvement to this embodiment of the invention, the step of "obtaining attitude quaternions by performing attitude calculations using quaternion differential equations based on the triaxial angular velocity signals, and updating the attitude quaternions in real time" specifically includes: calculating attitude quaternions based on quaternion differential equations... Attitude calculation is performed to obtain attitude quaternions. These attitude quaternions are updated in real time, and normalized within a preset period. Let q be the differential of the attitude quaternion, and q be the current attitude quaternion. This is the quaternion multiplication operator. It is a three-axis angular velocity vector; , where w is the real part, and x, y, and z are the imaginary parts of the attitude quaternion, respectively; ,in, This is a three-axis angular velocity signal.

[0012] As an improvement to this embodiment of the invention, the "normalization processing of the updated attitude quaternions within a preset period" specifically includes: through... Calculate the modulus of the updated attitude quaternion; then divide each component of the attitude quaternion by the modulus to obtain the normalized attitude quaternion.

[0013] As an improvement to this embodiment of the invention, the step of "interpolating the pose quaternions of adjacent time points within one frame of exposure time to obtain row-level quaternions, and generating a row-level rotation matrix based on the row-level quaternions" specifically includes:

[0014] Within one frame of exposure time, acquire pose samples from adjacent moments and the center exposure time of each row of pixels in the initial image. Obtain row interpolation coefficients based on the pose samples and the center exposure time of each row of pixels in the initial image. Then, calculate the angle between adjacent pose quaternions. When the angle is less than a preset angle, obtain row-level quaternions based on linear interpolation and normalization. When the angle is greater than or equal to the preset angle, obtain row-level quaternions based on spherical linear interpolation. Convert the row-level quaternions into a three-dimensional rotation matrix, and then combine it with the intrinsic parameter matrix of the image sensor in the endoscope to generate a row-level rotation matrix.

[0015] As an improvement to this embodiment of the invention, the step of "using inverse mapping and bilinear interpolation to correct and compensate the initial image based on the row-level rotation matrix to obtain a corrected image" specifically includes:

[0016] The inverse mapping relationship is constructed based on the row-level rotation matrix. The floating-point coordinates corresponding to the target pixel of the straightened image in the initial image are calculated. Four integer grid points around the floating-point coordinates are determined and the offset is calculated. The straightened image is obtained by linear interpolation in both the horizontal and vertical directions based on the gray values ​​of the four grid points and the offset.

[0017] As an improvement of this embodiment of the invention, during the process of acquiring the initial image and the three-axis angular velocity signal, the time difference between the sampling time of the three-axis angular velocity signal and the line exposure center time is calculated based on the line exposure center time of the initial image; and the sampling time of the three-axis angular velocity signal is offset compensated according to the time difference.

[0018] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides an image alignment device for an endoscope, comprising the following modules:

[0019] The data acquisition module is used to acquire the initial images and triaxial angular velocity signals of the endoscope;

[0020] The attitude calculation module is used to calculate the attitude quaternion by using quaternion differential equations based on the three-axis angular velocity signals, and to update the attitude quaternion in real time.

[0021] The alignment module is used to interpolate the pose quaternions of adjacent time points within one frame of exposure time to obtain row-level quaternions, and generate a row-level rotation matrix based on the row-level quaternions; based on the row-level rotation matrix, the initial image is aligned and compensated using inverse mapping and bilinear interpolation to obtain an aligned image.

[0022] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a storage medium storing program instructions, which, when executed, implement the endoscope image alignment method as described in any of the preceding claims.

[0023] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides an electronic terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the endoscope image alignment method as described in any of the preceding claims.

[0024] The image alignment method, device, storage medium, and terminal for endoscopes provided in this invention have the following advantages: This invention uses quaternions to represent spatial attitude, effectively avoiding gimbal lock and functional failure caused by it in traditional Euler angle attitude calculation. The attitude calculation is continuous within any angle range and can track the rapid flipping or rotation of the endoscope. Quaternions can be updated efficiently through linear multiplication and addition, without the need for multiple trigonometric function calculations, thus improving the stability and real-time performance of attitude calculation.

[0025] This invention calculates the attitude using gyroscope attitude signals and performs line-by-line dynamic compensation based on the calculation results using an interpolation algorithm. This effectively eliminates the rolling shutter effect caused by line-by-line exposure and avoids the problems of large delays, insufficient real-time performance, and difficulty in integration in confined spaces that exist in traditional software algorithms or external microcontroller post-processing. It can better meet the imaging needs of clinical operations in medical endoscopy. Attached Figure Description

[0026] Figure 1 This is a schematic flowchart of the image alignment method for the endoscope described in this invention;

[0027] Figure 2 This is a schematic diagram of the endoscope image before alignment in an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the endoscope image after alignment in an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the image alignment device of the endoscope described in this invention;

[0030] Figure 5 This is a schematic diagram of the structure of the electronic terminal described in this invention. Detailed Implementation

[0031] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0032] The scope of the embodiments described herein includes the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitations, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0033] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used in this document to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are used only for the convenience of describing this document and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. In the description herein, unless otherwise specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two elements, or direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0034] This invention provides an image alignment method for an endoscope, such as... Figure 1 As shown, it includes the following steps:

[0035] Step 101: Acquire the initial images and triaxial angular velocity signals from the endoscope;

[0036] In practice, the image sensor of the endoscope can be used to acquire the raw image stream as the initial image. This image sensor needs to be matched with the endoscope's optical system to acquire scene information captured in real time by the endoscope's probe and output it as a raw image stream. A gyroscope integrated with the endoscope acquires the three-axis angular velocity signals during endoscope movement. This gyroscope needs to be pre-calibrated with the endoscope's probe to ensure its detection axis is consistent with the endoscope's motion axis, thus acquiring angular velocity data generated along three orthogonal directions when the endoscope undergoes flipping, rotation, or other movements during clinical operations. During the acquisition of the initial image and the three-axis angular velocity signals, the time difference between the sampling time of the three-axis angular velocity signals and the line exposure center time of the initial image is calculated based on this time difference. The sampling time of the three-axis angular velocity signals is then offset compensated based on this time difference. This avoids deviations in attitude calculation and image processing due to asynchronous data acquisition, ensuring that the initial image and the three-axis angular velocity signals reflect the endoscope's imaging and motion states at the same moment.

[0037] Step 102: Based on the triaxial angular velocity signals, attitude quaternions are obtained by performing attitude calculations using quaternion differential equations, and the attitude quaternions are updated in real time; specifically, this includes: based on the quaternion differential equations... Attitude calculation is performed to obtain attitude quaternions. These attitude quaternions are updated in real time, and normalized within a preset period. Let q be the differential of the attitude quaternion, and q be the current attitude quaternion. This is the quaternion multiplication operator. It is a three-axis angular velocity vector; , where w is the real part, and x, y, and z are the imaginary parts of the attitude quaternion, respectively; ,in, This is a three-axis angular velocity signal.

[0038] Here, the phrase "normalizing the updated attitude quaternions within a preset period" specifically includes: through... Calculate the modulus of the updated attitude quaternion; then divide each component of the attitude quaternion by the modulus to obtain the normalized attitude quaternion.

[0039] In practice, attitude calculation and updating can be implemented on a Field Programmable Gate Array (FPGA) hardware platform, fully utilizing its parallel processing capabilities and efficient logic operation characteristics to ensure real-time performance and stability. The FPGA establishes a data connection with the gyroscope through its integrated I2C or SPI communication module, receiving triaxial angular velocity signals in real time and temporarily storing the data in an internal FIFO buffer to avoid data loss or delay. Attitude calculation uses a hardware description language to embed quaternion operation logic within the FPGA. Based on the quaternion differential equation, it performs hardware-level quaternion multiplication operations with the current attitude quaternion and the triaxial angular velocity vector to complete the attitude quaternion differential calculation, thereby achieving real-time updates of the attitude quaternion. Simultaneously, the FPGA can perform normalization operations on the updated attitude quaternion at preset intervals. First, the magnitude of the attitude quaternion is calculated using hardware circuitry. Then, the ratio of each component to the magnitude is output through division logic to obtain the normalized attitude quaternion. The quaternion multiplication strictly follows Hamilton's multiplication rule. The timing synchronization between angular velocity vector processing and quaternion operations is controlled by the FPGA's internal clock signal. It can be understood that this hardware implementation method abandons the complex process in traditional software operations, replacing multiple trigonometric function calculations with linear multiplication and addition logic, which significantly improves the computational efficiency. Moreover, the entire process does not require an external processor, ensuring the continuity and accuracy of attitude calculation.

[0040] Step 103: Interpolate the pose quaternions of adjacent moments within one frame of exposure time to obtain row-level quaternions, and generate a row-level rotation matrix based on the row-level quaternions. Specifically, this includes: acquiring pose samples of adjacent moments and the center exposure time of each row of pixels in the initial image within one frame of exposure time; obtaining row interpolation coefficients based on the pose samples and the center exposure time of each row of pixels in the initial image; then calculating the angle between adjacent pose quaternions; when the angle is less than a preset angle, obtaining row-level quaternions based on linear interpolation and normalization; when the angle is greater than or equal to a preset angle, obtaining row-level quaternions based on spherical linear interpolation; converting the row-level quaternions into a three-dimensional rotation matrix, and then combining it with the intrinsic parameter matrix of the image sensor in the endoscope to generate a row-level rotation matrix.

[0041] In practice, attitude samples from adjacent time points can be... , ,in, Indicates the i-th time. Indicates at time The corresponding attitude quaternion; This represents the (i+1)th time. Indicates at time The corresponding pose quaternion; the center exposure time of the j-th row pixel in the initial image is The interpolation coefficients of the j-th row pixels in the initial image. The included angle between the adjacent attitude quaternions. , ,in, The product is a dot product. When the included angle is less than a preset angle, a row-level quaternion is obtained by linear interpolation and normalization. , where norm( ) indicates the normalization operation. It is a moment The conjugate of the corresponding attitude quaternion is used, where the real part remains unchanged and the imaginary part is inverted. This conjugate is used in operations such as spherical linear interpolation. When the included angle is greater than or equal to a preset included angle, a row-level quaternion is obtained based on spherical linear interpolation. .

[0042] Then, the three-dimensional rotation matrix can be obtained using standard right-handed quaternions. Then, combined with the intrinsic parameter matrix of the image sensor in the endoscope Generate row-level rotation matrix ,in, Intrinsic parameter matrix The inverse matrix.

[0043] Based on the row-level rotation matrix, the initial image is corrected and compensated using inverse mapping and bilinear interpolation to obtain a corrected image. Specifically, this includes: constructing an inverse mapping relationship based on the row-level rotation matrix; calculating the floating-point coordinates corresponding to the target pixel of the corrected image in the initial image; determining four integer grid points around the floating-point coordinates and calculating the offset; and performing linear interpolation in both the horizontal and vertical directions based on the gray values ​​of the four grid points and the offset to obtain the corrected image.

[0044] In practice, the inverse mapping relationship ,in, This is a scaling factor used to convert homogeneous coordinates to non-homogeneous coordinates. v are the floating-point coordinate components in the initial image corresponding to the target pixel in the corrected image; , These are the coordinate components of the target pixel in the corrected image; It is a row-level rotation matrix The inverse matrix is ​​used to construct the inverse mapping relationship from the corrected image to the initial image, realizing the reverse lookup of pixel coordinates; homogeneous coordinates and These represent the homogeneous coordinates of pixels in the initial image and the corrected image, respectively. Through this matrix operation, the corresponding floating-point position of each pixel in the corrected image in the initial image can be determined, providing coordinate data for the bilinear interpolation.

[0045] Table 1. Four integer grid points surrounding the floating-point coordinates and related information.

[0046]

[0047] The four integer grid points surrounding the floating-point coordinates and their corresponding coordinates and grayscale values ​​are shown in Table 1. The offset... ,in, This indicates the horizontal offset. Indicates the offset in the vertical direction; , It is the integer grid point around the top left corner of the floating-point coordinate (u,v) coordinates ,in This indicates rounding down. Then, based on the grayscale values ​​and offsets of the four grid points, linear interpolation is performed in both the horizontal and vertical directions to obtain a corrected image. Specifically, the horizontal interpolation is as follows: ; The interpolation in the vertical direction specifically refers to: After the merger Here, in the horizontal direction, based on the offset... Intermediate results were obtained by linear interpolation of the gray values ​​of adjacent grid points. and Then, based on the offset in the vertical direction right and Linear interpolation is used to obtain the grayscale values ​​of the pixels in the corrected image, thereby achieving image correction compensation.

[0048] Here, the initial image is as follows: Figure 2 As shown, the image exhibits viewpoint shift and geometric distortion due to changes in the spyscope's posture; the corrected image is as follows. Figure 3 As shown, after the image is corrected by the image correction method described in this invention, the geometric structure of the image is restored to normal and the viewing angle is more regular. It can be understood that this invention can effectively eliminate the image distortion caused by the movement of the endoscope through row-level pose compensation, so that the image remains geometrically consistent and the accuracy of image correction is guaranteed. It solves the problems of image distortion and viewing angle shift caused by dynamic changes in the endoscope pose. Moreover, row-level compensation can effectively eliminate the rolling shutter effect caused by progressive exposure.

[0049] By acquiring two sets of chessboard images with and without row-level compensation enabled, the geometric consistency of the grid was compared, meeting the requirements of reprojection error ≤ 0.5px, straightness change ≤ 0.3px, and angle deviation ≤ 0.2°. The consistency between attitude change and gyroscope measurements was verified by simultaneously acquiring gyroscope and image data through a low-speed rotating motor module, and the image center drift after compensation was verified to be ≤ 0.5px, with the row-level update frequency and image sensor error ≤ 5µs. The row number correspondence was recorded using an LED row scanning light source, verifying a row timing error ≤ 2µs. Simultaneously, static video was recorded using a fixed module, and brightness variance was calculated, verifying that attitude calculation jitter did not introduce image flicker, with an average brightness variance ≤ 1gray. It can be understood that the image straightening method described in this invention meets the qualified standards in terms of geometric stability, straightness, row timing synchronization, row attitude phase error, static brightness variance, and low-speed rotation drift. This avoids the problems of large delays, insufficient real-time performance, and difficulty in integration in confined spaces inherent in traditional software algorithms or external microcontroller post-processing, and better meets the imaging needs of clinical medical endoscopy operations.

[0050] The present invention also provides an image alignment device for an endoscope, such as... Figure 4 As shown, it includes the following modules:

[0051] The data acquisition module 201 is used to acquire the initial image and triaxial angular velocity signal of the endoscope;

[0052] The attitude calculation module 202 is used to calculate the attitude quaternion by using quaternion differential equations based on the triaxial angular velocity signal, and to update the attitude quaternion in real time.

[0053] The alignment module 203 is used to interpolate the pose quaternions of adjacent time points within one frame of exposure time to obtain row-level quaternions, generate a row-level rotation matrix based on the row-level quaternions, and perform alignment and compensation on the initial image based on the row-level rotation matrix using inverse mapping and bilinear interpolation to obtain an aligned image.

[0054] The present invention also provides a storage medium storing program instructions that, when executed, implement the endoscope image alignment method as described in any of the preceding claims.

[0055] The present invention also provides an electronic terminal, such as Figure 5 As shown, it includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the image repositioning method of the endoscope as described in any of the preceding claims.

[0056] This invention can be an apparatus, method, and / or computer program product. A computer program product may include a readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0057] Storage media can be tangible devices that hold and store instructions for use by instruction execution devices. Storage media can include, but are not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0058] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0059] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image alignment method for an endoscope, characterized in that, Includes the following steps: Acquire initial images and triaxial angular velocity signals from the endoscope; Attitude quaternions are obtained by performing attitude calculations using quaternion differential equations based on the triaxial angular velocity signals. These attitude quaternions are then updated in real time, specifically including: [following the quaternion differential equations...] Attitude calculation is performed to obtain attitude quaternions. These attitude quaternions are updated in real time, and normalized within a preset period. Let q be the differential of the attitude quaternion, and q be the current attitude quaternion. This is the quaternion multiplication operator. It is a three-axis angular velocity vector; Within one exposure frame, row-level quaternions are obtained by interpolating the pose quaternions of adjacent time points. A row-level rotation matrix is ​​then generated based on these row-level quaternions. Specifically, this involves: acquiring pose samples of adjacent time points and the center exposure time of each row of pixels in the initial image within one exposure frame; obtaining row interpolation coefficients based on the pose samples and the center exposure time of each row of pixels in the initial image; then calculating the angle between adjacent pose quaternions; if the angle is less than a preset angle, row-level quaternions are obtained through linear interpolation and normalization; if the angle is greater than or equal to the preset angle, row-level quaternions are obtained through spherical linear interpolation. A row-level quaternion is converted into a three-dimensional rotation matrix, and then combined with the intrinsic parameter matrix of the image sensor in the endoscope to generate a row-level rotation matrix. Based on the row-level rotation matrix, the initial image is corrected and compensated using inverse mapping and bilinear interpolation to obtain a corrected image. Specifically, this includes: constructing an inverse mapping relationship based on the row-level rotation matrix, calculating the floating-point coordinates of the target pixel in the initial image corresponding to the corrected image; determining four integer grid points around the floating-point coordinates and calculating the offset; and performing linear interpolation in both the horizontal and vertical directions based on the gray values ​​of the four grid points and the offset to obtain the corrected image.

2. The endoscope image alignment method according to claim 1, characterized in that, The acquisition of the initial image and triaxial angular velocity signal of the endoscope specifically includes: acquiring the original image stream as the initial image through the image sensor of the endoscope; and acquiring the triaxial angular velocity signal of the endoscope during its movement through the gyroscope integrated with the endoscope.

3. The endoscope image alignment method according to claim 1, characterized in that, , where w is the real part, and x, y, and z are the imaginary parts of the attitude quaternion, respectively; ,in, This is a three-axis angular velocity signal.

4. The image alignment method for an endoscope according to claim 1, characterized in that, The normalization process for the updated attitude quaternions within a preset period specifically includes: through... Calculate the modulus of the updated attitude quaternion; then divide each component of the attitude quaternion by the modulus to obtain the normalized attitude quaternion.

5. The image alignment method for an endoscope according to claim 1, characterized in that, During the acquisition of the initial image and the three-axis angular velocity signal, the time difference between the sampling time of the three-axis angular velocity signal and the line exposure center time of the initial image is calculated based on the line exposure center time of the initial image; and the sampling time of the three-axis angular velocity signal is offset compensated according to the time difference.

6. An image alignment device for an endoscope, characterized in that, Includes the following modules: The data acquisition module is used to acquire the initial images and triaxial angular velocity signals of the endoscope; The attitude calculation module is used to calculate the attitude quaternions based on the triaxial angular velocity signals using quaternion differential equations, and to update the attitude quaternions in real time. Specifically, this includes: calculating the attitude quaternions based on the quaternion differential equations... Attitude calculation is performed to obtain attitude quaternions. These attitude quaternions are updated in real time, and normalized within a preset period. Let q be the differential of the attitude quaternion, and q be the current attitude quaternion. This is the quaternion multiplication operator. It is a three-axis angular velocity vector; The alignment module is used to interpolate the pose quaternions of adjacent time points within one exposure frame to obtain row-level quaternions, and generate a row-level rotation matrix based on the row-level quaternions. Specifically, it includes: acquiring pose samples of adjacent time points and the center exposure time of each row of pixels in the initial image within one exposure frame; obtaining row interpolation coefficients based on the pose samples and the center exposure time of each row of pixels in the initial image; then calculating the angle between adjacent pose quaternions; when the angle is less than a preset angle, obtaining the row-level quaternion based on linear interpolation and normalization; when the angle is greater than or equal to the preset angle, obtaining the row-level quaternion based on spherical linear interpolation. The process involves: converting row-level quaternions into a three-dimensional rotation matrix, then combining this matrix with the intrinsic parameter matrix of the image sensor in the endoscope to generate a row-level rotation matrix; and using inverse mapping and bilinear interpolation to correct and compensate for the initial image based on the row-level rotation matrix to obtain a corrected image. Specifically, this includes: constructing an inverse mapping relationship based on the row-level rotation matrix; calculating the floating-point coordinates corresponding to the target pixel of the corrected image in the initial image; determining four integer grid points around the floating-point coordinates and calculating their offsets; and performing linear interpolation in both the horizontal and vertical directions based on the gray values ​​of the four grid points and their offsets to obtain the corrected image.

7. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they implement the endoscope image alignment method as described in any one of claims 1 to 5.

8. An electronic terminal, characterized in that, It includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the image alignment method of the endoscope as described in any one of claims 1 to 5.

Images