Automatic rotating and aligning method for endoscope image

Abstract

The invention discloses an automatic rotating and aligning method for an endoscope image, relates to the technical field of medical instrument image processing, and solves the problems that the existing endoscope image inclines along with rotation of equipment and is insufficient in definition. The method comprises the following steps: after the system is started, the FPGA and the ISP respectively initialize the endoscope module and the gyroscope module; the image sensor acquires images and transmits the images to the FPGA through the DVP, and the gyroscope acquires original data and transmits the original data to the ISP through the I2C; the ISP calculates a rotation angle and transmits the rotation angle to the FPGA; the FPGA segments the image into 8 * 8 sub-modules, coordinates are calculated to realize alignment, parameters are adjusted to optimize definition, and then the parameters are transmitted to the ISP; and after being processed by the ISP, the image is sent to a display through an HDMI to display an upright and clear image. According to the invention, attitude data is collected in real time through the gyroscope and is rapidly processed through the FPGA, real-time image alignment is realized, the image definition is obviously improved after optimization, a stable mechanism for keeping a current angle during data interruption is provided, all modules are high in communication compatibility through a standard interface, and the medical diagnosis efficiency and accuracy can be improved.

Description

Technical Field

[0001] This invention belongs to the field of medical device image processing technology, and in particular relates to a method for automatically rotating and straightening endoscopic images. Background Technology

[0002] An endoscope is a commonly used medical diagnostic device. It involves inserting a portion (the endoscope) with a built-in image sensor into the human body to acquire images of internal tissues and transmit them to a monitor for observation and diagnosis by doctors. The workflow of a current endoscopic image processing system is as follows: after power-on, the endoscope is initialized; after acquiring the image signal, it is processed by an ISP (exposure, gain adjustment, white balance, etc.) and then transmitted to a monitor via an HDMI interface.

[0003] However, existing technologies have obvious drawbacks: the mirror rotates at an uncertain angle during clinical use, causing the image displayed on the monitor to rotate accordingly. Doctors need to spend extra time adjusting the observation angle, which not only affects diagnostic efficiency but may also increase surgical risks due to image misjudgment. At the same time, existing systems do not use a dedicated image rotation processing module and only perform basic image processing through ISP, resulting in insufficient image clarity and further affecting diagnostic accuracy.

[0004] Therefore, a method for automatically rotating and straightening endoscopic images is proposed. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an automatic rotation and alignment system and method for endoscopic images, enabling real-time image alignment and clarity optimization, solving the problems of insufficient image clarity due to endoscope rotation, and improving the efficiency and accuracy of medical diagnosis.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: This invention relates to a method for automatically rotating and aligning endoscopic images, comprising the following steps: Step S1. The system is powered on, the FPGA initializes the endoscope, and the ISP initializes the gyroscope module; Step S2. The endoscope's image sensor acquires image signals and transmits them to the FPGA via the DVP interface; Step S3. The accelerometer and angular velocity sensors of the gyroscope module collect raw data and transmit it to the ISP via the I2C bus; Step S4. The ISP performs attitude calculations on the raw data to calculate the rotation angle of the endoscope, and transmits it to the FPGA via the UART interface; Step S5. The FPGA receives rotation angle data and image signal, executes image rotation correction algorithm and sharpness optimization algorithm, and transmits the processed image signal to ISP through DVP interface; Step S6. The ISP performs exposure, gain adjustment and white balance processing on the received image signal, and transmits it to the monitor via the HDMI interface. The monitor displays an upright image.

[0007] In one embodiment, the attitude calculation in step S4 includes the following sub-steps: Step S41. Normalize the raw data collected by the accelerometer and calculate the components of the acceleration vector on the X, Y, and Z axes; Step S42. Integrate the raw data collected by the angular velocity sensor to obtain the rate of change of angular velocity; Step S43. Read the three-axis quaternion data q0, q1, q2, and q3 from the gyroscope module, and calculate the pitch, roll, and yaw angles using formulas to determine the rotation angle of the endoscope; the formulas are: pitch=asin(-2×q1×q3+2×q0×q2)×57.3; Roll=atan2(2×q2×q3+2×q0×q1,-2×q1×q1-2×q2×q2+1)×57.3; yaw=atan2(2×(q1×q2+q0×q3), q0×q0+q1×q1-q2×q2-q3×q3)×57.3.

[0008] In one embodiment, the attitude calculation in step S4 includes fusing acceleration and angular velocity data using an adaptive extended Kalman filter or complementary filter algorithm to update the attitude quaternion and correct the cumulative error of the gyroscope.

[0009] Specifically, the quaternion update and error correction include the following sub-steps: Data preprocessing: The raw data collected by the accelerometer and angular velocity sensor are normalized to convert the acceleration vector and angular velocity vector into unit vectors.

[0010] norm = invSqrt(ax×ax + ay×ay + az×az); ax = ax × norm; ay = ay × norm; az = az × norm; (If magnetometer data is used, perform the same processing.) Calculate the gravity vector estimate: Based on the quaternions q0, q1, q2, q3 at the current moment, calculate the estimated gravity vector values ​​[vx, vy, vz] in the volume coordinate system.

[0011] The formula for calculating the estimated gravity vector is: vx = 2×(q1×q3 - q0×q2); vy = 2×(q0×q1 + q2×q3); vz = q0×q0 - q1×q1 - q2×q2 + q3×q3; Calculate the vector error: Perform a cross product operation between the gravity vector [ax, ay, az] measured by the accelerometer and the gravity vector estimate [vx, vy, vz] obtained in step 2 to obtain the attitude error vector [ex, ey, ez].

[0012] The formula for calculating the error vector is: ex = (ay×vz - az×vy); ey = (az×vx - ax×vz); ez = (ax×vy - ay×vx); Error integration and gyroscope data correction: The attitude error vector is integrated and combined with the proportional term to form a PID correction value, which is used to correct the original angular velocity data [gx, gy, gz] read by the angular velocity sensor.

[0013] The correction process is achieved through the following formula: exInt = exInt + ex × Ki × halfT; eyInt = eyInt + ey × Ki × halfT; ezInt = ezInt + ez × Ki × halfT; gx = gx + Kp×ex + exInt; gy = gy + Kp×ey + eyInt; gz = gz + Kp×ez + ezInt; Where Kp and Ki are the proportional and integral coefficients that control the convergence speed of the filter, and halfT is half of the sampling period.

[0014] Update quaternions: Use the corrected angular velocity data [gx, gy, gz] to update the quaternions using the first-order Runge-Kutta method.

[0015] The update steps for the quaternion differential equation are as follows: q0 = q0 + (-q1×gx - q2×gy - q3×gz) × halfT; q1 = q1 + (q0×gx + q2×gz - q3×gy) × halfT; q2 = q2 + (q0×gy - q1×gz + q3×gx) × halfT; q3 = q3 + (q0×gz + q1×gy - q2×gx) × halfT; Quaternion normalization: Normalize the updated quaternion to maintain its unit quaternion state.

[0016] norm = invSqrt(q0×q0 + q1×q1 + q2×q2 + q3×q3); q0 = q0 × norm; q1 = q1 × norm; q2 = q2 × norm; q3 = q3 × norm; Through the above iterative process, the system can output high-precision, drift-free attitude quaternions in real time, providing accurate rotation angles for subsequent image rotation and alignment.

[0017] In one embodiment, the image rotation correction algorithm in step S5 includes the following sub-steps: Step S51. The FPGA stores the received image signal into DDR3 memory and divides the image into 8×8 sub-modules; Step S52. Using the center of each submodule as the rotation center, calculate the rotated coordinates based on the received rotation angle. The coordinate calculation equation is: x_cor=xc-w_src_mid; y_cor=h_src_mid-yc; xc_store=double (cosax_cor-sinay_cor); yc_store=double(sinax_cor+cosay_cor); xc_store=w_src_mid+xc_store; yc_store=h_src_mid-yc_store; xs=xc_store-w_mid; ys=yc_store-w_mid; Where xc and yc are the original center coordinates of the submodule, w_src_mid and h_src_mid are half the width and height of the original image, cosa and sina are the cosine and sine values ​​of the rotation angle, and w_mid is half the width and height of the submodule; Step S53. Map the coordinates of the top left corner of the rotated submodule onto a 400x400 image to obtain the complete image after rotation.

[0018] In one embodiment, the coordinate mapping calculation uses the following formula: Calculate the first coordinates (x_cor, y_cor) of the sub-block center relative to the image center: x_cor=xc-w_src_mid; y_cor=h_src_mid-yc; Calculate the second coordinates (xc_store, yc_store) of the center of the rotated sub-block: xc_store=double(cosa×x_cor-sina×y_cor); yc_store=double(sina×x_cor+cosa×y_cor); Transform the second coordinate system back into the image coordinate system: xc_store=w_src_mid+xc_store; yc_store=h_src_mid-yc_store; Where (xc, yc) are the original coordinates of the sub-block center, w_src_mid and h_src_mid are the width and height coordinates of the image center, and cosa and sina are the cosine and sine values ​​of the rotation angle.

[0019] In one embodiment, the sharpness optimization algorithm described in step S5 is processed in real time by the FPGA based on the brightness statistics of the image. By adjusting the gain, sharpness and noise reduction parameters in a coordinated manner, the visual sharpness of the image after correction is improved.

[0020] In one embodiment, when the gyroscope module data is interrupted or the I2C communication is interrupted, the system defaults to keeping the current image angle unchanged until data transmission resumes normally.

[0021] An automatic rotation and alignment system for endoscopic images to implement the above-described method includes an endoscope, a gyroscope module, an FPGA, an ISP, and a display. The endoscope has a built-in image sensor for acquiring images of the inside of the human body and outputting image signals; The gyroscope module includes an accelerometer and an angular velocity sensor, used to collect acceleration and angular velocity data of the endoscope; The FPGA is connected to the endoscope, gyroscope module and ISP respectively, and is used to initialize the endoscope, receive image signals output by the image sensor, acquire attitude data of the gyroscope module, and execute image rotation correction algorithm and image sharpness optimization algorithm. The ISP is connected to the FPGA, gyroscope module and display respectively. It is used to initialize the gyroscope module, receive the raw data of the gyroscope module and perform attitude calculation, receive the image signal processed by the FPGA and perform exposure, gain adjustment and white balance processing, and transmit the processed image signal to the display through the HDMI interface. The display is used to receive image signals output by the ISP and display upright images.

[0022] In one embodiment, the gyroscope module uses an ICM-42605 chip and transmits data with the ISP via an I2C bus; the accelerometer is configured with a range of ±4G and an output rate of 100Hz; the angular velocity sensor is configured with a range of ±1000dps and an output rate of 100Hz.

[0023] In one embodiment, the FPGA uses an XC7A35T-FGG484 chip, which transmits rotation angle data to the ISP via a UART interface and image data via a DVP interface; the UART interface has a baud rate of 115200, no parity bit, 8 data bits, and 1 stop bit.

[0024] In one embodiment, the image sensor uses an OV426 chip, outputs an image signal with a resolution of 400x400, a frame rate of 30fps, a pixel clock of 8MHz, a data bit width of 10 bits, and a data format of GBRG10BIT.

[0025] The present invention has the following beneficial effects: 1. Real-time image alignment: The gyroscope module accurately collects endoscope posture data, and combined with the FPGA's fast processing capabilities, it realizes real-time image rotation and alignment, ensuring that the monitor always displays an upright image, reducing the doctor's observation and judgment time and lowering surgical risks.

[0026] 2. Improved image clarity: The FPGA specifically executes an image clarity optimization algorithm, adjusting gain, sharpness, and noise reduction parameters based on brightness data. Compared to traditional solutions that rely solely on ISP processing, image clarity is significantly improved.

[0027] 3. High stability: It has an abnormal data interruption handling mechanism and low FPGA processing latency, which can still stably achieve image straightening when the endoscope rotates rapidly, making it suitable for complex clinical use scenarios.

[0028] 4. Good compatibility: Each module communicates through standard interfaces (I2C, UART, DVP, HDMI), which can be adapted to existing mainstream endoscope hardware platforms and is easy to promote and apply.

[0029] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a structural block diagram of the automatic rotation and alignment system for endoscope images in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the image processing workflow in an embodiment of the present invention; Figure 3 This is a schematic diagram of the vector relationship of the acceleration sensor in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the workflow of endoscopic image processing in existing technologies. Figure 5 This is a flowchart illustrating the image processing workflow of an alternative to the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] In the description of this invention, it should be understood that the terms "upper," "middle," "outer," "inner," etc., which indicate orientation or positional relationship, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting this invention.

[0034] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0035] Example 1

[0036] This embodiment provides an automatic rotation and alignment system for endoscopic images, including an endoscope, a gyroscope module, an FPGA, an ISP, and a display. The connection relationships and functions of each module are as follows: Endoscope: Built-in OV426 image sensor, which acquires images of the human body and outputs DVP format image signals with an image resolution of 400x400, a frame rate of 30fps, a pixel clock of 8MHz, a data bit width of 10 bits, and a data format of GBRG10BIT.

[0037] Gyroscope module: It adopts the ICM-42605 chip, which has built-in accelerometer and angular velocity sensor and communicates with ISP via I2C bus; the accelerometer has a range of ±4G and an output rate of 100Hz, and the angular velocity sensor has a range of ±1000dps and an output rate of 100Hz, acquiring raw acceleration and angular velocity data of the endoscope.

[0038] FPGA: Using the XC7A35T-FGG484 chip, it connects with the endoscope, gyroscope module, and ISP; it is responsible for endoscope initialization, receiving image signals, obtaining rotation angle data transmitted by the ISP through the UART interface, executing image rotation correction algorithm and sharpness optimization algorithm, and transmitting the processed image signal to the ISP through the DVP interface.

[0039] ISP: Uses the RV1108 chip and connects to the FPGA, gyroscope module and display; responsible for initializing the gyroscope module, receiving raw data and performing attitude calculations, receiving the image signal processed by the FPGA and performing exposure, gain adjustment and white balance processing, and transmitting the upright image to the display via the HDMI interface.

[0040] Display: Receives image signals output from the ISP and displays upright, clear endoscopic images.

[0041] An automatic rotation correction method for endoscopic images, implemented based on the above system, comprises the following steps: Step S1. The system is powered on and started. The FPGA initializes the endoscope and completes the parameter configuration of the image sensor. At the same time, the ISP initializes the gyroscope module and configures the range, output rate and working mode of the accelerometer and angular velocity sensor.

[0042] Step S2. The endoscope's image sensor acquires images of the tissue inside the body and transmits the image signals to the FPGA via the DVP interface.

[0043] Step S3. The accelerometer and angular velocity sensors of the gyroscope module acquire raw data in real time and transmit it to the ISP via the I2C bus.

[0044] Step S4. The ISP performs attitude calculations on the received raw data: first, it normalizes the acceleration data and calculates the acceleration vector components; it integrates the angular velocity data to obtain the rate of change; it reads the three-axis quaternion data, calculates the pitch angle, roll angle and yaw angle using preset formulas, determines the rotation angle of the endoscope, and then transmits the rotation angle data to the FPGA via the UART interface.

[0045] Step S5. The FPGA receives rotation angle data and image signal, first stores the image signal in DDR3 memory, and divides it into 8×8 sub-modules; with the center of the sub-module as the rotation center, it calculates the coordinates after rotation based on the rotation angle and maps them to a 400x400 image to complete image straightening; at the same time, it adjusts the gain, sharpness and 2D noise reduction parameters based on the image brightness data to optimize image clarity, and finally transmits the processed image signal to the ISP through the DVP interface.

[0046] Step S6. The ISP performs exposure, gain adjustment and white balance processing on the received image signal, and transmits it to the monitor through the HDMI interface. The monitor displays an upright and clear image.

[0047] In addition, the system also has an exception handling mechanism: when the gyroscope module data is interrupted or the I2C communication is interrupted, the current image angle is kept unchanged by default until the data transmission is restored to normal; due to the small image resolution and low data processing volume, the FPGA processing delay can be controlled within 100ms, and even if the endoscope rotates rapidly, it will not affect the real-time image alignment.

[0048] Example 2

[0049] An automatic rotation and alignment system for endoscopic images includes an endoscope, an ICM-42605 gyroscope module, an XC7A35T-FGG484 FPGA, an RV1108 ISP, and a display.

[0050] The endoscope has a built-in OV426 image sensor, and the output image signal parameters are: resolution 400x400, frame rate 30fps, pixel clock 8MHz, data bit width 10 bits, data format GBRG 10BIT, and it is connected to the FPGA through the DVP interface.

[0051] The gyroscope module connects to the ISP via the I2C bus, and its initial configuration is as follows: Accelerometer: Range ±4G, output rate 100Hz, configured via register Icm42605_ACCEL_CONFIG0 (AFS_4G<<5 | AODR_100Hz).

[0052] Angular velocity sensor: range ±1000dps, output rate 100Hz, configured via register Icm42605_GYRO_CONFIG0 (GFS_1000DPS<<5 | GODR_100Hz).

[0053] Power Management: Temperature measurement is enabled via register Icm42605_PWR_MGMT0. GYRO_MODE and ACCEL_MODE are set to low noise mode. After configuration, there is a 1ms delay before proceeding with subsequent operations.

[0054] The FPGA and ISP transmit rotation angle data (115200 baud rate, no parity, 8 data bits, 1 stop bit) via UART interface, and image data via DVP interface; the FPGA has built-in DDR3 memory (MT41K128M16JT) to store image data and execute processing algorithms.

[0055] The ISP connects to the monitor via an HDMI interface, receives the image signal processed by the FPGA, and outputs it to the monitor after exposure, gain adjustment, and white balance processing.

[0056] Example 3

[0057] An automatic rotation correction method for endoscopic images, based on the system of Embodiment 1, includes the following specific steps: Step S1: The system is powered on. The FPGA executes the endoscope initialization program to configure the output parameters of the OV426 image sensor; the ISP executes the gyroscope initialization program to complete the configuration of the range, output rate and working mode of the ICM-42605.

[0058] Step S2: The OV426 image sensor of the endoscope acquires images of tissues inside the body and transmits the image signals to the FPGA through the DVP interface. The FPGA stores the image signals in DDR3 memory.

[0059] In step S3, the accelerometer and angular velocity sensor of the ICM-42605 acquire raw data and transmit it to the ISP via the I2C bus.

[0060] Step S4: The ISP calls the IMU_AHRSupdate function to perform attitude calculations. After normalizing the acceleration data, three angular components, Axr, Ayr, and Azr, are calculated.

[0061] The rate of change, RateAxz, is calculated by integrating the angular velocity data.

[0062] Read the triaxial quaternion data q0, q1, q2, and q3, and calculate the pitch, roll, and yaw angles using formulas to determine the endoscope rotation angle.

[0063] The rotation angle data is transmitted to the FPGA via the UART interface.

[0064] Step S5: After receiving the rotation angle data, the FPGA performs image processing. The image in DDR3 memory is divided into 8×8 sub-modules.

[0065] The rotation coordinates of each submodule are calculated based on the rotation angle, and the image is straightened through coordinate mapping.

[0066] Calculate image brightness data, adjust gain, sharpness, and 2D noise reduction parameters to optimize image clarity.

[0067] The processed image signal is transmitted to the ISP via the DVP interface.

[0068] Step S6: The ISP performs exposure, gain adjustment and white balance processing on the image signal, and transmits it to the monitor through the HDMI interface. The monitor displays an upright and clear image.

[0069] When the gyroscope module data is interrupted or the I2C communication is interrupted, the system maintains the current image angle; after the data is recovered, the image alignment function is automatically restored.

[0070] Alternative solutions description This invention also provides two alternative solutions, as follows: Alternative Solution 1: Without using an FPGA, image straightening is achieved through the OpenCV algorithm. After system power-on, the endoscope initializes and outputs image signals. The ISP initializes the gyroscope and acquires attitude data, performs attitude calculations to obtain the rotation angle, and uses the OpenCV algorithm to rotate and straighten the image. After exposure and gain adjustment, the image is output to the display. This solution is lower in cost, but the OpenCV algorithm requires more hardware resources, has a longer computation time, and poorer real-time performance.

[0071] Alternative Solution 2: After calculating the rotation angle, the ISP uses an OpenCV algorithm to correct the image at the display layer. The workflow is similar to Alternative Solution 1, except that the image correction processing only applies to the display layer and does not optimize the original image data. This solution also suffers from low computational efficiency and insufficient image clarity, resulting in poorer clinical performance compared to the FPGA processing solution of this invention.

[0072] In the description of this specification, references to terms such as "an embodiment," "example," and "specific example" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0073] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for automatically rotating and straightening an endoscopic image, characterized in that, Includes the following steps: Step S1. The system is powered on, the FPGA initializes the endoscope, and the ISP initializes the gyroscope module; Step S2. The endoscope's image sensor acquires image signals and transmits them to the FPGA via the DVP interface; Step S3. The accelerometer and angular velocity sensors of the gyroscope module collect raw data and transmit it to the ISP via the I2C bus; Step S4. The ISP performs attitude calculations on the raw data to calculate the rotation angle of the endoscope, and transmits it to the FPGA via the UART interface; Step S5. The FPGA receives rotation angle data and image signal, executes image rotation correction algorithm and sharpness optimization algorithm, and transmits the processed image signal to ISP through DVP interface; Step S6. The ISP performs exposure, gain adjustment and white balance processing on the received image signal, and transmits it to the monitor via the HDMI interface. The monitor displays an upright image.

2. The method for automatic rotation and alignment of an endoscopic image according to claim 1, characterized in that, The attitude calculation in step S4 includes the following sub-steps: Step S41. Normalize the raw data collected by the accelerometer and calculate the components of the acceleration vector on the X, Y, and Z axes; Step S42. Integrate the raw data collected by the angular velocity sensor to obtain the rate of change of angular velocity; Step S43. Read the three-axis quaternion data q0, q1, q2, and q3 from the gyroscope module, and calculate the pitch, roll, and yaw angles using formulas to determine the rotation angle of the endoscope; the formulas are: pitch=asin(-2×q1×q3+2×q0×q2)×57.3; Roll=atan2(2×q2×q3+2×q0×q1,-2×q1×q1-2×q2×q2+1)×57.3; yaw=atan2(2×(q1×q2+q0×q3), q0×q0+q1×q1-q2×q2-q3×q3)×57.

3.

3. The method for automatically rotating and straightening an endoscopic image according to claim 1, characterized in that, The image rotation straightening algorithm described in step S5 includes the following sub-steps: Step S51. The FPGA stores the received image signal into DDR3 memory and divides the image into 8×8 sub-modules; Step S52. Using the center of each submodule as the rotation center, calculate the rotated coordinates based on the received rotation angle. The coordinate calculation equation is: x_cor=xc-w_src_mid; y_cor=h_src_mid-yc; xc_store=double (cosax_cor-sinay_cor); yc_store=double(sinax_cor+cosay_cor); xc_store=w_src_mid+xc_store; yc_store=h_src_mid-yc_store; xs=xc_store-w_mid; ys=yc_store-w_mid; Where xc and yc are the original center coordinates of the submodule, w_src_mid and h_src_mid are half the width and height of the original image, cosa and sina are the cosine and sine values ​​of the rotation angle, and w_mid is half the width and height of the submodule; Step S53. Map the coordinates of the top left corner of the rotated submodule onto a 400x400 image to obtain the complete image after rotation.

4. The method for automatically rotating and straightening an endoscopic image according to claim 3, characterized in that, The coordinate mapping calculation uses the following formula: Calculate the first coordinates (x_cor, y_cor) of the sub-block center relative to the image center: x_cor=xc-w_src_mid; y_cor=h_src_mid-yc; Calculate the second coordinates (xc_store, yc_store) of the center of the rotated sub-block: xc_store=double(cosa×x_cor-sina×y_cor); yc_store=double(sina×x_cor+cosa×y_cor); Transform the second coordinate system back into the image coordinate system: xc_store=w_src_mid+xc_store; yc_store=h_src_mid-yc_store; Where (xc, yc) are the original coordinates of the sub-block center, w_src_mid and h_src_mid are the width and height coordinates of the image center, and cosa and sina are the cosine and sine values ​​of the rotation angle.

5. The method for automatic rotation and alignment of an endoscopic image according to claim 1, characterized in that, The sharpness optimization algorithm in step S5 includes: the FPGA adjusts the image gain, sharpness parameters and 2D noise reduction parameters based on the image brightness data to optimize the image display effect.

6. The method for automatically rotating and straightening an endoscopic image according to claim 1, characterized in that, When the gyroscope module data is interrupted or the I2C communication is interrupted, the system will maintain the current image angle by default until data transmission is restored.

7. An automatic rotation and alignment system for an endoscope image used to implement the method of any one of claims 1-6, characterized in that, Includes endoscopes, gyroscope modules, FPGAs, ISPs, and displays; The endoscope has a built-in image sensor for acquiring images of the inside of the human body and outputting image signals; The gyroscope module includes an accelerometer and an angular velocity sensor, used to collect acceleration and angular velocity data of the endoscope; The FPGA is connected to the endoscope, gyroscope module and ISP respectively, and is used to initialize the endoscope, receive image signals output by the image sensor, acquire attitude data of the gyroscope module, and execute image rotation correction algorithm and image sharpness optimization algorithm. The ISP is connected to the FPGA, gyroscope module and display respectively. It is used to initialize the gyroscope module, receive the raw data of the gyroscope module and perform attitude calculation, receive the image signal processed by the FPGA and perform exposure, gain adjustment and white balance processing, and transmit the processed image signal to the display through the HDMI interface. The display is used to receive image signals output by the ISP and display upright images.

8. The automatic rotation and alignment system for endoscopic images according to claim 7, characterized in that, The gyroscope module uses an ICM-42605 chip and transmits data with the ISP via an I2C bus; the accelerometer is configured with a range of ±4G and an output rate of 100Hz; the angular velocity sensor is configured with a range of ±1000dps and an output rate of 100Hz.

9. The automatic rotation and alignment system for endoscopic images according to claim 7, characterized in that, The FPGA uses an XC7A35T-FGG484 chip, which transmits rotation angle data to the ISP via a UART interface and image data via a DVP interface. The UART interface has a baud rate of 115200, no parity bit, 8 data bits, and 1 stop bit.

10. The automatic rotation and alignment system for endoscopic images according to claim 7, characterized in that, The image sensor uses an OV426 chip, outputting an image signal with a resolution of 400x400, a frame rate of 30fps, a pixel clock of 8MHz, a data bit width of 10 bits, and a data format of GBRG 10BIT.

Images