Method and system for implementing image 90-degree integer rotation

Abstract

The application relates to a method and system for realizing 90-degree integer multiple rotation of an image, wherein the method comprises the following steps: an FPGA receives original image data output by an image sensor and image processing parameters issued by an ARM, the image processing parameters at least including width, height and a rotation angle, wherein the rotation angle is an integer multiple of 90 degrees; the original image data is preprocessed and demosaiced to obtain a full-color image in RGB or YUV format; according to the width, height and rotation angle of the image, a coordinate transformation formula for 90-degree integer multiple rotation is used to determine the position mapping relationship between each source pixel in the full-color image and a target pixel after rotation; the full-color image is written into a BRAM cache in a ping-pong mode, and each pixel is read according to the order of the position mapping relationship and written into a target address in a DDR, wherein the target address in the DDR is calculated according to the position relationship.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method and system for realizing image rotation in multiples of 90°. Background Technology

[0002] In the current image processing workflow of medical endoscope systems, the technical solutions for achieving 90° integer multiple rotations of images mainly fall into two categories: software implementation solutions and general hardware rotation solutions. Software implementation solutions typically rely on the ARM processor to run rotation functions in traditional image processing libraries (such as OpenCV). After the raw data output from the CMOS image sensor is initially corrected and formatted by the FPGA, it is transmitted to the ARM processor via a high-speed interface (such as USB 3.0, PCIe, or MIPI CSI). On the ARM side, the rotation operation is implemented through software algorithms. This method typically employs a reverse mapping method, requiring the target image pixel coordinates to be reverse-mapped into the source image coordinate system, and the pixel values ​​to be calculated using interpolation algorithms. This method requires the transfer of a large amount of image data between the FPGA and the ARM, consuming significant bus bandwidth and exhibiting substantial processing latency. Alternatively, the image rotation operation can be completed by CPU or GPU computation.

[0003] Another approach is a general-purpose rotation technique based on FPGAs, capable of rotating images at arbitrary angles. This type of technique typically employs forward and reverse mapping methods, requiring complex trigonometric function calculations and interpolation. It also necessitates storing a complete image frame in DDR memory before rotating via address mapping. This method incurs significant access latency when reading and writing to DDR memory, and the non-contiguous addressing during image rotation leads to low DDR access efficiency, thus impacting overall system performance. For high-resolution medical endoscopic images (e.g., 400×400 to 1080P), general-purpose rotators consume substantial FPGA logic resources and DSP modules, and due to their high computational complexity, they struggle to meet the real-time requirements of medical applications (typically, industry standards require display latency below 60ms).

[0004] Existing technical solutions have the following significant drawbacks in medical endoscopy applications: High resource consumption: General image rotation schemes require complex coordinate transformation calculations, including trigonometric function operations and multiplication operations, which consume a large amount of FPGA DSP and logic resources. For small and medium-sized FPGA chips commonly used in medical devices, this resource consumption is unacceptable.

[0005] Insufficient real-time performance: Software-based implementations suffer from significant processing latency, failing to meet the high real-time requirements of medical endoscopes. When processing high-resolution images, ARM processors require a considerable amount of time to complete rotation calculations, resulting in noticeable image display delays and negatively impacting the doctor's experience.

[0006] Low DDR access efficiency: Rotation schemes based on full-frame buffers require frequent access to DDR memory, and due to the discontinuous nature of image addresses after rotation, the burst transmission characteristics of DDR cannot be effectively utilized, resulting in low bandwidth utilization. This problem is particularly prominent in high frame rate (e.g., above 30fps) medical endoscopy applications.

[0007] In summary, existing image rotation technology has significant shortcomings in terms of resource efficiency, real-time performance, and bandwidth utilization when applied to medical endoscope systems.

[0008] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0009] The main objective of this invention is to provide a method and system for rotating an image by an integer multiple of 90°, aiming to solve the aforementioned technical problems in the prior art.

[0010] To achieve the above objective, the present invention provides a method for rotating an image by multiples of 90°, the method comprising: The FPGA receives raw image data output from the image sensor and image processing parameters sent by the ARM. The image processing parameters include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°. The original image data is preprocessed and de-mosaiced to obtain a full-color image in RGB or YUV format; Based on the width, height, and rotation angle of the image, the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image is determined using the coordinate transformation formula for rotation in multiples of 90°. The full-color image is written to the BRAM cache in a ping-pong manner, and each pixel is read in order according to the position mapping relationship and written to the target address in DDR. The target address in DDR is calculated based on the position relationship. Once all data blocks of a frame have been written to DDR, a frame completion interrupt is sent to the ARM.

[0011] Preferably, in the method for implementing the 90° integer multiple rotation of the image, the step of writing the full-color image into the BRAM cache in a ping-pong manner, reading each pixel in order according to the position mapping relationship, and writing it into the target address in DDR includes: The full-color image is written into one of the two BRAMs inside the FPGA in a ping-pong manner according to the original scanning order. The depth of each BRAM is W×h1, where W is the image width and h1 is the preset number of rows. The writing and reading are alternated in a cycle of h1 rows. During the readout phase, a read address sequence is generated based on the position mapping relationship, and pixel data is read out from the BRAM according to the rotated row order; Simultaneously, the target address of each W×h1 pixel block in DDR is calculated based on the position mapping relationship, and the read pixel data is continuously written into the target address in DDR with a burst length corresponding to the rotation angle.

[0012] Preferably, in the method for implementing the 90° integer multiple rotation of the image, the step of generating a read address sequence according to the position mapping relationship during the readout stage includes: Once a BRAM block is filled with W×h1 pixel data, the read-out stage begins. Based on the rotation angle, a read address sequence is generated from the position mapping relationship, including: When the rotation angle is 90°, the read address sequence is generated in column order. Starting from the first column of pixels in the last row of the original image, all rows in that column are read upwards one by one, and then the next column is switched in turn. The address step size in each column is -W. When the rotation angle is 180°, the read address sequence is generated in reverse row and reverse column order, starting from the last pixel of the last row of the original image, and the address is decremented by 1 each time until the first pixel; When the rotation angle is 270°, the read address sequence is generated in reverse column order. Starting from the last column of pixels in the first row of the original image, all rows in that column are read down one by one, and then the previous column is switched in turn. The address step size in each column is +W.

[0013] Preferably, in the method for implementing the 90° integer multiple rotation of the image, the step of calculating the target address of each W×h1 pixel block in DDR according to the position mapping relationship includes: When the rotation angle is 90°, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is DDR base address + (current original column number × rotated height + (height - 1 - block start row)) × number of pixels and bytes. The address decreases continuously during burst writes. When the rotation angle is 180°, each W×h1 pixel block corresponds to a consecutive h1 rows in the rotated image and the row numbers are reversed. The target base address is the DDR base address + ((height-1-block start row)×width)×pixel bytes. The burst length is W. When writing, the address decreases from the end of the row. When the rotation angle is 270°, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is DDR base address + ((width - 1 - current original column number) × height + block start row) × number of pixel bytes. The address increases continuously during burst writes.

[0014] Preferably, in the method for implementing the 90° integer multiple rotation of the image, the step of determining the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image using the coordinate transformation formula for 90° integer multiple rotation based on the image's width, height, and rotation angle includes: When the rotation angle is 90°, Xt = H-Ys+1, Yt=Xs; When the rotation angle is 180°, Xt = W-Xs, Yt = H-Ys+1; When the rotation angle is 270°, Xt = Ys, Yt = W - Xs; Where (Xt, Yt) are the target coordinates after rotation; (Xs, Ys) are the source coordinates before rotation; W is the width of the image; H represents the height of the image.

[0015] Preferably, in the method for rotating the image by an integer multiple of 90°, the original image data is a Bayer image; The step of preprocessing the original image data and removing mosaic to obtain a full-color image in RGB or YUV format includes: The original image data is processed by ISP pre-processing, and each pixel in Bayer format is interpolated to extract the R, G, and B components, outputting a complete RGB or converted YUV pixel stream to obtain a full-color image in RGB or YUV format.

[0016] To achieve the above objectives, the present invention provides a system for rotating an image by multiples of 90°, the system comprising: Image sensors are used to acquire raw image data; ARM is used to send image processing parameters. The FPGA is connected to the image sensor and the ARM, respectively; DDR, connected to the FPGA, is used to cache rotated image data processed by the FPGA; The FPGA is used for: The system receives raw image data output from an image sensor and image processing parameters sent by an ARM. The image processing parameters include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°. The original image data is preprocessed and de-mosaiced to obtain a full-color image in RGB or YUV format; Based on the width, height, and rotation angle of the image, the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image is determined using the coordinate transformation formula for rotation in multiples of 90°. The full-color image is written to the BRAM cache in a ping-pong manner, and each pixel is read in order according to the position mapping relationship and written to the target address in DDR. The target address in DDR is calculated based on the position relationship. Once all data blocks of a frame have been written to DDR, a frame completion interrupt is sent to the ARM.

[0017] Preferably, in the system for implementing image rotation at integer multiples of 90°, the ARM is further used for: In response to receiving an interrupt from the FPGA, the rotated image data is read from the DDR and the read image data is output to the display screen.

[0018] Preferably, in the system for implementing image rotation at integer multiples of 90°, the ARM is further used for: Real-time detection of user input and dynamic adjustment of rotation angle based on user input; The new rotation angle is sent to the FPGA.

[0019] Preferably, in the system for implementing image rotation at integer multiples of 90°, the ARM is further used for: Image processing parameters are sent to the FPGA. The image processing parameters include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°.

[0020] The present invention has at least the following beneficial effects: This invention utilizes an FPGA to receive raw image data output from an image sensor and image processing parameters issued by an ARM. These parameters include at least width, height, and rotation angle, where the rotation angle is an integer multiple of 90°. The raw image data is preprocessed and de-pixelated to obtain a full-color image in RGB or YUV format. Based on the image's width, height, and rotation angle, a coordinate transformation formula for 90° integer multiple rotations is used to determine the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image. The full-color image is written to a BRAM cache in a ping-pong manner, and each pixel is read sequentially according to the positional mapping relationship and written to a target address in DDR, which is calculated based on the positional relationship. After all data blocks of a frame are written to DDR, a frame completion interrupt is sent to the ARM. This satisfies the special requirements of medical endoscope systems for image rotation processing, fully utilizes the characteristic of 90° integer multiple rotations to simplify computational complexity, and avoids unnecessary resource waste.

[0021] Furthermore, this invention overcomes the shortcomings of traditional software solutions and general-purpose rotating hardware. Through a carefully designed pipeline architecture and memory access strategy, this invention can significantly reduce hardware resource consumption while ensuring low latency, making it particularly suitable for resource-constrained medical endoscopic applications. This invention achieves efficient DDR bandwidth utilization. By optimizing data access patterns, it can maximize the burst transfer capabilities of DDR memory, reduce unnecessary memory accesses, thereby improving overall system energy efficiency and meeting the demands of high-frame-rate medical endoscopic imaging.

[0022] Furthermore, compared to traditional image rotation schemes, this invention significantly reduces resource consumption by simplifying coordinate transformation calculations and optimizing memory structure. Specifically, this invention utilizes the characteristic of rotations in multiples of 90° to simplify complex trigonometric function operations into simple coordinate exchange and rearrangement operations, reducing the usage of DSP and BRAM resources on the FPGA chip by more than 90%. Simultaneously, through a carefully designed pipeline architecture, this invention significantly reduces the usage of FPGA logic resources, making it possible to achieve high-quality image rotation on low-cost FPGAs.

[0023] Furthermore, this invention employs a highly parallel pipelined architecture, enabling rotation processing simultaneously with image data reception, achieving a near-zero latency rotation effect. Actual testing shows that for typical 400×400 resolution medical endoscopic images, the processing latency of this method is less than 1 millisecond, fully meeting the real-time requirements of medical imaging (less than 60ms). This low-latency characteristic ensures real-time display of endoscopic images, providing doctors with a smooth operating experience. Through an innovative data access strategy, this invention maximizes the bandwidth utilization of DDR memory. Compared to traditional DDR random access schemes, this method fully utilizes the burst transfer characteristics of DDR, increasing access efficiency to over 80%. This optimization allows the system to support higher image resolutions and frame rates with limited storage bandwidth, laying the foundation for high-definition medical endoscopic imaging.

[0024] Furthermore, the module parameters of this invention (such as image width, height, and pixel bit width) are configurable, and the rotation angle can be configured in real time via the bus to adapt to different medical endoscope application requirements. Attached Figure Description

[0025] Figure 1 A schematic diagram of the first embodiment of the method for implementing image rotation at integer multiples of 90° provided by the present invention; Figure 2 A schematic diagram of a second embodiment of the method for implementing image rotation at integer multiples of 90° provided by the present invention; Figure 3 This is a schematic diagram of an embodiment of the system for implementing image rotation in multiples of 90° provided by the present invention.

[0026] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0030] In this embodiment of the invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0031] In this invention, unless otherwise stated, directional terms such as "upper," "lower," "top," and "bottom" are generally used in relation to the direction shown in the accompanying drawings, or in relation to the vertical, perpendicular, or gravitational direction of the component itself; similarly, for ease of understanding and description, "inner" and "outer" refer to the inner and outer contours of each component itself, but the above directional terms are not intended to limit this invention.

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the embodiments of the present invention to facilitate a better understanding of the invention. However, the technical solutions claimed in the present invention can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0033] Figure 1 The diagram illustrates a method for implementing image rotation in multiples of 90° according to the present invention. This method includes steps S1000 to S5000.

[0034] In step S1000, the FPGA receives the raw image data output by the image sensor and the image processing parameters issued by the ARM. These image processing parameters include at least width, height, and rotation angle, where the rotation angle is an integer multiple of 90°. In endoscope systems, the image sensor is typically a CMOS image sensor, which outputs raw Bayer image data. The FPGA receives the raw image data output by the image sensor through a standard DVP interface or an AXI-VIDEO video interface.

[0035] The ARM processor connects to a set of configuration registers inside the FPGA via a low-speed control bus (such as UART, IIC, APB, or AXI-Lite). During system power-on initialization, the ARM writes image processing parameters to the FPGA. These parameters include the image width W and the image height H. The rotation angle is an integer multiple of 90°, such as 90°, 180°, or 270°. In some other implementations, a pixel bit width may also be included.

[0036] Step S2000 preprocesses the original image data and removes mosaic effects to obtain a full-color image in RGB or YUV format. In some embodiments, the original image data is a Bayer image, and step S2000 includes performing ISP preprocessing on the original image data and interpolating the R, G, and B components of each pixel in the Bayer format to output a complete RGB or converted YUV pixel stream, thus obtaining a full-color image in RGB or YUV format.

[0037] More specifically, the ISP pipeline within the FPGA sequentially performs pre-processing on the Bayer format raw image, including black level correction, lens shading correction, and noise reduction. This eliminates noise and distortion introduced by the image sensor itself and the optical environment. Then, by using de-mosaic interpolation, complete RGB or YUV components are generated for each pixel, outputting a full-color image data stream.

[0038] The image after pre-processing is still in Bayer format (each pixel has only one color component). The demosaic module inside the FPGA uses an edge-guided adaptive interpolation algorithm to calculate the complete R, G, and B components for each pixel. In some other implementations, the RGB full-color image can also be converted to YUV format as needed for subsequent rotation, compression, or display.

[0039] Step S3000 determines the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image based on the image's width, height, and rotation angle using a coordinate transformation formula for rotations that are multiples of 90°.

[0040] When the rotation angle is 90°, Xt = H-Ys+1, Yt=Xs; When the rotation angle is 180°, Xt = W-Xs, Yt = H-Ys+1; When the rotation angle is 270°, Xt = Ys, Yt = W - Xs; Where (Xt, Yt) are the target coordinates after rotation; (Xs, Ys) are the source coordinates before rotation; W is the width of the image; H represents the height of the image.

[0041] It should be noted that coordinate transformation in traditional rotation algorithms usually requires complex trigonometric function calculations, while this invention makes full use of the characteristics of rotations that are multiples of 90°, simplifying the transformation matrix into simple coordinate exchange and inversion operations.

[0042] In FPGAs, coordinates are not calculated in real-time for each pixel (which would introduce a lot of combinational logic and latency). Instead, the above formula is solidified into the calculation rules of the subsequent address generation module. By adopting the simplified coordinate transformation provided by this invention, compared with traditional rotation algorithms (such as affine transformations) which require multipliers and trigonometric function operations, consuming a lot of FPGA logic resources (DSP units) and increasing pipeline stages, this invention utilizes the characteristic of rotations in multiples of 90° to simplify the transformation to only adders, subtractors, and data selectors, completely avoiding the use of multipliers and lookup tables, reducing resource consumption by about an order of magnitude.

[0043] Furthermore, due to the simplicity of the coordinate mapping logic and the short combinational logic path, the critical path latency is significantly reduced. On typical FPGA devices, the rotation module can operate at clock frequencies of 200MHz or even higher, outputting one pixel per clock cycle, achieving line-speed processing. The deterministic nature of the coordinate mapping rules allows address generation to be completed within one clock cycle, eliminating the need for multi-cycle iterative calculations. Combined with the subsequent ping-pong BRAM architecture, the latency of the entire rotation module is only h1 rows of data buffer time (microseconds), far lower than solutions based on frame buffers or general-purpose processors (milliseconds), which is crucial for applications requiring real-time feedback, such as medical endoscopes.

[0044] Furthermore, the coordinate transformation formula relies on only three parameters: W, H, and the rotation angle. The ARM can dynamically modify the angle via the bus, eliminating the need for FPGA resynthesis or reconfiguration. This flexibility allows the same IP core to adapt to endoscope sensors of different resolutions (e.g., 400×400, 720P, 1080P) and supports in-process image rotation (e.g., one-click image orientation correction when a doctor changes the endoscope insertion direction). Once the mapping relationship is determined, BRAM read address generation and DDR write address generation can use the same coordinate transformation logic, avoiding redundant design. Simultaneously, because the mapping relationship is bijective and regular, complex boundary handling and pixel duplication / discarding are unnecessary, ensuring the output image maintains a completely lossless geometric transformation from the original image. The simplified coordinate transformation can be easily replicated multiple times, enabling multi-pixel parallel processing (e.g., processing 2×2 or 4×4 pixel blocks per clock cycle), further improving throughput. Traditional transformations involving multiplication are difficult to parallelize efficiently while maintaining accuracy.

[0045] Step S4000 involves writing the full-color image into the BRAM cache in a ping-pong manner, reading each pixel according to the position mapping relationship, and writing it to the target address in the DDR. The target address in the DDR is calculated based on the position relationship. It should be noted that a dual BRAM ping-pong structure is used to cache the full-color pixels in the original order. Simultaneously, a read address sequence is generated based on the rotation angle, pixels are read out in the rotated row order, and continuously written to the mapped address in the DDR with the optimal burst length, achieving efficient and low-latency rotation.

[0046] By reading each pixel in the order of the position mapping relationship, the order of the read pixels directly conforms to the row priority order of the rotated image.

[0047] Step S5000: After all data blocks of a frame image have been written to DDR, a frame completion interrupt is sent to the ARM.

[0048] Figure 2 The flowchart illustrates the method for implementing image rotation at integer multiples of 90° provided by the present invention in a second embodiment. Step S4000 includes steps S4100 to S4300.

[0049] In step S4100, the full-color image is written into one of the two BRAMs inside the FPGA in a ping-pong manner according to the original scanning order. Each BRAM has a depth of W×h1, where W is the image width and h1 is the preset number of rows. Writing and reading alternate in a cycle of h1 rows. It should be noted that the FPGA has two independent BRAMs (denoted as BRAM0 and BRAM1), each with a depth of W×h1, where W is the image width and h1 is the preset number of rows (e.g., 8, 16, or an optimal value calculated based on DDR timing). The full-color image data stream enters the pre-buffer module in the original scanning order (row-first, from left to right, from top to bottom).

[0050] Ping-pong operations are performed alternately in cycles of h1: In the first h1 row cycle: BRAM0 is in write mode, receiving and storing the current h1 row pixels (W pixels per row, W×h1 pixels in total); BRAM1 is in read mode, reading out the W×h1 data that was written in the previous cycle.

[0051] The second h1 row cycle: roles are reversed, BRAM1 writes a new h1 row, and BRAM0 reads it out.

[0052] This cycle repeats, achieving complete parallelism between writing and reading, with an uninterrupted data flow.

[0053] The write address is a simple incrementing counter, ranging from 0 to W×h1-1, incrementing by 1 for each pixel written. After the write operation is complete, the BRAM's read address generator takes over control.

[0054] Ping-pong operation allows pixels output from the ISP pipeline to be continuously input without waiting for DDR to finish writing, avoiding the throughput drop caused by the "write full-stop-clear" process in traditional single-buffered solutions, and achieving true line-speed processing.

[0055] In step S4200, during the readout stage, a read address sequence is generated based on the position mapping relationship, and pixel data is read from the BRAM according to the rotated row order. It should be noted that the BRAM read address sequence is derived from the position mapping relationship based on the rotation angle, ensuring that the readout order directly corresponds to the row priority order of the rotated image.

[0056] Once a BRAM is filled with W×h1 pixels, the read-out phase begins. The BRAM read / write control module generates a corresponding read address sequence based on the currently configured rotation angle (90°, 180°, or 270°). The design goal of this sequence is to retrieve pixels from the BRAM sequentially in row-major order of the rotated image, so that the read data stream can be directly used for DDR writing or display without needing to be rearranged.

[0057] Specifically, step S4200 involves entering the read-out stage after a BRAM block is filled with W×h1 pixel data; generating a read address sequence from the position mapping relationship based on the rotation angle, wherein: When the rotation angle is 90°, the read address sequence is generated in column order. Starting from the first column of pixels in the last row of the original image, all rows in that column are read upwards one by one, and then the next column is switched in turn. The address step size in each column is -W. When the rotation angle is 180°, the read address sequence is generated in reverse row and reverse column order, starting from the last pixel of the last row of the original image, and the address is decremented by 1 each time until the first pixel; When the rotation angle is 270°, the read address sequence is generated in reverse column order. Starting from the last column of pixels in the first row of the original image, all rows in that column are read down one by one, and then the previous column is switched in turn. The address step size in each column is +W.

[0058] By changing the read address order, the rotation operation is incorporated into the BRAM read process, eliminating the need for additional pixel buffers or rearrangement circuits and significantly reducing logic resources. Since the read order is already the rotated row-priority order, DDR write addresses can be continuous (or progressively stepped), which is beneficial for burst transfers. From the time the BRAM is full to the output of the first rotated pixel, only one clock cycle of address setup time is required. Subsequent outputs are one pixel per clock cycle, with the overall rotation latency being only h1 row buffer time plus the pipeline stage count (in microseconds).

[0059] In step S4300, the target address of each W×h1 pixel block in the DDR is calculated according to the position mapping relationship, and the read pixel data is continuously written to the target address in the DDR with a burst length corresponding to the rotation angle. By calculating the target base address in the DDR based on the position mapping relationship and continuously writing the read pixels according to the optimal burst length corresponding to the rotation angle, the DDR bandwidth utilization is maximized. It should be noted that steps S4300 and S4200 can be performed simultaneously, or they can be performed asynchronously as needed.

[0060] While reading pixels from the BRAM, the DDR address mapping module calculates the target base address in the DDR for each W×h1 pixel block (i.e., the data block written in each burst) according to the position mapping relationship in step S3000. The DDR uses a row-first contiguous storage method to store the rotated whole frame image.

[0061] Since the order read from the BRAM is already the rotated row-priority order, but at different rotation angles, the data corresponding to each burst may be part of a row or the entire row in the rotated image. In order to match the burst transmission characteristics of DDR, in some implementations, different burst lengths are selected for different rotation angles, specifically including steps S4310 to S4330.

[0062] In step S4310, when the rotation angle is 90°, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is DDR base address + (current original column number × rotated height + (height - 1 - block start row)) × number of pixel bytes. The address decreases continuously during burst writing.

[0063] Specifically, the order of reading from BRAM is column-wise and row-wise reversed, with each burst reading h1 pixels (h1 consecutive pixels in a column). When the rotation angle is 90°, the original image (width W, height H) is rotated to obtain a new image with width H and height W. The rotated image is stored in DDR in row-major order, i.e., row 0 (H pixels in total), row 1, ..., row W-1.

[0064] Each W×h1 pixel block (i.e., a data block cached in the FPGA ping-pong BRAM) corresponds to a consecutive h1 rows in the original image and contains a complete W column. Based on the coordinate mapping relationship after a 90° rotation: the source pixel is (Xs, Ys) (column number Xs, row number Ys), and the target position after rotation is (Xt=Ys, Yt=H-1-Xs) (using 0 index). Therefore, a pixel located in column j and row i in the original image will be located in row j and column H-1-i of the target image after rotation.

[0065] In step S4320, when the rotation angle is 180°, each W×h1 pixel block corresponds to a consecutive h1 rows in the rotated image with the row numbers reversed. The target base address is the DDR base address + ((height - 1 - block start row) × width) × number of pixels / bytes. The burst length is W, and the address decreases continuously from the end of the row during writing. During a 180° rotation, each W×h1 pixel block corresponds to a consecutive h1 rows in the rotated image with the row numbers reversed. The target base address is calculated from the block start row number, and the burst length is W. During writing, the address decreases continuously from the end of the row, thus fully utilizing the burst transfer capability of DDR.

[0066] When the rotation angle is 180°, the size of the image obtained after rotating the original image (width W, height H) remains unchanged, still being width W and height H, but the pixel order is completely reversed: the r-th row and c-th column pixels of the rotated image correspond to the H-1-r-th row and W-1-c-th column pixels of the original image.

[0067] For a 180° rotation, the order of reading from the BRAM is row-reverse and column-reverse: that is, from the W×h1 pixel block cached in the BRAM, the following order is followed: first read the last pixel (original column number W-1) of the last row (original row number i0+h1-1), then the column address within that row is decremented until the first pixel of that row; then read the last pixel of the previous row (original row number i0+h1-2), and so on.

[0068] In step S4330, when the rotation angle is 270°, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is the DDR base address + ((width - 1 - current original column number) × height + block start row) × number of pixel bytes. During burst writing, the address increases continuously. With a 270° rotation, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is uniquely determined by the current original column number and the block start row. Burst writing uses continuously increasing addresses, fully utilizing the burst transfer characteristics of DDR. When the rotation angle is 270°, the new image obtained after rotating the original image (width W, height H) has a width of H and a height of W (the same as the 90° rotation), but the pixel arrangement direction is opposite to that of 90°. The rotated image is stored in DDR in row-major order, i.e., row 0 (H pixels in total), row 1, ..., row W-1.

[0069] The present invention also provides a system for implementing image rotation in multiples of 90°, such as... Figure 3 As shown, the system for implementing 90° integer multiple rotation of the image includes an image sensor, ARM, FPGA, and DDR.

[0070] Among them, the image sensor is used to acquire raw image data.

[0071] The FPGA is connected to both the image sensor and the ARM. The FPGA receives raw image data from the image sensor and image processing parameters from the ARM. These parameters include at least width, height, and rotation angle, where the rotation angle is an integer multiple of 90°. The FPGA preprocesses the raw image data and removes mosaic effects to obtain a full-color image in RGB or YUV format. Based on the image's width, height, and rotation angle, the coordinate transformation formula for 90° integer multiple rotations determines the positional mapping between each source pixel and the rotated target pixel in the full-color image. The full-color image is written to the BRAM cache in a ping-pong manner, and each pixel is read according to the positional mapping order and written to the target address in DDR, which is calculated based on the positional relationship. After all data blocks of a frame are written to DDR, a frame completion interrupt is sent to the ARM. This FPGA embodiment includes an embodiment of the above-described method for implementing 90° integer multiple rotation of an image, and the beneficial effects of this method can also be applied to this FPGA.

[0072] The ARM processor is used to send image processing parameters. In response to an interrupt sent by the FPGA, the ARM processor reads rotated image data from the DDR and outputs the read image data to the display screen. The ARM processor enters an interrupt service routine and reads the latest rotated image data from the DDR via memory mapping.

[0073] The ARM is also used to detect user input in real time and dynamically adjust the rotation angle based on the user input; the new rotation angle is then sent to the FPGA.

[0074] The ARM is also used to send image processing parameters to the FPGA, which include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°.

[0075] The ARM processor monitors button presses, touch inputs, and gesture sensors in real time. Based on user needs or changes in endoscope orientation, it calculates new rotation angles and sends them to the FPGA. The ARM processor polls or receives user input via interrupts (such as rotation buttons, foot switches, and lens gesture sensors on the interface). If the rotation angle needs to be changed, the ARM processor writes the new angle configuration to the FPGA via a low-speed bus. The FPGA applies the new parameters at the start of the next frame, achieving seamless switching without screen tearing.

[0076] The DDR (Memory Memory) is connected to the FPGA to cache the rotated image data processed by the FPGA. As an external high-capacity memory, the DDR connects to the FPGA via a dedicated memory interface to temporarily store the entire frame of image data after the FPGA has completed the rotation processing, enabling data buffering and cross-module sharing. The rotated image stored in the DDR is not only for potential reuse by the FPGA (e.g., multi-frame overlay), but more importantly, it is read by the ARM processor. After the FPGA completes the rotation of each frame and writes it to the DDR, it sends a frame completion interrupt to the ARM. Upon receiving the interrupt, the ARM reads the frame from the same physical address in the DDR through its internal memory controller (or via a shared bus), and then performs display, compression, storage, or network transmission. This pipelined mechanism of "FPGA write-DDR-ARM read" achieves efficient data exchange, avoiding the inefficient overhead of pixel-by-pixel transmission between the FPGA and the ARM.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for implementing image rotation in multiples of 90°, characterized in that, include: The FPGA receives raw image data output from the image sensor and image processing parameters sent by the ARM. The image processing parameters include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°. The original image data is preprocessed and de-mosaiced to obtain a full-color image in RGB or YUV format; Based on the width, height, and rotation angle of the image, the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image is determined using the coordinate transformation formula for rotation in multiples of 90°. The full-color image is written to the BRAM cache in a ping-pong manner, and each pixel is read in order according to the position mapping relationship and written to the target address in DDR. The target address in DDR is calculated based on the position relationship. Once all data blocks of a frame have been written to DDR, a frame completion interrupt is sent to the ARM.

2. The method for implementing image rotation at integer multiples of 90° as described in claim 1, characterized in that, The step of writing the full-color image into the BRAM cache in a ping-pong manner, reading each pixel in order according to the position mapping relationship, and writing it into the target address in DDR includes: The full-color image is written into one of the two BRAMs inside the FPGA in a ping-pong manner according to the original scanning order. The depth of each BRAM is W×h1, where W is the image width and h1 is the preset number of rows. The writing and reading are alternated in a cycle of h1 rows. During the readout phase, a read address sequence is generated based on the position mapping relationship, and pixel data is read out from the BRAM according to the rotated row order; Simultaneously, the target address of each W×h1 pixel block in DDR is calculated based on the position mapping relationship, and the read pixel data is continuously written into the target address in DDR with a burst length corresponding to the rotation angle.

3. The method for implementing image rotation at integer multiples of 90° as described in claim 2, characterized in that, In the read-out phase, generating a read address sequence based on the position mapping relationship includes: Once a BRAM block is filled with W×h1 pixel data, the read-out stage begins. Based on the rotation angle, a read address sequence is generated from the position mapping relationship, including: When the rotation angle is 90°, the read address sequence is generated in column order. Starting from the first column of pixels in the last row of the original image, all rows in that column are read upwards one by one, and then the next column is switched in turn. The address step size in each column is -W. When the rotation angle is 180°, the read address sequence is generated in reverse row and reverse column order, starting from the last pixel of the last row of the original image, and the address is decremented by 1 each time until the first pixel; When the rotation angle is 270°, the read address sequence is generated in reverse column order. Starting from the last column of pixels in the first row of the original image, all rows in that column are read down one by one, and then the previous column is switched in turn. The address step size in each column is +W.

4. The method for implementing image rotation at integer multiples of 90° as described in claim 2, characterized in that, The step of calculating the target address of each W×h1 pixel block in DDR according to the position mapping relationship includes: When the rotation angle is 90°, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is DDR base address + (current original column number × rotated height + (height - 1 - block start row)) × number of pixels and bytes. The address decreases continuously during burst writes. When the rotation angle is 180°, each W×h1 pixel block corresponds to a consecutive h1 rows in the rotated image and the row numbers are reversed. The target base address is the DDR base address + ((height-1-block start row)×width)×pixel bytes. The burst length is W. When writing, the address decreases from the end of the row. When the rotation angle is 270°, each W×h1 pixel block corresponds to h1 different rows in the rotated image. The base address of each target row is DDR base address + ((width - 1 - current original column number) × height + block start row) × number of pixel bytes. The address increases continuously during burst writes.

5. The method for implementing image rotation at integer multiples of 90° as described in claim 2, characterized in that, The step of determining the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image based on the image's width, height, and rotation angle, using a coordinate transformation formula for rotations in multiples of 90°, includes: When the rotation angle is 90°, Xt = H-Ys+1, Yt=Xs; When the rotation angle is 180°, Xt = W-Xs, Yt = H-Ys+1; When the rotation angle is 270°, Xt = Ys, Yt = W - Xs; Where (Xt, Yt) are the target coordinates after rotation; (Xs, Ys) are the source coordinates before rotation; W is the width of the image; H represents the height of the image.

6. The method for implementing image rotation at integer multiples of 90° as described in claim 1, characterized in that, The original image data is a Bayer image; The step of preprocessing the original image data and removing mosaic to obtain a full-color image in RGB or YUV format includes: The original image data is processed by ISP pre-processing, and each pixel in Bayer format is interpolated to extract the R, G, and B components, outputting a complete RGB or converted YUV pixel stream to obtain a full-color image in RGB or YUV format.

7. A system for implementing image rotation in multiples of 90°, characterized in that, include: Image sensors are used to acquire raw image data; ARM is used to send image processing parameters. The FPGA is connected to the image sensor and the ARM, respectively; DDR, connected to the FPGA, is used to cache rotated image data processed by the FPGA; The FPGA is used for: The system receives raw image data output from an image sensor and image processing parameters sent by an ARM. The image processing parameters include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°. The original image data is preprocessed and de-mosaiced to obtain a full-color image in RGB or YUV format; Based on the width, height, and rotation angle of the image, the positional mapping relationship between each source pixel and the rotated target pixel in the full-color image is determined using the coordinate transformation formula for rotation in multiples of 90°. The full-color image is written to the BRAM cache in a ping-pong manner, and each pixel is read in order according to the position mapping relationship and written to the target address in DDR. The target address in DDR is calculated based on the position relationship. Once all data blocks of a frame have been written to DDR, a frame completion interrupt is sent to the ARM.

8. The system for implementing image rotation at integer multiples of 90° as described in claim 7, characterized in that, The ARM is also used for: In response to receiving an interrupt from the FPGA, the rotated image data is read from the DDR and the read image data is output to the display screen.

9. The system for implementing image rotation at integer multiples of 90° as described in claim 7, characterized in that, The ARM is also used for: Real-time detection of user input and dynamic adjustment of rotation angle based on user input; The new rotation angle is sent to the FPGA.

10. The system for implementing image rotation at integer multiples of 90° as described in claim 7, characterized in that, The ARM is also used for: Image processing parameters are sent to the FPGA. The image processing parameters include at least width, height, and rotation angle, wherein the rotation angle is an integer multiple of 90°.

Images