Image superimposition method and system based on motion sensor

Abstract

The application discloses a kind of image superimposition methods based on motion sensor, comprising the following steps: according to the displacement speed of image x direction and y direction, the pixel offset of two directions is calculated respectively, as the pixel offset of front and rear two frames of image;Accumulate the pixel offset of x, y direction, obtain para_x, para_y;The accumulated pixel offset para_x of x direction is divided by 8, and quotient para_x_quot and remainder para_x_remd are obtained, according to para_y, para_x_quot and image field address determine DDR superimposition address;According to DDR superimposition address and the number of input pixels, the current frame image data is added with the last frame image data in DDR.The application can complete image superimposition according to rotation angle or displacement, and is used for monitoring device movement, such as inclination, shaking, rotation or swing.

Description

Technical Field

[0001] This invention relates to image processing, and more particularly to an image overlay method based on a motion sensor. Background Technology

[0002] Motion sensors detect the gravity, linear acceleration, rotation vector, and vibration frequency of moving objects or people. They can be used to monitor equipment movement, such as tilting, shaking, rotating, or swaying. During monitoring, images can be overlaid for better display. However, current image overlay implementations are often limited; for example, they may not be able to store or display the entire overlay area, or the overlay parameters may be fixed, resulting in unsatisfactory overlay effects. Summary of the Invention

[0003] The main objective of this invention is to provide an image overlay method and system based on a motion sensor that can effectively overlay images according to the displacement velocity of an object and provide overall resolution.

[0004] The technical solution adopted in this invention is:

[0005] A motion sensor-based image overlay method is provided, comprising the following steps:

[0006] The pixel offsets in the x and y directions of the image are calculated respectively, and used as the pixel offsets between the two frames.

[0007] Accumulate the pixel offsets in the x and y directions to obtain para_x and para_y;

[0008] Divide the accumulated pixel offset para_x in the x-direction by 8 to obtain the quotient para_x_quot and the remainder para_x_remd. Determine the DDR overlay address based on para_y, para_x_quot and the image field address. Each pixel occupies two DDR addresses, and each DDR address is 8 bits. The number of rows to offset in the y-direction is determined by para_y, the number of pixels to offset in the x-direction is determined by para_x_quot, and the number of frames to be taken for the input pixel is determined by para_x_remd.

[0009] Based on the DDR overlay address and the number of shots of the input pixels, the current frame image data is added to the previous frame image data in the DDR.

[0010] Following the above technical solution, the DDR read / write enable is generated from the image row and field addresses.

[0011] Following the above technical solution, when the image moves in a straight line, the pixel offset is directly calculated based on the displacement velocity in the x and y directions of the image.

[0012] Following the above technical solution, when the image rotates, the pixel offset in the y and x directions is calculated based on the sine and cosine values ​​of the rotation angle θ.

[0013] Following the above technical solution, the pixel offset S in the x-direction x Pixel offset S in the y direction y Obtained from the following formula:

[0014]

[0015]

[0016] Among them, V x V is the displacement velocity in the x-direction. y Let S be the displacement velocity in the y-direction, f be the image frame rate, and d be the width of a single pixel. The calculated S... x S y Rounding is performed.

[0017] Following the above technical solution, the object's rotation angle θ is obtained by the following formula:

[0018]

[0019] Where ω is the angular velocity of the object's rotation, f is the image frame rate, and the final calculated pixel offsets in the y and x directions are rounded.

[0020] The present invention also provides an image overlay system based on a motion sensor, comprising:

[0021] The pixel offset calculation module is used to calculate the pixel offset in the x and y directions of the image based on the displacement velocity in the x and y directions, respectively, as the pixel offset of the two frames; and to accumulate the pixel offsets in the x and y directions to obtain para_x and para_y.

[0022] The DDR overlay address calculation module is used to divide the accumulated pixel offset para_x in the x-direction by 8 to obtain the quotient para_x_quot and the remainder para_x_remd. The DDR overlay address is determined based on para_y, para_x_quot and the image field address. Each pixel occupies two DDR addresses, and each DDR address is 8 bits. The number of rows to be offset in the y-direction is determined based on para_y, the number of pixels to be offset in the x-direction is determined based on para_x_quot, and the number of frames to be captured for the input pixel is determined based on para_x_remd.

[0023] The data overlay module is used to add the current frame image data to the previous frame image data in the DDR according to the DDR overlay address and the number of times the input pixels are captured.

[0024] Following the above technical solution, the pixel offset calculation module is specifically used to: calculate the pixel offset directly based on the displacement velocity in the x and y directions of the image when the image moves in a straight line.

[0025] Following the above technical solution, the pixel offset calculation module is specifically used to: calculate the pixel offset of the image in the y and x directions based on the sine and cosine values ​​of the rotation angle θ when the image rotates.

[0026] The present invention also provides a computer storage medium storing a computer program executable by a processor, the computer program executing the image overlay method based on a motion sensor described in the above technical solution.

[0027] The beneficial effects of this invention are as follows: This invention measures the displacement velocity of an object in the x and y directions of a coordinate system using a displacement sensor, thereby calculating the displacement amount within a certain time period. Alternatively, it measures the angular velocity of the object's rotation using an angular velocity meter, thereby calculating the rotation angle within a certain time period. Furthermore, it performs image overlay based on the displacement amount or rotation angle. This invention allows for arbitrary changes to the motion direction in the x and y directions, and the image overlay results can be entirely stored in DDR (Residual Generator). Attached Figure Description

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0029] Figure 1 This is a flowchart of an image overlay method based on a motion sensor according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of image motion (moving to the lower right) according to an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of image motion (moving to the upper left) according to an embodiment of the present invention;

[0032] Figure 4 yes Figure 2 DDR read / write diagram when the image moves to the lower right;

[0033] Figure 5 yes Figure 3 A schematic diagram of DDR read / write operations when the image moves to the upper left.

[0034] Figure 6 It is a schematic diagram of the image motion rotating clockwise;

[0035] Figure 7 It is a schematic diagram of the image rotating counterclockwise. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0037] like Figure 1 As shown, the image overlay method based on a motion sensor in this embodiment of the invention includes the following steps:

[0038] S1. Calculate the pixel offset in the x and y directions of the image based on the displacement velocity in the x and y directions respectively, and use it as the pixel offset between the two frames.

[0039] S2. Accumulate the pixel offsets in the x and y directions to obtain para_x and para_y;

[0040] S3. Divide the accumulated pixel offset para_x in the x-direction by 8 to obtain the quotient para_x_quot and the remainder para_x_remd. Determine the DDR overlay address based on para_y, para_x_quot and the image field address (i.e., the address corresponding to the image field signal). Each pixel occupies two DDR addresses, and each DDR address is 8 bits. The number of rows to be offset in the y-direction is determined by para_y, the number of pixels to be offset in the x-direction is determined by para_x_quot, and the number of frames (i.e., the number of clock cycles of delay) of the input pixel is determined by para_x_remd.

[0041] S4. Based on the DDR overlay address and the number of times the input pixel is hit, add the current frame image data to the previous frame image data in the DDR.

[0042] The specific implementation platform of this patent is Xilinx's Vivado software. The specific implementation scheme is explained separately for the cases of x and y direction offset and angular rotation.

[0043] 1. Offset in the x and y directions

[0044] Pixel offset S in the x direction x Pixel offset S in the y direction y It can be obtained from the following formula.

[0045]

[0046]

[0047] Among them, V x V is the displacement velocity in the x-direction. y Let S be the displacement velocity in the y-direction, f be the image frame rate, and d be the width of a single pixel. The pixel offset is the offset between two consecutive frames and is an integer; therefore, S needs to be...x S y After rounding, the images are overlaid based on their values.

[0048] When an object moves to the right in the x-direction and downwards in the y-direction, the motion diagram is as follows: Figure 2 As shown; when the object moves to the left in the x-direction and upward in the y-direction, the motion diagram is as follows. Figure 3 As shown in the diagram, the black area represents the image overlay region (the common area of ​​n frames), and the dashed line represents the portion stored in DDR (the entire area of ​​n frames). Starting from the address of BLOCK1, the overlaid image is gradually stored in DDR. It is worth noting that the DDR needs to be cleared before overlaying the image. Similarly, the situation is the same when the object moves in the lower left or upper right direction.

[0049] Each pixel (14 bits) occupies 2 DDR addresses (each DDR address can store 8 bits). The x-axis offset of the DDR read / write address must be a multiple of 16 (for pixels, the offset is a multiple of 8), otherwise data cannot be read or written accurately from the target address. The y-axis offset has no requirements (because data can be read from any row in the DDR, and the row corresponds to the y-axis). Accumulate the x and y-axis motion parameters to obtain para_x and para_y. Divide para_x by 8 to obtain the quotient para_x_quot and remainder para_x_remd. Determine the DDR read / write address based on para_y, para_x_quot, and the image field address. para_y determines the number of rows to offset in the y-axis, and para_x_quot determines the number of pixels to offset in the x-axis (a multiple of 8). The calculation process is as follows: Let the image field address be ver, the starting address of DDR BLOCK1 be a1, and the maximum number of pixels stored per row in DDR be 1024. When the image moves to the lower right, the DDR read / write address is a1 + (ver + para_y) * 1024 * 2 + para_x_quot * 2; when the image moves to the upper left, the DDR read / write address is b1 + (ver - para_y) * 1024 * 2 - para_x_quot * 2, where b1 is a1 + (1024 - 648) * 2. The 648 is derived from the image resolution of 640 plus 8. 1024 is the maximum number of pixels that DDR can store per line; currently, only 648 are stored, and other addresses are invalid.

[0050] The number of shots for the input pixel is determined based on para_x_remd so that it can be added to the DDR read data; while the generation of DDR read / write enable is not based on para_x and para_y, but is directly obtained from the image line and field addresses (the addresses corresponding to the image line and field signals).

[0051] The DDR write address is the same as the read address. Assuming the image resolution is 640*N and the para_x_remd value is 3, then the DDR read / write length should be 648. The read / write behavior of the first line of the DDR memory when the current frame moves to the lower right is as follows: Figure 4 As shown, the read / write status of the first row of DDR when the image moves to the upper left is as follows. Figure 5 As shown in the diagram, addr is the address of the first row of DDR, rd_data is the data read from DDR (for the current frame, already written to DDR in the previous frame), data_in is the input pixel of the current frame (first row), wr_data is the data written to the corresponding address in DDR, and a and d-16 are both the starting addresses of the first row of DDR in the current frame. That is, the DDR read and write addresses are the same; reading starts from a, and writing must also start from a, because the x-axis offset of the DDR read and write addresses must be a multiple of 16.

[0052] 1024 is the maximum number of pixels that can be stored per row in DDR, but only 648 are currently stored, and the other addresses are invalid. When the image moves to the upper left, the overlay result is stored in the right half of DDR, which is exactly 648 addresses from d to the end of this row.

[0053] 2. Angle rotation

[0054] The rotation angle θ of an object can be obtained by the following formula.

[0055]

[0056] Where ω is the angular velocity of the object's rotation, and f is the image frame rate.

[0057] As the object rotates clockwise, the image shows a motion diagram. Figure 6 As shown; when the object rotates counterclockwise, the image motion diagram is as follows. Figure 7 As shown. Starting from the address of BLOCK1, the superimposed image is gradually stored in DDR.

[0058] When the image rotates, the offset in the x-direction when the image shifts one line in the y-direction can be determined using the sine of the rotation angle θ. Non-integer values ​​need to be rounded. Based on this, the stored data at the corresponding DDR address can be read and written to the same address after image overlay. It is important to note that the x-direction offset of the DDR read / write address must be a multiple of 16. Therefore, during DDR read / write, the starting address of each line must be a multiple of 16. After reading the data, the sum of the read data and the current frame's input image is written to the address where the image needs to be overlaid, and the original read data is written to the address where no image overlay is needed.

[0059] The image overlay system based on a motion sensor in this invention is used to implement the image overlay method described in the above embodiments. The system includes:

[0060] The pixel offset calculation module is used to calculate the pixel offset in the x and y directions of the image based on the displacement velocity in the x and y directions, respectively, as the pixel offset of the two frames; and to accumulate the pixel offsets in the x and y directions to obtain para_x and para_y.

[0061] The DDR overlay address calculation module is used to divide the accumulated pixel offset para_x in the x-direction by 8 to obtain the quotient para_x_quot and the remainder para_x_remd. The DDR overlay address is determined based on para_y, para_x_quot and the image field address. Each pixel occupies two DDR addresses, and each DDR address is 8 bits. The number of rows to be offset in the y-direction is determined based on para_y, the number of pixels to be offset in the x-direction is determined based on para_x_quot, and the number of frames to be captured for the input pixel is determined based on para_x_remd.

[0062] The data overlay module is used to add the current frame image data to the previous frame image data in the DDR according to the DDR overlay address and the number of times the input pixels are captured.

[0063] Furthermore, the pixel offset calculation module is specifically used to: calculate the pixel offset directly based on the displacement velocity in the x and y directions of the image when the image moves in a straight line.

[0064] Furthermore, the pixel offset calculation module is specifically used to: calculate the pixel offset of the image in the y and x directions based on the sine and cosine values ​​of the rotation angle θ when the image rotates.

[0065] This invention also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, app store, etc., which stores a computer program, which performs corresponding functions when executed by a processor. The computer-readable storage medium of this embodiment is used to implement the motion sensor-based image overlay method of the method embodiment when executed by a processor.

[0066] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An image overlay method based on a motion sensor, characterized in that, Includes the following steps: The pixel offsets in the x and y directions of the image are calculated respectively, and used as the pixel offsets between the two frames. Accumulate the pixel offsets in the x and y directions to obtain para_x and para_y; Divide the accumulated pixel offset para_x in the x-direction by 8 to obtain the quotient para_x_quot and the remainder para_x_remd. Determine the DDR overlay address based on para_y, para_x_quot and the image field address. Each pixel occupies two DDR addresses, and each DDR address is 8 bits. The number of rows to offset in the y-direction is determined by para_y, the number of pixels to offset in the x-direction is determined by para_x_quot, and the number of frames to be taken for the input pixel is determined by para_x_remd. Based on the DDR overlay address and the number of shots of the input pixels, the current frame image data is added to the previous frame image data in the DDR.

2. The image overlay method based on a motion sensor according to claim 1, characterized in that, The DDR read / write enable is generated from the image row and field addresses.

3. The image overlay method based on a motion sensor according to claim 1, characterized in that, When the image moves in a straight line, the pixel offset is calculated directly based on the displacement velocity in the x and y directions of the image.

4. The image overlay method based on a motion sensor according to claim 1, characterized in that, When the image rotates, the pixel offset in the y and x directions is calculated based on the sine and cosine values ​​of the rotation angle θ.

5. The image overlay method based on a motion sensor according to claim 3, characterized in that, Pixel offset S in the x direction x Pixel offset S in the y direction y Obtained from the following formula: Among them, V x V is the displacement velocity in the x-direction. y Let S be the displacement velocity in the y-direction, f be the image frame rate, and d be the width of a single pixel. The calculated S... x S y Rounding is performed.

6. The image overlay method based on a motion sensor according to claim 4, characterized in that, The rotation angle θ of an object is obtained by the following formula: Where ω is the angular velocity of the object's rotation, f is the image frame rate, and the final calculated pixel offsets in the y and x directions are rounded.

7. An image overlay system based on a motion sensor, characterized in that, include: The pixel offset calculation module is used to calculate the pixel offset in the x and y directions of the image based on the displacement velocity in the x and y directions, respectively, as the pixel offset of the two frames; and to accumulate the pixel offsets in the x and y directions to obtain para_x and para_y. The DDR overlay address calculation module is used to divide the accumulated pixel offset para_x in the x-direction by 8 to obtain the quotient para_x_quot and the remainder para_x_remd. The DDR overlay address is determined based on para_y, para_x_quot and the image field address. Each pixel occupies two DDR addresses, and each DDR address is 8 bits. The number of rows to be offset in the y-direction is determined based on para_y, the number of pixels to be offset in the x-direction is determined based on para_x_quot, and the number of frames to be captured for the input pixel is determined based on para_x_remd. The data overlay module is used to add the current frame image data to the previous frame image data in the DDR according to the DDR overlay address and the number of times the input pixels are captured.

8. The image overlay system based on a motion sensor according to claim 7, characterized in that, The pixel offset calculation module is specifically used to calculate the pixel offset directly based on the displacement velocity in the x and y directions of the image when the image moves in a straight line.

9. The image overlay system based on a motion sensor according to claim 7, characterized in that, The pixel offset calculation module is specifically used to calculate the pixel offset of the image in the y and x directions based on the sine and cosine values ​​of the rotation angle θ when the image rotates.

10. A computer storage medium, characterized in that, It contains a computer program that can be executed by a processor, which performs the image overlay method based on a motion sensor as described in any one of claims 1-6.

Images