Frame buffer merging method based on dirty rectangular region calculation

Abstract

The invention provides a frame buffer merging method based on dirty rectangular region calculation, which comprises the following steps of: S1, setting the number of reserved frame buffers as MAXNUM after a frame buffer device driver applies for reserved frame buffers; s2, synchronization among frame buffers of the MAXNUM meets a formula: Si = S (i-1) + Ri; i = (i + 1)% MAXNUM; s3, MAXNUM is set to be equal to 3, and R1 can be updated to a frame buffer S1 for a dirty rectangular region R1 needing to be drawn currently, namely S1 is equal to R1; s4, R2 needs to be drawn at present, R2 is updated to a frame buffer S2, and S2 = R1xR2; s5, the R3 needs to be drawn at present, the R3 is updated to the frame buffer S3, and S3 = R1 * R2 * R3; s6, when no new dirty rectangular area exists temporarily, for frame buffer equipment which starts a triple frame buffer vertical synchronization mechanism, updating the rectangular area in the record to a frame buffer S1, and recording as S4 = S1 + R2xR3; s7, on the updating frame buffer S2, S5 = S2 + R3; and S8, on the updating frame buffer S3, S6 is equal to S3. According to the method, calculation can be carried out based on the dirty rectangular region, and image merging can be carried out between multiple frame buffers correspondingly using the vertical synchronization technology.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a frame buffer merging method based on dirty rectangular area calculation. Background Art

[0002] As the requirements for graphics and images in the embedded field continue to increase, more and more graphic controls are used to draw software application interfaces, and higher requirements are placed on the frame rate of graphics display and the display quality of images. In the existing graphical user interface development framework, the dirty rectangle mechanism is usually used when interacting with the frame buffer device for image data, reducing resource consumption by reducing the amount of data updates. However, the dirty rectangle mechanism cannot solve the screen tearing problem. In the scene where the image is frequently updated, we can easily see abnormal phenomena such as misalignment and discontinuity of the image displayed on the screen. Therefore, we introduced a vertical synchronization mechanism in the graphical user interface to limit the frame buffer currently read by the hardware from being overwritten by the graphical user interface at the same time, and provide other operable frame buffers to the graphical user interface through multiple buffers to reduce display delays.

[0003] However, there is no combination between the existing dirty rectangle technology and the vertical synchronization technology. When applying dirty rectangles, a common graphical user interface only updates data on a fixed frame buffer, and does not handle the image merging problem between frame buffers.

[0004] In addition, technical terms commonly used in this field include:

[0005] Dirty rectangle: An important concept in graphics performance optimization, it refers to the local area that needs to be redrawn when the image is updated. The use of dirty rectangles can reduce the amount of data for image updates, reduce resource consumption, and enhance display performance.

[0006] Frame buffer device: A frame buffer device is an abstraction of graphics hardware that represents the frame buffer area of ​​some video hardware and allows software applications to access the graphics hardware through a well-defined interface.

[0007] Vertical synchronization: When the display device screen reads the frame buffer line by line, the image is prohibited from updating the frame buffer currently used by the hardware. The image is allowed to overwrite the current frame buffer only after the screen display is completed and the current frame buffer is no longer read, so as to avoid screen tearing problems, but it will cause display delays.

[0008] Screen tearing: When the display device screen reads the image in the frame buffer line by line for display, the program is drawing the screen and overwrites the read frame buffer, resulting in abnormal phenomena such as misalignment and discontinuity of the image displayed on the screen.

[0009] Multiple buffers: When the display device enables the vertical synchronization mechanism, in order to reduce display delay, the number of frame buffers can be expanded to provide more buffers for image updates. In this way, when a frame buffer is disabled, the image can be overwritten on other frame buffers, and when the display device screen finishes reading a frame buffer, it can promptly switch to another frame buffer that has been updated for reading. Summary of the invention

[0010] In order to solve the above problems, the purpose of the present application is to provide a method that can perform image merging between multiple frame buffers based on dirty rectangular area calculation and correspondingly using vertical synchronization technology.

[0011] Specifically, the present invention provides a frame buffer merging method based on dirty rectangular area calculation, the method comprising the following steps:

[0012] Step S1. After the frame buffer device driver applies for reserved frame buffer, the number of reserved frame buffers is set to MAX_NUM;

[0013] Step S2. For MAX_NUM frame buffer synchronization, the following formula is satisfied:

[0014] Si=S(i-1)+Ri;

[0015] i = (i + 1)% MAX_NUM;

[0016] Where i represents the subscript value corresponding to the frame buffer of the current operation;

[0017] Si represents the graphics data value of the i-th frame buffer;

[0018] S(i-1) represents the graphics data value of the i-1th frame buffer;

[0019] Ri represents the graphic data that needs to be drawn currently;

[0020] After each drawing of a graphic data and synchronization between frame buffers, the current i value is incremented by one, and the incremented value is modulo MAX_NUM to ensure that the i value is within the frame buffer subscript range;

[0021] Step S3. When the frame buffer MAX_NUM=N, SN=S(N-1)+RN; if MAX_NUM=3, that is, 3 frame buffers are reserved, then when the graphical user interface obtains the dirty rectangular area R1 to be drawn, R1 can be updated to the frame buffer S1, at this time i=1, that is,

[0022] S1=R1;

[0023] Step S4. When the graphical user interface obtains the dirty rectangular area R2 that needs to be drawn, R2 can be updated to the frame buffer S2, that is, the area after the R1 and R2 matrices are merged and deduplicated is updated, and R1xR2 represents the matrix merge and deduplication. At this time

[0024] S2 = R1 x R2;

[0025] Step S5. When the graphical user interface obtains the dirty rectangular area R3 that needs to be drawn, R3 can be updated to the frame buffer S3.

[0026] S3 = R1xR2xR3;

[0027] Step S6. When there is no new dirty rectangular area in the graphical user interface, that is, when the user no longer draws a new image, there is no new dirty rectangular area input at this time. For the frame buffer device with the triple frame buffer vertical synchronization mechanism enabled, that is, the case of MAX_NUM=3 and the case of MAX_NUM>1, it is still necessary to update the rectangular area in the record to the frame buffer S1. Here, the selection of updating the frame buffer S1, frame buffer S2, and frame buffer S3 is performed in sequence. After updating the frame buffer S3, the frame buffer S1 is updated again, and the cycle continues; the recorded rectangular area needs to comply with the effective number not exceeding MAX_NUM, so the R1 area is discarded from the record. Since MAX_NUM is set to 3, there are only three frame buffers in total. Therefore, when a rectangular area, here is the R1 area, is synchronized 3 times between the frame buffers, all frame buffers have updated the R1 area at this time, and the R1 area no longer needs to be synchronized and can be discarded; this is recorded as

[0028] S4 = S1 + R2 x R3;

[0029] Step S7. For the frame buffer device with triple frame buffer vertical synchronization mechanism enabled when there is no new dirty rectangular area, that is, the case of MAX_NUM=3, update the frame buffer S2 in sequence.

[0030] S5=S2+R3;

[0031] Here, step S7 corresponds to the case where there is no new dirty rectangular area input. If there is a dirty rectangular area input, the process is performed normally according to step S4;

[0032] Step S8. For the frame buffer device with the triple frame buffer vertical synchronization mechanism enabled when there is no new dirty rectangular area, that is, the case where MAX_NUM=3, update the frame buffer S3 in sequence.

[0033] S6 = S3;

[0034] Here, step S8 corresponds to the case where no new dirty rectangular region is input. If a dirty rectangular region is input, the process proceeds normally according to step S5.

[0035] In step S1, the method for the frame buffer device driver to apply for reserved frame buffer is to reserve frame buffer shared memory in the kernel display driver, with continuous physical addresses and hardware alignment; the function for reserving memory is the system function dma_alloc_coherent, and the parameter is the size of the memory.

[0036] In step S2, the synchronization between frame buffers is to avoid the waiting time of frame drawing by switching between multiple buffers during the graphics frame drawing process, so as to make data processing smoother. At this time, when switching between multiple buffers, the drawing process of the previous frame buffer needs to be copied to avoid the loss of drawing data. This process is the synchronization between frame buffers.

[0037] The graphical user interface, that is, when the user needs to display an image on the frame buffer, the user will specify the size and position of the display image, and the user will prepare the graphic data; after the dirty rectangular area currently to be drawn is obtained according to the display image size and position specified by the user, the width and height of the dirty rectangular area are obtained from the image size, and the x and y values ​​of the image two-dimensional space are obtained from the image display position, where the corresponding origin of the coordinate system is the starting address of the frame buffer.

[0038] In step S4, the specific steps of updating the area after the R1 and R2 matrices are merged and deduplicated are as follows:

[0039] Each rectangle structure contains four parameters, namely the horizontal coordinate x, vertical coordinate y of the upper left corner point, and the width w and height h of the rectangle;

[0040] First, determine whether the two rectangles intersect; this can be determined by comparing the positions of the right border, left border, upper border, and lower border of the two rectangles;

[0041] If the right border of any rectangle is smaller than the left border of the other rectangle, or the left border is larger than the right border of the other rectangle, or the lower border is smaller than the upper border of the other rectangle, or the upper border is larger than the lower border of the other rectangle, then the two rectangles do not intersect and the original image before merging is directly returned;

[0042] If two rectangles intersect, the merged image can be represented by finding the union rectangle of the two rectangles; the x coordinate of the upper left corner of the new rectangle is the minimum of the x coordinates of the upper left corners of the two rectangles, the y coordinate of the upper left corner is the minimum of the y coordinates of the upper left corners of the two rectangles, the width of the rectangle is the maximum of the widths of the two rectangles, and the height of the rectangle is the maximum of the heights of the two rectangles;

[0043] Then, the overlapping area between the two rectangles needs to be removed; the deduplicated rectangle can be calculated by determining the positions of the boundaries of the two rectangles; the right boundary of the deduplicated rectangle is the minimum value of the right boundaries of the two rectangles, the left boundary is the maximum value of the left boundaries of the two rectangles, the lower boundary is the minimum value of the lower boundaries of the two rectangles, and the upper boundary is the maximum value of the upper boundaries of the two rectangles;

[0044] Finally, the deduplicated rectangles are output as the merged deduplicated image.

[0045] Therefore, the advantage of the present application is that by applying the solution of the present application, the requirements of multiple buffers for image merging between frame buffers can be met while using the dirty rectangle mechanism of the graphic user interface, thereby enhancing the effect of the graphic display of the frame buffer device. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0047] Figure 1 is a schematic diagram of an embodiment of the present method.

[0048] Figure 2 is a schematic diagram of the process of this method. DETAILED DESCRIPTION

[0049] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0050] This application provides a frame buffer merging method based on dirty rectangular area calculation, refer to Figure 1 As shown, the core method logic of the present application is described as follows: S1 to S6 in the figure here refer to the graphics memory operated by the corresponding steps, R1, R2, and R3 refer to the dirty rectangular areas input by the user, which represent a part of the graphics area corresponding to S1, S2, and S3 that needs to be updated. Further, as Figure 2 As shown:

[0051] Step S1. After the frame buffer device driver applies for reserved frame buffer, the method for the frame buffer device driver to apply for reserved frame buffer is to reserve frame buffer shared memory in the kernel display driver, with continuous physical addresses and hardware alignment; the function of reserving memory is the system function dma_alloc_coherent, and the parameter is the size of the memory; the number of reserved frame buffers is set to MAX_NUM;

[0052] Step S2. For synchronization between frame buffers of MAX_NUM, the formula must be satisfied:

[0053] Si=S(i-1)+Ri;

[0054] i = (i + 1)% MAX_NUM;

[0055] Where i represents the subscript value corresponding to the frame buffer of the current operation;

[0056] Si represents the graphics data value of the i-th frame buffer;

[0057] S(i-1) represents the graphics data value of the i-1th frame buffer;

[0058] Ri represents the graphic data that needs to be drawn currently;

[0059] After each drawing of a graphic data and synchronization between frame buffers, the current i value is incremented by one, and the incremented value is modulo MAX_NUM to ensure that the i value is within the frame buffer subscript range;

[0060] The inter-frame buffer synchronization is to avoid the waiting time of frame drawing by switching between multiple buffers during the graphics frame drawing process, so that data processing is smoother. At this time, when switching between multiple buffers, the drawing process of the previous frame buffer needs to be copied to avoid the loss of drawn data. This process is the inter-frame buffer synchronization;

[0061] Step S3. When the frame buffer MAX_NUM=N, SN=S(N-1)+RN; assuming MAX_NUM=3, that is, 3 frame buffers are reserved; when the graphical user interface, i.e., the user, needs to display an image on the frame buffer, the user will specify the size and position of the displayed image, and the user will prepare the graphic data. After the dirty rectangular area R1 to be drawn is obtained according to the size and position of the displayed image specified by the user, the width and height of the dirty rectangular area are obtained from the image size, and the x and y values ​​of the image two-dimensional space are obtained from the image display position. The corresponding origin of the coordinate system here is the starting address of the frame buffer; R1 can be updated to the frame buffer S1, and at this time i=1, i.e.

[0062] S1=R1;

[0063] Step S4. When the graphical user interface obtains the dirty rectangular area R2 to be drawn, it can be updated to the frame buffer S2, that is, the area after the R1 and R2 matrices are merged and deduplicated is updated, and R1xR2 represents the matrix merge and deduplication. At this time

[0064] S2 = R1 x R2;

[0065] The specific steps are:

[0066] Each rectangle structure contains four parameters, namely the horizontal coordinate x, vertical coordinate y of the upper left corner, and the width w and height h of the rectangle.

[0067] First, determine whether the two rectangles intersect. This can be determined by comparing the positions of the right, left, upper, and lower boundaries of the two rectangles. If the right boundary of any rectangle is smaller than the left boundary of the other rectangle, or the left boundary is larger than the right boundary of the other rectangle, or the lower boundary is smaller than the upper boundary of the other rectangle, or the upper boundary is larger than the lower boundary of the other rectangle, then the two rectangles do not intersect, and the original image before merging is directly returned.

[0068] If two rectangles intersect, the merged image can be represented by finding the union rectangle of the two rectangles. The x coordinate of the upper left corner of the new rectangle is the minimum of the x coordinates of the upper left corners of the two rectangles, the y coordinate of the upper left corner is the minimum of the y coordinates of the upper left corners of the two rectangles, the width of the rectangle is the maximum of the widths of the two rectangles, and the height of the rectangle is the maximum of the heights of the two rectangles.

[0069] Then, the overlapping area between the two rectangles needs to be removed. The deduplicated rectangle can be calculated by determining the positions of the boundaries of the two rectangles. The right boundary of the deduplicated rectangle is the minimum value of the right boundaries of the two rectangles, the left boundary is the maximum value of the left boundaries of the two rectangles, the lower boundary is the minimum value of the lower boundaries of the two rectangles, and the upper boundary is the maximum value of the upper boundaries of the two rectangles.

[0070] Finally, the deduplicated rectangles are output as the merged deduplicated image.

[0071] Step S5. When the GUI obtains the dirty rectangular area R3 to be drawn, it can be updated to the frame buffer S3.

[0072] S3 = R1xR2xR3;

[0073] Step S6. When there is no new dirty rectangular area in the graphical user interface, that is, when the user no longer draws a new image, there is no new dirty rectangular area input at this time. For the frame buffer device with the triple frame buffer vertical synchronization mechanism enabled, that is, the case of MAX_NUM=3 mentioned above and the case of MAX_NUM>1, it is still necessary to update the rectangular area in the record to the frame buffer S1. Here, the selection of updating the frame buffer S1, frame buffer S2, and frame buffer S3 is performed in sequence. After updating the frame buffer S3, the frame buffer S1 is updated again, and the cycle continues; however, the recorded rectangular area needs to comply with the effective number not exceeding MAX_NUM, so the R1 area is discarded from the record. Since MAX_NUM is set to 3, there are only three frame buffers in total. Therefore, when a rectangular area, such as the R1 area, is synchronized 3 times between the frame buffers, all frame buffers have updated the R1 area at this time, and the R1 area no longer needs to be synchronized and can be discarded. At this time, it is recorded as

[0074] S4 = S1 + R2 x R3;

[0075] Step S7. For a frame buffer device that enables the triple frame buffer vertical synchronization mechanism when there is no new dirty rectangular area, that is, the case where MAX_NUM=3, update the frame buffer S2 in sequence. Here, step S7 corresponds to the case where there is no new dirty rectangular area input. If there is a dirty rectangular area input, proceed normally according to step S4.

[0076] S5 = S2 + R3;

[0077] Step S8. For the frame buffer device with the triple frame buffer vertical synchronization mechanism enabled when there is no new dirty rectangular area, that is, the case where MAX_NUM=3, the frame buffer S3 is updated in sequence. Step S8 here corresponds to the case where there is no new dirty rectangular area input. If there is a dirty rectangular area input, the process is performed normally according to step S5.

[0078] S6=S3.

[0079] In summary, the key points of the method of the present application are the relevant steps of the frame buffer image merging method.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A frame buffer merging method based on dirty rectangular area calculation, characterized in that: The method comprises the following steps: Step S1. After the frame buffer device driver applies for reserved frame buffer, the number of reserved frame buffers is set to MAX_NUM; Step S2. For MAX_NUM frame buffer synchronization, the following formula is satisfied: Si=S(i-1)+Ri; i = (i + 1)% MAX_NUM; Where i represents the subscript value corresponding to the frame buffer of the current operation; Si represents the graphics data value of the i-th frame buffer; S(i-1) represents the graphics data value of the i-1th frame buffer; Ri represents the graphic data that needs to be drawn currently; After each drawing of a graphic data and synchronization between frame buffers, the current i value is incremented by one, and the incremented value is modulo MAX_NUM to ensure that the i value is within the frame buffer subscript range; Step S3. When the frame buffer MAX_NUM=N, SN=S(N-1)+RN; if MAX_NUM=3, that is, 3 frame buffers are reserved, then when the graphical user interface obtains the dirty rectangular area R1 to be drawn, R1 can be updated to the frame buffer S1, at this time i=1, that is, S1=R1; Step S4. When the graphical user interface obtains the dirty rectangular area R2 that needs to be drawn, R2 can be updated to the frame buffer S2, that is, the area after the R1 and R2 matrices are merged and deduplicated is updated, and R1xR2 represents the matrix merge and deduplication. At this time S2 = R1 x R2; Step S5. When the graphical user interface obtains the dirty rectangular area R3 that needs to be drawn, R3 can be updated to the frame buffer S3. S3 = R1xR2xR3; Step S6. When there is no new dirty rectangular area in the graphical user interface, that is, when the user no longer draws a new image, there is no new dirty rectangular area input at this time. For the frame buffer device with the triple frame buffer vertical synchronization mechanism enabled, that is, the case of MAX_NUM=3 and the case of MAX_NUM>1, it is still necessary to update the rectangular area in the record to the frame buffer S1. Here, the selection of updating the frame buffer S1, frame buffer S2, and frame buffer S3 is performed in sequence. After updating the frame buffer S3, the frame buffer S1 is updated again, and the cycle continues; the recorded rectangular area needs to comply with the effective number not exceeding MAX_NUM, so the R1 area is discarded from the record. Since MAX_NUM is set to 3, there are only three frame buffers in total. Therefore, when a rectangular area, here is the R1 area, is synchronized 3 times between the frame buffers, all frame buffers have updated the R1 area at this time, and the R1 area no longer needs to be synchronized and can be discarded; this is recorded as S4 = S1 + R2 x R3; Step S7. For the frame buffer device with triple frame buffer vertical synchronization mechanism enabled when there is no new dirty rectangular area, that is, the case of MAX_NUM=3, update the frame buffer S2 in sequence. S5 = S2 + R3; Here, step S7 corresponds to the case where there is no new dirty rectangular area input. If there is a dirty rectangular area input, the process is performed normally according to step S4; Step S8. For the frame buffer device with the triple frame buffer vertical synchronization mechanism enabled when there is no new dirty rectangular area, that is, the case where MAX_NUM=3, update the frame buffer S3 in sequence. S6 = S3; Here, step S8 corresponds to the case where no new dirty rectangular region is input. If a dirty rectangular region is input, the process proceeds normally according to step S5.

2. A frame buffer merging method based on dirty rectangular area calculation according to claim 1, characterized in that: In step S1, the method for the frame buffer device driver to apply for reserved frame buffer is to reserve frame buffer shared memory in the kernel display driver, with continuous physical addresses and hardware alignment; the function for reserving memory is the system function dma_alloc_coherent, and the parameter is the size of the memory.

3. The frame buffer merging method based on dirty rectangular area calculation according to claim 1, characterized in that: In step S2, the synchronization between frame buffers is to avoid the waiting time of frame drawing by switching between multiple buffers during the graphics frame drawing process, so as to make data processing smoother. At this time, when switching between multiple buffers, the drawing process of the previous frame buffer needs to be copied to avoid the loss of drawing data. This process is the synchronization between frame buffers.

4. The frame buffer merging method based on dirty rectangular area calculation according to claim 1, characterized in that: The graphical user interface, that is, when the user needs to display an image on the frame buffer, the user will specify the size and position of the display image, and the user will prepare the graphic data; after the dirty rectangular area currently to be drawn is obtained according to the display image size and position specified by the user, the width and height of the dirty rectangular area are obtained from the image size, and the x and y values ​​of the image two-dimensional space are obtained from the image display position, where the corresponding origin of the coordinate system is the starting address of the frame buffer.

5. The frame buffer merging method based on dirty rectangular area calculation according to claim 1, characterized in that: In step S4, the specific steps of updating the area after the R1 and R2 matrices are merged and deduplicated are as follows: Each rectangle structure contains four parameters, namely the horizontal coordinate x, vertical coordinate y of the upper left corner, and the width w and height h of the rectangle; First, determine whether the two rectangles intersect; this can be determined by comparing the positions of the right border, left border, upper border, and lower border of the two rectangles; If the right border of any rectangle is smaller than the left border of the other rectangle, or the left border is larger than the right border of the other rectangle, or the lower border is smaller than the upper border of the other rectangle, or the upper border is larger than the lower border of the other rectangle, then the two rectangles do not intersect and the original image before merging is directly returned; If two rectangles intersect, the merged image can be represented by finding the union rectangle of the two rectangles; the x coordinate of the upper left corner of the new rectangle is the minimum of the x coordinates of the upper left corners of the two rectangles, the y coordinate of the upper left corner is the minimum of the y coordinates of the upper left corners of the two rectangles, the width of the rectangle is the maximum of the widths of the two rectangles, and the height of the rectangle is the maximum of the heights of the two rectangles; Then, the overlapping area between the two rectangles needs to be removed; the deduplicated rectangle can be calculated by determining the positions of the boundaries of the two rectangles; the right boundary of the deduplicated rectangle is the minimum value of the right boundaries of the two rectangles, the left boundary is the maximum value of the left boundaries of the two rectangles, the lower boundary is the minimum value of the lower boundaries of the two rectangles, and the upper boundary is the maximum value of the upper boundaries of the two rectangles; Finally, the deduplicated rectangles are output as the merged deduplicated image.

Images