Visual anti-aliasing method, system and device based on Kalman filter and medium

Abstract

The invention relates to a visual anti-aliasing method, system and device based on a Kalman filter and a medium, and the method comprises the steps: carrying out the rasterization of a visual output image, and determining the RGB actual display value and measurement value of a pixel with a rasterization error; carrying out filtering processing on the RGB actual display value and the measured value by adopting a Kalman filter to obtain an optimal estimated value of the pixel; processing each pixel with the rasterization error of the visual output picture by adopting the above steps, and taking a processing result as an actual output pixel of the visual output picture; and adjusting parameters of a Kalman filter according to the actual output pixels. According to the method, the change rule of the color of the pixel point can be automatically predicted, so that each pixel can be converged to the optimal weight, the image can be directly processed on line, and manual drawing of multiple levels of textures and models is not needed.

Description

Technical Field

[0001] The present invention belongs to the field of virtual vision, and in particular relates to a vision anti-aliasing method, system, device and medium based on a Kalman filter. Background Art

[0002] In the visual system, flicker is a very difficult problem to avoid and solve. The fundamental reason for its occurrence is that when the visual image is output, the output image must be rasterized, and the continuous image information is rasterized into discrete pixels, while the current graphics card can only output 4k, 2k or even lower resolution (resolution represents the number of pixels). For tiny objects or lines in the picture, if there is an area smaller than one pixel in the imaging of the picture, then when the picture is rasterized, there will definitely be a rasterization error in this area, that is, the pixel can only be fully displayed or completely hidden. These pixels are sometimes displayed and sometimes hidden with the movement of the viewpoint, thus showing a flickering feeling. When there are a large number of sharp-edged lines in the picture, as the distance from the viewpoint increases, the lines gradually become thinner and smaller, and this phenomenon will occur repeatedly.

[0003] At present, the common solution is to perform Gaussian blur on the final rendered image, or use other anti-aliasing algorithms to make a smooth transition between the fine lines in the image and the surrounding pixels, thereby reducing the flickering phenomenon. However, this method directly operates on the pixels of the image and cannot distinguish between the foreground and the background, which can easily cause the overall clarity of the image to decrease. Another type of method, such as MIPMAP, LOD, etc., is based on the basic principle of pre-making a variety of models or textures with different precisions. When the distance to the viewpoint is close, high-precision models and textures are used. At this time, due to the close distance to the viewpoint, the lines on the model and texture require multiple pixels to describe, so there is no obvious flickering phenomenon; when the distance to the viewpoint becomes farther and farther, low-precision models and textures are loaded. These low-precision models and textures do not have too many details, but only large-scale contours, which also need to be drawn with multiple pixels to avoid flickering. Since the blur is set according to the viewpoint position distance, it is able to better preserve nearby details. However, the effectiveness of this method depends on the number of model accuracy levels. The finer the level division of the same model, the better the effect. This often requires the pre-production of multiple sets of textures and models, which not only consumes a lot of resources and has high requirements on computer performance, but also requires a lot of manual work to produce textures and models of various accuracy. Summary of the invention

[0004] In order to overcome the problems existing in the prior art, the present invention provides a Kalman filter-based visual anti-aliasing method, system, device and medium, which are used to overcome the current defects.

[0005] A Kalman filter-based visual anti-aliasing method, the method comprising the steps of:

[0006] S1. rasterizing the scene output image to determine the actual RGB display value and measurement value of the pixel with rasterization error;

[0007] S2. The actual RGB display value and the measured value are filtered using a Kalman filter to obtain the optimal estimated value of the pixel;

[0008] S3. Processing each pixel of the visual output screen having a rasterization error by using steps S1-S2, and using the processing result as the actual output pixel of the visual output screen;

[0009] S4. Adjusting the parameters of the Kalman filter according to the actual output pixels.

[0010] According to the aspects and any possible implementations described above, an implementation is further provided, wherein the pixel with rasterization error is formed when an object in the visual output image is smaller than a single pixel or the pixel is located at the intersection of multiple objects.

[0011] According to the above aspects and any possible implementation, an implementation is further provided, wherein the actual RGB display value of the pixel is the actual value of the texture of the object in the scene; and the RGB measurement value of the pixel is the value read by measurement.

[0012] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S2 specifically includes:

[0013] S21. Calculate the RGB predicted value of the pixel in the current frame using a Kalman filter according to the actual RGB display value of the pixel in the previous frame;

[0014] S22. Obtain an optimal RGB estimation value of the pixel in the current frame according to the RGB prediction value and the measurement value of the pixel in the current frame.

[0015] According to the above aspects and any possible implementation, an implementation is further provided, wherein the calculation formula in S21 is: in, is the predicted value of the current frame, x k-1 is the actual RGB display value of the pixel in the previous frame, u k is the adjustable control input of the K frame state, A is the frame pixel value parameter matrix, and B is the control input matrix.

[0016] According to the above aspects and any possible implementation, an implementation is further provided, wherein the calculation formula of the RGB optimal estimation value of the pixel is:

[0017] in, is the optimal RGB estimate of the pixel in the current frame, is the RGB prediction value of the pixel in the current frame, G g (k) is the Kalman gain matrix, Z k is the RGB measurement value of the pixel in the current frame, and H is the measurement matrix.

[0018] According to the above aspects and any possible implementation, a further implementation is provided, wherein the Kalman gain matrix G g The calculation formula of (k) is: Where R is the measurement noise covariance matrix of the current frame, is the covariance matrix of the current frame.

[0019] The present invention also provides a visual anti-aliasing system based on a Kalman filter, wherein the system implements the method described above and comprises the following modules:

[0020] A rasterization processing module is used to perform rasterization processing on the visual output picture and determine the RGB prediction value and measurement value of the pixel with rasterization error;

[0021] A filtering processing module, used for filtering the RGB prediction value and the measurement value using a Kalman filter to obtain the actual display value of the pixel;

[0022] A repeated processing module, used for performing the same processing on each pixel of the visual output picture with a rasterization error, and using the processing result as the actual output pixel of the visual output picture;

[0023] An adjustment module is used to adjust the parameters of the Kalman filter according to the actual output pixel.

[0024] The present invention further provides an electronic device, comprising:

[0025] A memory storing executable instructions;

[0026] A processor is used to execute the executable instructions in the memory to implement the method.

[0027] The present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method described.

[0028] Beneficial effects of the present invention

[0029] The visual anti-aliasing method based on Kalman filter of the present invention performs rasterization processing on the visual output picture, determines the RGB actual display value and measurement value of the pixel with rasterization error; uses Kalman filter to filter the RGB actual display value and measurement value to obtain the optimal estimated value of the pixel; uses the above steps to process each pixel with rasterization error in the visual output picture, and uses the processing result as the actual output pixel of the visual output picture; adjusts the parameters of Kalman filter according to the actual output pixel. It has the following beneficial effects: 1. Compared with the traditional convolution method, the Kalman filter of the present invention can automatically predict the color change law of the pixel point, so that each pixel point can converge to the optimal weight, that is, the weight of the display area proportion in a single pixel after rasterization; while the traditional convolution method requires manual setting of the convolution kernel, and the global convolution kernels of each pixel are consistent, and it is difficult to ensure that the convolution kernel is optimal at each pixel position.

[0030] 2. Compared with MIP and LOD, which require the pre-production of multiple models and textures, this method can directly process the image online without the need to manually draw multiple levels of textures and models. In addition, by controlling the parameters of the Kalman filter, the sharpening, blurring, contrast and other effects can be adjusted quickly, and the effect control is flexible.

[0031] 3. The present invention can achieve different scene effects by controlling the convergence speed of the Kalman filter. When the convergence speed of the Kalman filter is set to be faster, the image aliasing can be eliminated; when the convergence speed is set to be slower, real motion blur, smear and other effects can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of discrete pixels after rasterization of the visual system of the present invention;

[0033] Figure 2 is a schematic diagram of Kalman filtering of the present invention;

[0034] Figure 3 This is a schematic diagram of Kalman filtering under the linear change condition of the present invention;

[0035] Figure 4 It is a schematic diagram of Kalman filtering under nonlinear change conditions of the present invention. DETAILED DESCRIPTION

[0036] In order to better understand the technical solution of the present invention, the content of the present invention includes but is not limited to the specific implementation methods described below, and similar technologies and methods should be considered to be within the scope of protection of the present invention. In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0037] It should be clear that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] A Kalman filter-based visual anti-aliasing method of the present invention comprises the following steps:

[0039] S1. rasterizing the scene output image to determine the actual RGB display value and measurement value of the pixel with rasterization error;

[0040] S2. The actual RGB display value and the measured value are filtered using a Kalman filter to obtain the optimal estimated value of the pixel;

[0041] S3. Processing each pixel of the visual output screen having a rasterization error by using steps S1-S2, and using the processing result as the actual output pixel of the visual output screen;

[0042] S4. Adjusting the parameters of the Kalman filter according to the actual output pixels.

[0043] Preferably, the pixel with rasterization error is formed when an object in the visual output picture is smaller than a single pixel or the pixel is located at the intersection of multiple objects.

[0044] Preferably, the actual displayed RGB value of the pixel is the actual value of the texture of the object in the visual scene; the measured RGB value of the pixel is the value read by measurement.

[0045] Preferably, the S2 specifically includes:

[0046] S21. Calculate the RGB predicted value of the pixel in the current frame using a Kalman filter according to the actual RGB display value of the pixel in the previous frame;

[0047] S22. Obtain an optimal RGB estimation value of the pixel in the current frame according to the RGB prediction value and the measurement value of the pixel in the current frame.

[0048] Preferably, the calculation formula in S21 is: in, is the predicted value of the current frame, x k-1 is the actual RGB display value of the pixel in the previous frame, u k is the adjustable control input of the K frame state, A is the frame pixel value parameter matrix, and B is the control input matrix.

[0049] Preferably, the calculation formula for the RGB optimal estimation value of the pixel is:

[0050] in, is the optimal RGB estimate of the pixel in the current frame, is the RGB prediction value of the pixel in the current frame, G g (k) is the Kalman gain matrix, Z k is the RGB measurement value of the pixel in the current frame, and H is the measurement matrix.

[0051] Preferably, the Kalman gain matrix G g The calculation formula of (k) is: Where R is the measurement noise covariance matrix of the current frame, is the covariance matrix of the current frame.

[0052] Specifically, the present invention finds that the flicker is mainly caused by the truncation error during rasterization, presenting a relatively regular alternation of light and dark, and analyzes the flicker phenomenon generated when the pixel moves with the viewpoint, and filters it out through a Kalman filter. The real-time RGB value of each pixel in the picture that changes with time is regarded as a signal input of a Kalman filter, and then the Kalman filter is used to quickly predict the law of pixel change at the point, and the fitted RGB value is output. Finally, all the pixels that have passed the Kalman filter together constitute the picture to be output, thereby realizing anti-aliasing of the visual picture.

[0053] The specific implementation process of the present invention is as follows:

[0054] Step 1: Establish a Kalman filter. In the process of visual imaging, the output image must be rasterized to rasterize the continuous image information into discrete pixels. When the object is smaller than a single pixel or a pixel is at the intersection of multiple objects, there will be rasterization errors, that is, the object and the background appear alternately, and the vision presents a flickering feeling. Especially when the linear object is far away from the viewpoint, the flickering phenomenon is particularly obvious. Figure 1 As shown in the example, Figure 1 U 4 V 4 Represents a display pixel after rasterization. Figure 1 After rasterization, the triangle object becomes 3 to 4 pixels to represent the triangle object, where U 2 V 2 , U 3 V 2 , U 3 V 3 occupies almost all the area of ​​a single pixel, so there is no pixel change, but U 4 V 4 It occupies about half of the area of ​​a single pixel or the U4 V 4 The visual scene can be regarded as a real-time video screen, with more than 60 frames per second. Each frame will be rasterized into pixels for display. The U 4 V 4 , which indicates that the displayed pixel will appear in some frames and not in others. The reason for this is U 4 V 4 Half of the pixels represented are triangular objects and the other half are background objects interlaced with the triangular objects. When the background objects appear multi-colored, it may be because the color of the triangular objects is the majority. 4 V 4 It will appear, but when the background object is monochrome, the color of the triangle object accounts for a small proportion. 4 V 4 Then when the viewpoint moves, this pixel U 4 V 4 There will be a situation where the triangle object color and the background object color appear alternately, making U 4 V 4 Repeatedly hide and appear in the dynamic sequence frame. In this real-time display situation, a well-founded prediction is needed for the next display, that is, various interferences are accompanied during rasterization. Therefore, the present invention uses Kalman filtering to process, which can always point out the real situation in real time, and it also has the advantages of occupying small memory (except for the previous state quantity, no other historical data needs to be retained), and the speed is very fast, which is very suitable for real-time problems.

[0055] The visual scene is a simulated three-dimensional virtual scene, in which one can move in real time from a first-person perspective. The movement process consists of multiple frames. The previous frame of each frame is called the historical frame of this frame, and the current frame is called the current real-time frame. The current frame is random and moves in the visual scene. Each frame seen from the first-person perspective can be called the current real-time picture. The object to be studied in the present invention is the pixel value in the adjacent frames (i.e., the RGB actual value of a single pixel display). Assuming that the eye point moves in a single direction, the pixel value in the adjacent picture (frame) output according to the calculation speculation is constant, that is, the pixel value of the next frame is equal to the value of the previous frame. Of course, this process may have calculation deviations, and these deviations are regarded as Gaussian white noise, that is, these deviations have nothing to do with the two frames before and after and conform to Gaussian distribution. In addition, the actual pixel display value in the frame is directly read out using a test tool at the same time, and the actual pixel display value is used as a measured value. The RGB actual value of the texture of the object in the visual scene is used as a known value, and the low probability deviation between the actual pixel display value and the known value is also regarded as Gaussian white noise.

[0056] Under the premise of the above settings, there are now two RGB values ​​for a pixel in a certain frame: the actual RGB value and the RGB measurement value obtained by the test tool. The following will use these two values ​​combined with their respective noise to estimate the actual display value of the pixel. First, we need to introduce a discrete control process, which is described by a linear stochastic differential equation, and use the process model to predict the state of the next frame. Assuming that the current frame is k, according to the process model, the current state can be predicted based on the state of the previous frame. The state prediction equation (i.e., the process model) is established as follows:

[0057]

[0058] In formula 1, is the state RGB prediction value of the current frame obtained by using the prediction of the previous frame pixel, x k-1 is the actual RGB value of the previous frame, u k It is the control amount of the K frame state, which is a settable control input. If there is no control amount, it can be 0. A is the state transfer matrix of the frame pixel value, that is, the parameter matrix from the previous frame to the current frame, and B is the control input matrix. Both A and B are known and can be obtained.

[0059]

[0060] Formula 2 is the variance prediction equation, which is used to obtain the predicted variance covariance matrix of the current frame state yes The corresponding covariance matrix, p k-1 is x k-1 The corresponding covariance matrix, A T represents the transposed matrix of A, and Q is the process noise covariance matrix. Formulas 1 and 2 are Kalman filter formulas established by the present invention, which are used in the present invention to observe adjacent frames, and the prediction result of the current frame K is obtained through the above observations.

[0061] With the prediction result of the current frame k, we can collect the RGB measurement value of the current frame k. Combining the prediction value and measurement value of the current frame k, we can get the optimal estimate value of the current frame k.

[0062]

[0063]

[0064] In formula 3, according to the RGB measurement value Z of the current frame k k and the measurement matrix H to update the state prediction vector in Eq. Among them, formula 4 is the gain equation, which is used to obtain the Kalman gain matrix G g(k), where R is the measurement noise covariance matrix, and the measurement matrix

[0065] H is a matrix composed of some or all pixels in the current frame. The specific dimension is set according to actual needs. So far, the optimal estimate of the current k frame has been obtained. In order to keep the Kalman filter running until the end of the process, the covariance p of the k frame must be updated k :

[0066]

[0067] Where l is the identity matrix, G is G in Eq. g (k).

[0068] When the system enters the k+1 frame state, p k It is the variance covariance matrix of the current frame state predicted by formula 2 That is, use formula 5 to update the two-state variance covariance matrix In this way, the whole process can be operated in an autoregressive manner.

[0069] Step 2: Perform Kalman filtering on each pixel RGB of the real-time visual image, and output the result as the actual output pixel RGB of the image.

[0070] Combined with step 1, an example is given to illustrate the working process of the Kalman filter. If time k happens to show x k According to the description in step 1, the image is modeled based on the RGB values. Of course, the model does not need to be very accurate. In order for the Kalman filter to start working, it is necessary to tell the Kalman filter the initial values ​​of the two zero moments. and You don't need to worry about their specific values. You can give them any value. As the Kalman filter works, the above equations 1 to 5 are executed. X is the pixel matrix of a single frame, x k-1 It represents the matrix of all pixels in the picture at frame k-1, x k For all pixel matrices in the current frame, X will gradually converge, but the covariance P of X cannot be 0, otherwise it may make Kalman completely believe the given is the optimal system, so that the process cannot converge. For this reason, the present invention preferably selects The Kalman filter behaves like Figure 2 Shown is the optimization result of the Kalman filter output.

[0071] Step 3: Adjust the Kalman filter parameters according to the image improvement effect.

[0072] According to the filtering effect of step 2, that is, the output result, Bu in formula 1 k The control input matrix data value is adjusted as follows. In the case of formula 1, the pixel value is a linear change ax+b, that is, the pixel value of the previous frame picture and the pixel value of the current frame picture are linear. The fitting effect is as follows: Figure 3 As shown, it means that Bu is added to the RGB value of the pixel in the previous frame. k Control matrix, after step 2, Kalman filter operation is performed to obtain the RGB value of the current frame pixel, and so on, the RGB value of each frame pixel is obtained according to the control matrix. Step 3 modifies Bu k Control matrix value, in rolling Kalman filtering, the whole process presents a linear or nonlinear relationship, such as Figure 4 shown.

[0073] As an embodiment disclosed by the present invention, the present invention further discloses a visual anti-aliasing system based on a Kalman filter, wherein the system implements the method described, and comprises the following modules:

[0074] A rasterization processing module is used to perform rasterization processing on the visual output picture and determine the RGB prediction value and measurement value of the pixel with rasterization error;

[0075] A filtering processing module, used for filtering the RGB prediction value and the measurement value using a Kalman filter to obtain the actual display value of the pixel;

[0076] A repeated processing module, used for performing the same processing on each pixel of the visual output picture with a rasterization error, and using the processing result as the actual output pixel of the visual output picture;

[0077] An adjustment module is used to adjust the parameters of the Kalman filter according to the actual output pixel.

[0078] As an embodiment disclosed in the present invention, the present invention further discloses an electronic device, the electronic device comprising:

[0079] A memory storing executable instructions;

[0080] A processor is used to execute the executable instructions in the memory to implement the method described in the present invention.

[0081] As an embodiment disclosed in the present invention, the present invention further discloses a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the present invention.

[0082] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0083] The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the application concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A visual anti-aliasing method based on a Kalman filter, characterized in that: The method comprises the steps of: S1. rasterizing the scene output image to determine the actual RGB display value and measurement value of the pixel with rasterization error; S2. The actual RGB display value and the measured value are filtered using a Kalman filter to obtain the optimal estimated value of the pixel; S3. Processing each pixel of the visual output screen having a rasterization error by using steps S1-S2, and using the processing result as the actual output pixel of the visual output screen; S4. Adjusting the parameters of the Kalman filter according to the actual output pixels.

2. The visual anti-aliasing method based on Kalman filter according to claim 1, characterized in that: The pixel with rasterization error is formed when the object in the visual output picture is smaller than a single pixel or the pixel is located at the intersection of multiple objects.

3. The visual anti-aliasing method based on Kalman filter according to claim 2, characterized in that: The actual RGB display value of the pixel is the actual value of the texture of the object in the visual scene; the RGB measurement value of the pixel is the value read by measurement.

4. The visual anti-aliasing method based on Kalman filter according to claim 1, characterized in that: The S2 specifically includes: S21. Calculate the RGB predicted value of the pixel in the current frame using a Kalman filter according to the actual RGB display value of the pixel in the previous frame; S22. Obtain an optimal RGB estimation value of the pixel in the current frame according to the RGB prediction value and the measurement value of the pixel in the current frame.

5. The visual anti-aliasing method based on Kalman filter according to claim 4, characterized in that: The calculation formula in S21 is: in, is the predicted value of the current frame, x k-1 is the actual RGB display value of the pixel in the previous frame, u k is the adjustable control input of the K frame state, A is the frame pixel value parameter matrix, and B is the control input matrix.

6. The visual anti-aliasing method based on Kalman filter according to claim 5, characterized in that: The calculation formula for the optimal estimated value of RGB of the pixel is: in, is the optimal RGB estimate of the pixel in the current frame, is the RGB prediction value of the pixel in the current frame, G g (k) is the Kalman gain matrix, Z k is the RGB measurement value of the pixel in the current frame, and H is the measurement matrix.

7. The visual anti-aliasing method based on Kalman filter according to claim 6, characterized in that: The Kalman gain matrix G g The calculation formula of (k) is: Where R is the measurement noise covariance matrix of the current frame, is the covariance matrix of the current frame.

8. A visual anti-aliasing system based on a Kalman filter, characterized in that: The system implements the method described in any one of claims 1 to 7, and includes the following modules: A rasterization processing module is used to perform rasterization processing on the visual output picture and determine the RGB prediction value and measurement value of the pixel with rasterization error; A filtering processing module, used for filtering the RGB prediction value and the measurement value using a Kalman filter to obtain the actual display value of the pixel; A repeated processing module, used for performing the same processing on each pixel of the visual output picture with a rasterization error, and using the processing result as the actual output pixel of the visual output picture; An adjustment module is used to adjust the parameters of the Kalman filter according to the actual output pixel.

9. An electronic device, characterized in that: The electronic device comprises: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images