Kalman filter based visual anti-aliasing method, system, device and medium

Abstract

The application relates to a Kalman filter-based visual anti-aliasing method, system, device and medium, which comprises the following steps: rasterizing a visual output picture to determine the RGB actual display value and the measured value of a pixel with a rasterization error; filtering the RGB actual display value and the measured value by using a Kalman filter to obtain the optimal estimated value of the pixel; processing each pixel with a rasterization error of the visual output picture by using the foregoing step, and taking the processing result as the actual output pixel of the visual output picture; and adjusting the parameters of the Kalman filter according to the actual output pixel. The method can automatically predict the color change rule of the pixel point, so that each pixel can converge to the optimal weight, and the picture can be processed directly on line without wasting manpower to draw multiple hierarchical textures and models.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of virtual scenes, and particularly relates to a scene anti-aliasing method, system, device and medium based on a Kalman filter. BACKGROUND

[0002] In a scene system, flickering is a very difficult problem to avoid and solve. The root cause of the problem is that when a scene picture is output, the output picture must be rasterized to discretize continuous picture information into pixel points, and the current graphics card can only output a resolution (the resolution represents the number of pixels) of 4k, 2k or even lower. For a small object or line in a picture, if there is a region smaller than a pixel in the imaging of the object or line in the picture, there will be a rasterization error in the region when the picture is rasterized, that is, the pixel point can only be completely displayed or completely hidden. These pixel points will be displayed and hidden at times with the movement of the viewpoint, thereby presenting a flickering feeling. When there are a large number of sharp lines in a picture, the lines gradually become thin and small with the increase of the distance between the lines and the viewpoint position, and the phenomenon will repeatedly occur.

[0003] The most simple solution is to perform Gaussian blur on the final rendering picture or use other anti-aliasing algorithms to make the fine lines in the picture smoothly transition with the surrounding pixels, thereby reducing the flickering phenomenon. However, this method directly operates on the picture pixels and cannot distinguish the foreground and the background, and is likely to cause the overall clarity of the picture to decrease. Another method, such as MIPMAP and LOD, has the basic principle of pre-producing a plurality of models or textures of different precisions. When the distance from the viewpoint is close, the model and the texture of high precision are used. At this time, because the distance from the viewpoint is close, the lines on the model and the texture need to be described by a plurality of pixels, and therefore there is no obvious flickering phenomenon. When the distance from the viewpoint is farther and farther, the model and the texture of low precision are loaded. These low-precision models and textures have no too many details but only outlines, and also need to be drawn by a plurality of pixels to avoid the occurrence of flickering. Because the blur is set according to the distance from the viewpoint, the details in the near place can be better maintained. However, the effect of this method depends on the number of model precision levels. The finer the division of the same model level, the better the effect. This often requires a plurality of sets of textures and models to be pre-produced. The resource occupation is large, the computer performance requirement is high, and a large amount of manual work is required to produce textures and models of various precisions. SUMMARY

[0004] In order to overcome the problems in the prior art, the present application provides a scene anti-aliasing method, system, device and medium based on a Kalman filter, which is used to overcome the existing defects.

[0005] A scene anti-aliasing method based on a Kalman filter, the method comprising the steps of:

[0006] S1. rasterizing the visual output picture to determine the RGB actual display value and the measured value of the pixel with rasterization error;

[0007] S2. filtering the RGB actual display value and the measured value by using a Kalman filter to obtain the optimal estimated value of the pixel;

[0008] S3. processing each pixel with rasterization error of the visual output picture by using steps S1-S2, and taking the processing result as the actual output pixel of the visual output picture;

[0009] S4. adjusting the parameters of the Kalman filter according to the actual output pixel.

[0010] According to the aspect and any possible implementation manner above, further provided is an implementation manner, wherein the pixel with rasterization error is formed when an object in the visual output picture is smaller than a single pixel or the pixel is at an intersection of multiple objects.

[0011] According to the aspect and any possible implementation manner above, further provided is an implementation manner, wherein the RGB actual display value of the pixel is an actual value of a texture of the object in the visual output picture; and the measured value of the RGB of the pixel is a value read by measurement.

[0012] According to the aspect and any possible implementation manner above, further provided is an implementation manner, wherein S2 specifically comprises:

[0013] S21. calculating the RGB predicted value of the pixel in the current frame by using the Kalman filter according to the RGB actual display value of the pixel in the previous frame;

[0014] S22. obtaining the RGB optimal estimated value of the pixel in the current frame according to the RGB predicted value and the measured value of the pixel in the current frame.

[0015] According to the aspect and any possible implementation manner above, further provided is an implementation manner, wherein the calculation formula in S21 is: wherein, is the predicted value of the current frame, x k-1 is the RGB actual display value of the pixel in the previous frame, u k is the adjustable control input of the K-frame state, A is a frame pixel value parameter matrix, and B is a control input matrix.

[0016] According to the aspect and any possible implementation manner above, further provided is an implementation manner, wherein the calculation formula of the RGB optimal estimated value of the pixel is:

[0017] wherein, is the optimal estimation of RGB of the pixel of the current frame, is the prediction of RGB of the pixel of the current frame, G g (k) is the Kalman gain matrix, Z k is the measurement of RGB of the pixel of the current frame, H is the measurement matrix.

[0018] As the aspects and any possible implementation manners described above, further provided is an implementation manner, the calculation formula of the Kalman gain matrix G g (k) is as follows: wherein, R is the measurement noise covariance matrix of the current frame, is the covariance matrix of the current frame.

[0019] The application further provides a vision anti-aliasing system based on a Kalman filter, which implements the method and comprises the following modules:

[0020] A rasterization processing module is configured to perform rasterization processing on a vision output picture, and determine the prediction and measurement of RGB of a pixel with rasterization error;

[0021] A filtering processing module is configured to perform filtering processing on the prediction and measurement of RGB by using a Kalman filter to obtain the actual display value of the pixel;

[0022] A repeating processing module is configured to perform the same processing on each pixel with rasterization error of the vision output picture, and take the processing result as the actual output pixel of the vision output picture;

[0023] An adjusting module is configured to adjust the parameters of the Kalman filter according to the actual output pixel.

[0024] The application further provides an electronic device, which comprises:

[0025] A memory storing executable instructions;

[0026] A processor configured to execute the executable instructions in the memory to implement the method.

[0027] The application further provides a computer storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method.

[0028] Advantages of the application

[0029] The Kalman filter-based anti-aliasing method of the present application carries out rasterization processing on a visual output picture, determines the RGB actual display value and measured value of a pixel with rasterization error; carries out filtering processing on the RGB actual display value and measured value by using a Kalman filter to obtain the optimal estimated value of the pixel; carries out processing on each pixel with rasterization error of the visual output picture by using the foregoing step, and takes the processing result as the actual output pixel of the visual output picture; and adjusts the parameters of the Kalman filter according to the actual output pixel. The present application has the following beneficial effects: 1. Compared with the traditional convolution method, the Kalman filter of the present application can automatically predict the variation law of the color of the pixel point, so that each pixel point can converge to the optimal weight, i.e. the weight of the display area in the single pixel after rasterization. The traditional convolution method needs to manually set the convolution kernel, and the convolution kernel of each pixel is consistent, which makes it difficult to ensure that the convolution kernel is optimal at each pixel position.

[0030] 2. Compared with MIP and LOD, which need to pre-prepare multiple sets of models and textures, the present method can directly process the picture online without wasting manpower to draw multiple levels of textures and models. In addition, by controlling the parameters of the Kalman filter, the adjustment of sharpening, blurring, contrast and other effects can be quickly realized, and the effect control is flexible.

[0031] 3. By controlling the convergence speed of the Kalman filter, the present application can realize different scene special effects. When the convergence speed of the Kalman filter is set to be fast, the picture aliasing can be eliminated; when the convergence speed is set to be slow, the effects of real motion blur and trailing can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a schematic diagram of a discrete pixel after rasterization of the visual system of the present application;

[0033] Figure 2 is a Kalman filtering schematic diagram of the present application;

[0034] Figure 3 is a Kalman filtering schematic diagram under linear variation of the present application;

[0035] Figure 4 is a Kalman filtering schematic diagram under non-linear variation of the present application. DETAILED DESCRIPTION

[0036] In order to better understand the technical solutions of the present application, the present application includes but is not limited to the specific embodiments described below, and similar technologies and methods should be considered as falling within the scope of protection of the present application. In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail in conjunction with the drawings and specific embodiments.

[0037] It should be noted that the embodiments described in the present application are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0038] The method comprises the steps of:

[0039] S1. rasterizing the visual output picture to determine the RGB actual display value and the measured value of the pixel with rasterization error;

[0040] S2. filtering the RGB actual display value and the measured value by using the Kalman filter to obtain the optimal estimated value of the pixel;

[0041] S3. processing each pixel with rasterization error of the visual output picture by using steps S1-S2, and taking the processing result as the actual output pixel of the visual output picture;

[0042] S4. adjusting the parameters of the Kalman filter according to the actual output pixel.

[0043] Preferably, the pixel with rasterization error is formed when the object in the visual output picture is smaller than a single pixel or the pixel is at the intersection of multiple objects.

[0044] Preferably, the RGB actual display value of the pixel is the actual value of the object texture in the visual; and the measured value of the RGB of the pixel is the value read by measurement.

[0045] Preferably, S2 specifically comprises:

[0046] S21. calculating the RGB predicted value of the pixel in the current frame by using the Kalman filter according to the RGB actual display value of the pixel in the previous frame;

[0047] S22. obtaining the RGB optimal estimated value of the pixel in the current frame according to the RGB predicted value and the measured value of the pixel in the current frame.

[0048] Preferably, the calculation formula in S21 is: wherein, is the predicted value of the current frame, x k-1 is the RGB actual display value of the pixel in the previous frame, u k is the adjustable control input of K frame states, A is the frame pixel value parameter matrix, and B is the control input matrix.

[0049] Preferably, the calculation formula of the RGB optimal estimation value of the pixel is:

[0050] wherein, is the RGB optimal estimation value of the pixel of the current frame, is the RGB prediction value of the pixel of the current frame, G g (k) is the Kalman gain matrix, Z k is the RGB measurement value of the pixel of the current frame, and H is the measurement matrix.

[0051] Preferably, the calculation formula of the Kalman gain matrix G g (k) is: wherein, R is the measurement noise covariance matrix of the current frame, is the covariance matrix of the current frame.

[0052] Specifically, the present application finds that the flicker is mainly caused by the truncation error in gridding, and presents a relatively regular light and dark alternation. The flicker phenomenon generated when the pixel point moves with the view point is analyzed, and is filtered through a Kalman filter. The real-time RGB value of each pixel point in the picture changing with time is regarded as a signal input of a Kalman filter, then the Kalman filter is used to quickly predict the change rule of the pixel point, and output the fitted RGB value. Finally, all the pixel points filtered by the Kalman filter jointly constitute the picture to be output, so that the anti-aliasing of the view picture is realized.

[0053] The specific implementation process of the present application is as follows:

[0054] Step one: establishing a Kalman filter. In the process of view imaging, the output picture must be gridded, and the continuous picture information is gridded into discrete pixels. When the object is smaller than a single pixel or a pixel is at the intersection of multiple objects, there will be gridding error, that is, the object and the background appear alternately, and the visual presents a flickering feeling. Especially when the line-shaped object is far away from the view point, the flickering phenomenon is particularly obvious. Figure 1 As shown in the figure, wherein Figure 1 U4V4 in the figure represents a display pixel after gridding, Figure 1The middle triangle object becomes 3 to 4 pixels after rasterization to represent the triangle object, wherein U2V2, U3V2, U3V3 occupy almost all the area of a single pixel, thus, there is no pixel change, but U4V4 occupies about half of the area of a single pixel or is formed at the intersection of multiple triangle objects. The view picture can be regarded as a video picture played in real time, more than 60 pictures per second, each picture is rasterized into pixels for display, and the display pixel represented by U4V4 formed by rasterization of the triangle object in different pictures (i.e. frames) appears in some frames and does not appear in some frames, which is caused by the fact that half of the pixel represented by U4V4 is the triangle object and the other half is the background object intersecting the triangle object. When the background object has multiple colors, U4V4 may appear due to the majority of the triangle object color, but when the background object is single-colored, U4V4 may not appear due to the small proportion of the triangle object color. Then, when the viewpoint moves, the pixel U4V4 appears to represent the triangle object color and the background object color alternately, causing U4V4 to repeatedly hide and appear in the dynamic sequence frames. In this real-time display case, the next display needs to be predicted based on the basis, that is, various disturbances occur during rasterization. Therefore, the Kalman filter is used for processing, which can always indicate the real situation in real time, has the advantage of small memory occupation (except for the previous state quantity, no other historical data needs to be retained), and is fast, which is very suitable for real-time problems.

[0055] The view is a simulated three-dimensional virtual scene, in which the first-person perspective can be moved in real time, and the movement process is composed of multiple frames. The previous frame of each frame is called the history picture of the frame, and the current frame is called the real-time picture. Each frame seen from the first-person perspective can be called the real-time picture. The object to be studied in the present application is the pixel value (i.e. the actual RGB value of a single pixel display) in adjacent frames. Assuming that the eye point moves in a single direction, the pixel value in the adjacent picture (frame) is constant according to the calculation and prediction output, that is, the pixel value of the next frame is equal to the value of the previous frame. Of course, there may be calculation deviations in this process, which are regarded as Gaussian white noise, that is, these deviations have nothing to do with the previous and next two frames and conform to Gaussian distribution. In addition, a test tool is used to directly read the actual pixel display value in the frame, which is taken as the measured value, and the RGB actual value of the object texture in the view is taken as the known value. The low-probability deviation between the actual pixel display value and the known value is also regarded as Gaussian white noise.

[0056] Under the above setting, there are two RGB values for a pixel in a frame: the actual RGB value and the RGB measurement value obtained by the test tool. The actual display value of the pixel is estimated by combining the two values with their respective noises. First, a discrete control process is introduced, which is described by a linear stochastic differential equation. The process model is used to predict the state of the next frame. Assuming that the current frame is k, the present state can be predicted based on the state of the previous frame according to the process model. The state prediction equation (i.e., the process model) is established as follows:

[0057]

[0058] In equation one, is the predicted state RGB value of the current frame obtained by the prediction of the pixel of the previous frame, x k-1 is the actual RGB value of the previous frame, u k is the control quantity of the Kth frame state, which is a settable control input. If there is no control quantity, it can be 0. A is the state transition matrix of the frame pixel value, i.e., the parameter matrix from the previous frame to the current frame. B is the control input matrix. Both A and B are known and can be obtained.

[0059]

[0060] Equation two is the variance prediction equation, which is used to obtain the predicted state variance covariance matrix of the current frame is the corresponding covariance matrix, p k-1 is x k-1 the corresponding covariance matrix, A T denotes the transpose matrix of A, and Q is the process noise covariance matrix. Equations one and two are the Kalman filter formulas established in the present application, which are used to observe adjacent frames to obtain the prediction result of the current frame K through the above observation.

[0061] With the prediction result of the current frame k, the RGB measurement value of the current frame k is collected. By combining the prediction value and the measurement value of the current frame k, the optimal estimated value of the current frame k can be obtained

[0062]

[0063]

[0064] In equation three, the state prediction vector x k in equation one is updated according to the RGB measurement value Z of the current frame k and the measurement matrix H g(k), where R is the measurement noise covariance matrix, and the measurement matrix

[0065] H is a matrix composed of part or all of the pixels in the current frame, and the dimensions are set according to actual needs. So far, the optimal estimate of the current k frame has been obtained In order to make the Kalman filter run continuously until the end of the process, the covariance p of the k frame is also updated k :

[0066]

[0067] where l is the unit matrix, and G is G in equation four g (k).

[0068] When the system enters the k+1 frame state, p k is the predicted current frame state variance covariance matrix of equation two That is, the state variance covariance matrix of equation two is updated by equation five In this way, the entire process can be operated recursively.

[0069] Step two: Kalman filtering is performed on each pixel RGB of the real-time view picture, and the output result is taken as the actual output pixel RGB of the picture.

[0070] In combination with step one, an example is given to illustrate the working process of the Kalman filter. Suppose that the x k picture is displayed at time k, according to the description of step one, the picture is modeled according to the RGB value, of course, the model does not need to be very accurate. In order to start the Kalman filter, two initial values at time zero are needed to be told to the Kalman and The specific values do not need to be too concerned, and an arbitrary value can be given, because with the working of the Kalman filter, the running of equations one to five, X is the pixel matrix of a single frame picture, x k-1 represents the pixel matrix of all pixels in the k-1 frame, and x k is the pixel matrix of all pixels in the current frame. X will gradually converge, but the covariance P of X cannot be taken as 0, otherwise in the case of 0, it may make the Kalman completely believe that the given is the optimal system, so that the process cannot converge. Therefore, the Kalman filter is preferentially selected The performance of the Kalman filter is shown in the following table. Figure 2 The optimization result output by the Kalman filter is shown in the following table.

[0071] Step three: adjust the parameters of the Kalman filter according to the picture improvement effect.

[0072] According to the filtering effect of step two, that is, the output result, Bu k The control input matrix data value is adjusted as follows, and the pixel value in the case of formula one is linearly changed as ax+b, that is, linear between the pixel value of the previous frame and the pixel value of the current frame, and the fitting effect is as shown in Figure 3 It is shown that Bu k The control matrix is subjected to Kalman filtering operation in step two to obtain the current frame pixel RGB value, and in this way, the pixel RGB value of each frame is obtained according to the control matrix, and step three modifies Bu k The control matrix value, and the whole process presents a linear or nonlinear relationship when the Kalman filter is rolled, as shown in Figure 4 .

[0073] As an embodiment disclosed by the present application, the present application further discloses a visual anti-aliasing system based on a Kalman filter, which realizes the method and comprises the following modules:

[0074] A rasterization processing module is configured to perform rasterization processing on the visual output picture to determine the RGB predicted value and the measured value of the pixel with rasterization error;

[0075] A filtering processing module is configured to filter the RGB predicted value and the measured value by using a Kalman filter to obtain the actual display value of the pixel;

[0076] A repeated processing module is configured to perform the same processing on each pixel with rasterization error of the visual output picture, and take the processing result as the actual output pixel of the visual output picture;

[0077] An adjustment module is configured to adjust the parameters of the Kalman filter according to the actual output pixel.

[0078] As an embodiment disclosed by the present application, the present application further discloses an electronic device, which comprises:

[0079] A memory storing executable instructions;

[0080] A processor running the executable instructions in the memory to realize the method of the present application.

[0081] As an embodiment disclosed by the present application, the present application further discloses a computer storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method of the present application.

[0082] The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the description of the application and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0083] The foregoing description illustrates and describes the only preferred embodiments of the application, but as mentioned above, it is to be understood that the application is not limited to the disclosed embodiments, and is not to be limited to the specific embodiments described and should not be limited to only the embodiments set forth herein, but should be given the fullest range of equivalents usually associated with claims of patents. Changes and modifications can be made in the application in accordance with the teachings of the foregoing description and accompanying drawings, within the skill or knowledge of those in the art, from the nature of the application. The application is therefore intended to cover any and all changes and modifications within the scope of the inherent principles of the application.

Claims

1. A Kalman filter based visual anti-aliasing method, characterized in that, The method comprises the steps of: S1. rasterizing the visual output picture to determine the RGB actual display value and measured value of a pixel with rasterization error, which is formed when an object in the visual output picture is smaller than a single pixel or the pixel is at the intersection of multiple objects; S2. filtering the RGB actual display value and measured value using a Kalman filter to obtain the optimal estimated value of the pixel; specifically comprising: S21. calculating the RGB predicted value of the pixel in the current frame using a Kalman filter according to the RGB actual display value of the pixel in the previous frame; The state prediction equation is established as follows: wherein, is a state RGB prediction value of the current frame obtained by prediction of the previous frame pixel, is an actual RGB display value of the previous frame, is a control quantity of K frames of state, A is a parameter matrix from the previous frame to the current frame, and B is a control input matrix, both A and B are known; The variance prediction equation is established as follows: , for obtaining a predicted current frame state variance covariance matrix , which is a corresponding covariance matrix, denotes the transpose matrix of A, Q is a process noise covariance matrix, and a prediction result of a current frame K is obtained through the above observation; combining the prediction value of the current frame k and the collected RGB measurement value of the current frame k, obtaining an optimized estimation value of the current frame k : According to the RGB measurements of the current frame k and the measurement matrix to update the state prediction vector , is the Kalman gain matrix, the measurement matrix is a matrix composed of part or all of the pixels in the current frame k, and the covariance of the current frame k is updated as : where l is the identity matrix, and G is ; S3. processing each pixel with rasterization error of the visual output picture using steps S1-S2, and taking the processing result as the actual output pixel of the visual output picture; S4. adjusting the parameters of the Kalman filter according to the actual output pixel.

2. The Kalman filter based visual anti-aliasing method of claim 1, wherein, The RGB actual display value of the pixel is the actual value of the object texture in the visual; and the measured value of the RGB of the pixel is the value read by measurement.

3. The Kalman filter based visual anti-aliasing method of claim 1, wherein, The Kalman gain matrix The calculation formula is: Wherein, R is the measurement noise covariance matrix of the current frame, is the covariance matrix of the current frame.

4. A Kalman filter based visual anti-aliasing system, characterized by The system implements the method of any one of claims 1-3, and comprises the following modules: A rasterization processing module, configured to rasterize the visual output picture to determine the RGB predicted value and measured value of a pixel with rasterization error; A filtering processing module, configured to filter the RGB predicted value and measured value using a Kalman filter to obtain the actual display value of the pixel; A repeated processing module, configured to process each pixel with rasterization error of the visual output picture in the same way, and take the processing result as the actual output pixel of the visual output picture; An adjusting module, configured to adjust the parameters of the Kalman filter according to the actual output pixel.

5. An electronic device, comprising: The electronic device comprises: A memory storing executable instructions; A processor running the executable instructions in the memory to implement the method of any one of claims 1-3.

6. A computer storage medium, characterized in that The medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of claims 1-3.

Images