Picture processing method, electronic equipment, computer storage medium and computer program product

Abstract

The embodiment of the invention provides a picture processing method, electronic equipment, a computer storage medium and a computer program product. The picture processing method comprises the following steps: in a virtual shooting process, carrying out picture-based pixel flicker analysis on a to-be-presented picture on a screen for presenting a virtual scene; based on a pixel flicker analysis result, determining flicker pixels to be processed in the picture; and performing flicker removal processing on the to-be-processed flicker pixels, and presenting a picture subjected to flicker removal processing on a screen. Through the embodiment of the invention, the quality of the to-be-presented picture on the screen for presenting the virtual scene is improved, and the quality of the video obtained through virtual shooting is further improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of virtual shooting, and in particular to a method for processing images in virtual shooting, an electronic device, a computer storage medium, and a computer program product. Background Art

[0002] With the rapid development of VR (Virtual Reality), AR (Augmented Reality) and MR (Mixed Reality) technologies, virtual shooting, as an emerging film and television production method integrating multiple advanced technologies, has been widely used in film and television, games, advertising and other fields.

[0003] Virtual shooting is a technology that uses a "virtual" virtual scene as the background and a "real" actual object as the foreground for shooting. In this technology, using a screen as an intermediary to achieve virtual shooting is one of the main ways. For example, LED virtual shooting technology, which uses an LED (Light Emitting Diode) screen as an intermediary, uses a virtual scene in a high-resolution LED screen as the background, combined with real-time image processing technology, to combine virtual scenes with real shooting. In LED virtual shooting, assuming that the actual foreground object is an actor, the actor is placed in front of the virtual scene projected by the LED screen, and processed through computer graphics technology, which can make the actor's movements, expressions and virtual scenes merge, making the shooting background more realistic. The implementation of other virtual shooting technologies using screens as intermediaries is also similar to LED virtual shooting.

[0004] However, in the actual virtual shooting process, the screen used to present the virtual scene, i.e. the background, often flickers due to various reasons. This phenomenon not only affects the quality of the shot picture, but also affects the quality of the final virtual shot video, which has a negative impact on the audience's viewing experience. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a picture processing solution to at least partially solve the above-mentioned problem.

[0006] According to a first aspect of an embodiment of the present application, a picture processing method is provided, including:

[0007] During the virtual shooting process, a picture to be presented on a screen for presenting a virtual scene is subjected to a pixel flicker analysis based on the picture; based on the result of the pixel flicker analysis, flicker pixels to be processed in the picture are determined; flicker removal processing is performed on the flicker pixels to be processed, and the picture after flicker removal processing is presented on the screen.

[0008] According to the second aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect.

[0009] According to a third aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0010] According to a fourth aspect of an embodiment of the present application, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method described in the first aspect.

[0011] According to the solution provided by the embodiment of the present application, in the method of presenting a virtual scene through a screen, or projecting a virtual scene through a screen as a shooting background to realize virtual shooting, a picture to be presented on the screen for presenting the virtual scene is subjected to a picture-based pixel flicker analysis, and based on the result of the pixel flicker analysis, flicker pixels to be processed in the picture are determined, and flicker removal processing is then performed. As a result, the flicker pixels of the picture are processed before being presented on the screen, so that when the picture is presented on the screen, the picture presented is a picture after flicker removal processing, that is, there are no flicker pixels in the picture, but the picture quality is relatively high, the picture presentation is relatively stable, and the presentation effect is relatively good. On this basis, the quality of the video obtained by virtual shooting based on the picture presented on the screen is also higher.

[0012] Compared to the traditional method of performing flicker removal processing through hardware, the solution of the embodiment of the present application does not require the addition of additional hardware, or the need to modify the hardware, which not only improves the quality of the presented image, but also reduces the implementation cost of the flicker removal processing; compared to the traditional method of performing flicker removal processing through complex rendering algorithms, the implementation of the solution of the embodiment of the present application is simpler, more efficient, and has better versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] Figure 1A A schematic diagram of an exemplary system applicable to the embodiment of the present application;

[0015] Figure 1B A schematic diagram of another exemplary system applicable to the embodiment of the present application;

[0016] Figure 2 is a flowchart of a method for processing an image according to an embodiment of the present application;

[0017] Figure 3A is a schematic diagram of a presentation trajectory of a pixel in a two-dimensional coordinate system according to an embodiment of the present application;

[0018] Figure 3B is a schematic diagram of a presentation trajectory of a second type of pixel in a two-dimensional coordinate system according to an embodiment of the present application;

[0019] Figure 3C is a schematic diagram of a presentation trajectory of a third type of pixel in a two-dimensional coordinate system according to an embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0022] The following, combined Figure 1A and Figure 1B , a virtual shooting scene and an exemplary system applicable to the embodiment scheme of the present application are described.

[0023] As mentioned above, virtual shooting technology has both "real" and "virtual". Among them, "real" refers to the physical screen, physical camera, etc. in the actual environment during the virtual shooting process. The "virtual" is a virtual scene built based on shooting needs. The virtual scene can be presented through a physical screen, or it can be projected onto a curtain or wall with the physical screen as the medium. During the virtual shooting process, actors in the physical environment can perform in front of a physical screen, curtain, wall, etc. with a virtual background, and be filmed by a physical camera, thereby realizing the combination of "virtual" and "real".

[0024] For example, refer to Figure 1A , shows an exemplary system applicable to the embodiment of the present application. Figure 1AAs shown, the system 100A may include: a physical camera 102A and a screen 104A for presenting a virtual scene. Among them, the picture presented by the screen 104A is a picture of the pre-constructed virtual scene under the shooting angle of the virtual camera. It should be clear to those skilled in the art that in the embodiment of the present application, there is no restriction on the positional relationship between the physical camera 102A and the screen 104A, and the physical camera 102A can better capture the actual object and the complete or partial area of ​​the virtual scene presented by the screen 104A. The actual object, such as an actor, performs activities in front of the screen 104A, with the virtual scene presented by the screen 104A as the background, and is photographed by the physical camera 102A to achieve effective "virtual" and "real" fusion, forming an effect similar to the actual object's activities in a real scene. Exemplarily, the screen 104A in this example can be implemented as an LED screen.

[0025] For the screen 104A presenting the virtual scene, when presenting the picture of the virtual scene, there may be a phenomenon of flickering pixels in the picture, so that the physical camera 102A also captures the phenomenon into the video frame, resulting in poor video presentation quality. For this reason, in an embodiment of the present application, before the picture of the virtual scene is presented on the screen 104A, the flickering pixels in the picture to be presented are processed, so that the picture presented on the screen 104A is a picture after flicker removal processing. That is, there are no more flickering pixels in the picture, and on this basis, the quality of the video frame obtained by the physical camera 102A based on the picture presented on the screen 104A through virtual shooting is also higher.

[0026] However, it is not limited thereto. In another feasible exemplary system applicable to the embodiment of the present application, for example, Figure 1B As shown, the system 100B includes: a physical camera 102B, a screen 104B for projecting a virtual scene, and a foreground curtain 106B. Among them, the foreground curtain 106B can be located at any position that can receive the virtual scene projected by the screen 104B, and can better adapt to a more complex shooting environment. It should be clear to those skilled in the art that in the embodiment of the present application, the foreground curtain is used to represent any medium that can present a virtual scene. In practical applications, it includes but is not limited to the form of a curtain, and can also be in other forms, such as a wall or a large screen, etc., which are all applicable to the solution of the embodiment of the present application. The positional relationship between the physical camera 102B and the foreground curtain 106B is also not limited. The physical camera 102B only needs to be able to better capture the actual object and the entire or partial area of ​​the virtual scene presented by the foreground curtain 106B. The actual object, such as an actor, performs activities in front of the foreground curtain 106B, and the virtual scene presented by the foreground curtain 106B is used as the background, and is photographed by the physical camera 102B to achieve effective "virtual" and "real" fusion, forming an effect similar to the actual object in the real scene.

[0027] In this example, the image of the virtual scene presented on the screen 104B to be projected onto the foreground curtain 106B may have flickering pixels in the image. Therefore, the image of the virtual scene presented on the foreground curtain 106B may also have flickering pixels, causing the physical camera 102B to also capture the phenomenon into the video frame, resulting in poor video presentation quality. For this reason, in the embodiment of the present application, before the image of the virtual scene is presented on the screen 104B, the flickering pixels in the image to be presented are processed, so that the image presented on the screen 104B is a image after flicker removal processing, and then the screen 104B projects the image of the virtual scene after flicker removal processing onto the foreground curtain 106B. That is, there are no more flickering pixels in the image, and on this basis, the quality of the video frame obtained by the physical camera 102B through virtual shooting based on the image presented on the foreground curtain 106B is also higher.

[0028] Based on the above system, the solution provided by the embodiments of the present application is described below through multiple embodiments.

[0029] Reference Figure 2 , shows a step flow chart of a picture processing method according to an embodiment of the present application.

[0030] The picture processing method of this embodiment is used for a virtual shooting scene, and comprises the following steps:

[0031] Step S202: During the virtual shooting process, a picture to be presented on the screen for presenting the virtual scene is subjected to picture-based pixel flicker analysis.

[0032] In the embodiment of the present application, the screen used to present the virtual scene may be any appropriate screen capable of rendering and presenting images, for example, Figure 1A 104A, or as shown in Figure 1B Screen 104B etc. shown in FIG.

[0033] Whether the screen is used directly as a background presentation carrier or the screen image is projected onto other media, the image needs to be presented on the screen, and the basis of image presentation is pixels. Due to the influence of hardware or software performance, some pixels in the image often flicker, affecting the image presentation effect and ultimately affecting the presentation quality of the video formed by virtual shooting. Therefore, it is necessary to identify flickering pixels and then remove the flicker to improve the image presentation quality.

[0034] Based on this, before a frame of picture is displayed on the screen, this step performs picture-based pixel flicker analysis on it.

[0035] Taking the use of UE (Unreal Engine) for image rendering as an example, first load the models and materials that constitute the virtual scene to build the virtual scene, and set the position of the virtual camera to realize the framing of the virtual scene. Among them, the virtual camera is a device that simulates the working principle of a real camera and can capture objects in the virtual scene. After setting the focal length and other parameters of the virtual camera and determining the position and orientation of the virtual camera, the virtual scene being rendered can be previewed in real time through the shooting angle of the virtual camera during the operation of UE. Furthermore, objects that are not in the current viewing angle range are removed; the visibility of objects in the current viewing angle range is determined, and invisible objects are removed, that is, objects that are blocked in the current viewing angle range are removed. In the image processor, the 3D (Three Dimensions) coordinate system is mapped to the 2D (Two Dimensions) coordinate system, thereby converting the original 3D scene data into pixels that can be seen on the screen. Thus, the objects, lighting and other information in the pre-rendered 3D virtual scene are converted into multiple frames of 2D (Two Dimensions) images in a specific time sequence, so that the audience can feel the 3D visual effect in the 2D images. In practical applications, virtual scenes can be pre-rendered by rendering engines such as UE, Unity, and Arnold, so that they can be flexibly used during virtual shooting. All 2D images converted by the rendering engine are analyzed for pixel flicker based on the frame before being presented on the screen used to present the virtual scene.

[0036] Regarding pixel flicker, in the solution of the embodiment of the present application, the picture to be presented is analyzed for pixel flicker based on the frame picture from multiple angles. In practical applications, only some of the angles may be used, or all of the angles may be used.

[0037] From one perspective, during virtual shooting, the pixels that make up the object in the presented 2D image may flicker due to the rapid movement or shaking of the photographed object; or, if there are moving objects in the virtual scene, if the relevant settings of the virtual camera, such as the shutter speed or motion blur, are not set properly, the rendering engine will not have a good motion blur effect on the moving objects. For example, the shutter speed of the virtual camera is slow but there are fast-moving objects in the virtual scene. In the 2D image converted from the virtual scene, the pixels that make up the moving objects may also produce irregular movements and cause pixel flickering; or, when the rendering engine renders the virtual scene, if the anti-aliasing related parameters are not set properly or the anti-aliasing algorithm is not selected properly, for example, the number of anti-aliasing samples is set too low, in the 2D image converted from the virtual scene, the edge of the image may appear jagged, causing pixel flickering.

[0038] That is, in the picture to be presented, pixel flickering related to the motion information of the pixel will occur. Based on this, the pixel flickering analysis of the picture to be presented can be implemented by performing motion analysis on the picture to achieve the detection of pixel flickering. Exemplarily, based on the motion information of each pixel, it can be determined whether there is a flickering pixel in the picture to be presented due to a change in motion information during the process of displaying the picture to be presented in a time sequence. In one example, the motion information can be a velocity vector of the pixel.

[0039] From another perspective, when using a rendering engine to pre-render a virtual scene, it is necessary to use PDO (Pixel Depth Offset, pixel depth offset) technology to handle the transition between multiple models, or between models and terrain when they are connected, so as to achieve a natural transition effect. Specifically, when rendering a virtual scene, PDO technology changes the distance perception between pixels and virtual cameras, that is, by changing the depth information of pixels, adjusts the details and occlusion relationship of the model or terrain at the contact position with other objects, so that pixels originally at the same depth are visually different to achieve a natural transition effect. In practical applications, the depth information of each pixel in the virtual scene is recorded through a depth buffer to determine which pixels should be displayed and which pixels should be blocked when rendering the virtual scene. When the depth information of the pixels of two objects is very close, and the depth buffer cannot accurately distinguish the display or occlusion relationship between the two objects due to precision limitations, a depth conflict may occur between the two objects, and pixels with close depth information are affected and appear alternately between different pictures when rendering the virtual scene, that is, pixels with two different depth information appear alternately at the same position in the picture, causing pixel flickering. In another case, if the PDO of the pixel is set unreasonably, some pixels may be incorrectly rendered in front or blocked. The incorrectly rendered pixels appear alternately between different pictures due to the change in depth information, which causes pixel flickering.

[0040] That is, in the picture to be presented, pixel flickering related to the depth information of the pixel will occur. Based on this, the pixel flickering analysis of the picture to be presented can be implemented as a depth analysis of the picture to achieve the detection of pixel flickering. Exemplarily, based on the depth information of each pixel, it can be determined whether there is a change in depth information during the process of displaying the picture to be presented in a time sequence, thereby causing flickering pixels in the picture to be presented. In one example, the depth information can be the depth value of the pixel.

[0041] From another perspective, light source is a key factor in building virtual scenes. It can provide lighting and visual guidance. In the rendering engine, the lighting conditions on the surface of the object are obtained through lighting calculation. If the light source is unstable, or the lighting algorithm has defects such as improper light source parameter settings or improper object material settings, when rendering the virtual scene, the pixels of the objects in the virtual scene may show abnormal changes in color information, such as sudden changes in pixel color, or irregular fluctuations in pixel brightness, etc., which will cause visual pixel flickering.

[0042] That is, in the picture to be presented, pixel flickering related to the color information of the pixel will occur. Based on this, the pixel flickering analysis of the picture to be presented can be implemented by performing color analysis on the picture to achieve pixel flicker detection. Exemplarily, based on the color information of each pixel, it can be determined whether there is a change in color information during the process of displaying the picture to be presented in a time sequence, resulting in flickering pixels in the picture to be presented. In one example, the color information can be the color value of the pixel represented by the red, green and blue color model.

[0043] In summary, in an example of an embodiment of the present application, the pixel flicker analysis based on the picture may include at least one of the following: motion analysis of the picture, depth analysis of the picture, and color analysis of the picture. Any one of these analyses can achieve the effect of detecting flickering pixels, but using multiple analyses at the same time can achieve a more comprehensive pixel flicker detection, and the detection result is more accurate.

[0044] The above-mentioned various pixel flicker analysis methods are described below respectively.

[0045] (I) Motion analysis of the picture

[0046] When the pixel flicker analysis includes motion analysis, the motion information of each pixel in the picture to be presented can be obtained, so as to further confirm whether there are flickering pixels in the picture of the virtual scene caused by irregular motion of pixels through the motion information. Specifically, in a feasible manner, the motion information of the pixels in the picture to be presented can be confirmed based on the historical pictures that are temporally adjacent to the picture to be presented, and the motion information is used to reflect the motion of the pixels in the process of moving from the position in the historical pictures to the position in the picture to be presented.

[0047] For an object in the picture, as it moves or as the shooting angle changes, it will show a certain motion law. Correspondingly, the motion information of its pixels will also show the motion law. If the motion information of the pixel does not conform to the corresponding motion law, it can be considered that the pixel has flickered.

[0048] In one example, at least one frame of historical picture that is adjacent in time sequence to a picture to be presented is obtained; pixels in at least one frame of historical picture that correspond to pixels in a picture to be presented, and motion information of the corresponding pixels are determined; and based on the motion information of the corresponding pixels, motion information of the pixels in the picture to be presented is determined.

[0049] In the process of each pixel in the picture to be presented moving from the position in the historical picture to the position in the picture to be presented, between the pictures that are adjacent in time sequence, under normal circumstances, the change in the motion state of the pixel is relatively small. Therefore, when obtaining the motion information of the pixel in the picture to be presented, at least one frame of historical picture that is adjacent in time sequence to the picture to be presented can be referred to to obtain more accurate motion information. Among them, the number of historical pictures obtained can be set according to actual needs, for example, according to the frame rate when the Unreal Engine renders the virtual scene, that is, the number of pictures generated per second: the number of historical pictures obtained that are adjacent in time sequence to the picture to be presented is set to the number of pictures generated within one second according to the current frame rate. However, it is not limited to this, and other setting methods, such as setting based on data statistical results or experimental simulation results, are also applicable to the scheme of the embodiment of the present application.

[0050] Based on at least one frame of historical picture that is adjacent to the picture to be presented in time sequence, the pixels in each frame of historical picture that correspond to the pixels in the picture to be presented and the motion information of the corresponding pixels are determined respectively to confirm the motion information of the pixels in the picture to be presented in time sequence. For example, there is a coin in each of the 10 consecutive frames of historical pictures. As the picture advances, the motion information of the coin will change. From the pixel dimension, the motion information of the pixels corresponding to the coin has changed. Because these pixels are all pixels corresponding to the object coin, although their motion information has changed, the correspondence between multiple frames and the overall change of their motion information can be determined based on the similarity of pixels between adjacent frames. On this basis, accurate motion information is determined for the pixel corresponding to the pixel in the picture to be presented (such as the 11th frame).

[0051] Based on this, in a feasible manner, pixels in the historical picture corresponding to pixels in the picture to be presented may be confirmed based on the similarity between the pixels.

[0052] In one example, the position of a pixel in a picture to be presented is determined, and a pixel area of ​​a preset size containing the pixel is determined based on the position; and pixels corresponding to the pixels in the picture to be presented are determined from an area corresponding to the pixel area in at least one frame of historical picture.

[0053] In an optional manner, for a certain pixel in the picture to be presented, the position of the pixel is confirmed in the picture to be presented, and a pixel area of ​​a preset size including the pixel can be confirmed in the picture to be presented according to the position of the pixel, and a pixel similar to the pixel can be determined from the corresponding same pixel area in the historical picture. In the multi-frame picture of the virtual scene, there is continuity between the pictures that are adjacent in time sequence or close on the time axis according to the time sequence, and the change of the pixel motion state between the two frames of pictures that are adjacent in time sequence is small, thereby avoiding changes that overly affect the user's visual experience in the continuous picture. In the pictures that are adjacent in time sequence or close on the time axis according to the time sequence, if the pixel moves, the position of the pixel in the above picture is close, and the movement range of the pixel is within a certain area. Therefore, when confirming the pixel corresponding to the pixel in the picture to be presented in at least one frame of the historical picture, the confirmation is based on the pixel area of ​​the preset size including the pixel in the picture to be presented. For example, a first pixel area of ​​a preset size including pixel A is confirmed in the picture to be presented, and the first pixel area is a square area with a side length of five pixels centered on the pixel A determined in the picture to be presented.

[0054] In practical applications, the preset size of the first pixel area can be adjusted according to the resolution or size of the picture to be presented, and the distance of the confirmed historical picture from the picture to be presented on the time axis in accordance with the time sequence. For example, the larger the resolution or size of the picture to be presented, the larger the range of movement of the pixels in the picture to be presented in the confirmed historical picture, and therefore the range of the first pixel area is correspondingly larger, so as to more accurately confirm the pixels corresponding to the pixels in the picture to be presented. For another example, when the number of confirmed historical pictures is large, the first pixel area is set for different historical pictures according to the distance of different historical pictures from the picture to be presented on the time axis in accordance with the time sequence. In one example, when 100 frames of historical pictures are confirmed, for a certain pixel in the picture to be presented (such as the 101st frame), the pixel moves from the position in the 1st frame of the historical picture to the position in the 2nd, 3rd, 4th, ..., 99th, 100th frames of the historical picture, and finally moves to the position in the picture to be presented. When the pixel moves from a position in a historical picture far from the picture to be presented on the time axis to a position in the picture to be presented, the number of movements experienced is large, so the displacement of the pixel may be large, and a larger first pixel area can be set in the historical picture accordingly; on the contrary, when the pixel moves from a position in a historical picture close to the picture to be presented on the time axis to a position in the picture to be presented, the number of movements experienced is small, so the displacement of the pixel may be small, and a smaller first pixel area can be set in the historical picture accordingly. Therefore, based on the above principle, in the historical pictures from the 1st frame to the 30th frame, the first pixel area is set to be a square area with a side length of 6 pixels, centered on the pixel A determined in the picture to be presented; in the historical pictures from the 31st frame to the 60th frame, the first pixel area is a square area with a side length of 4 pixels, centered on the pixel A determined in the picture to be presented; in the historical pictures from the 61st frame to the 100th frame, the first pixel area is a square area with a side length of 2 pixels, centered on the pixel A determined in the picture to be presented. Thus, according to the size of the displacement that may occur to the pixel in actual situations, the first pixel area is set for different historical pictures respectively, thereby increasing the accuracy of confirming the pixel corresponding to the pixel in the picture to be presented.

[0055] After confirming a first pixel area of ​​a preset size including pixel A in the picture to be presented, confirm a second pixel area corresponding to the first pixel area in at least one frame of the historical picture. Since the change in the pixel motion state between two frames of pictures that are adjacent in time sequence is small, the pixels corresponding to the pixels in the picture to be presented are determined in the second pixel area of ​​at least one frame of the historical picture, thereby improving the accuracy of obtaining the pixels corresponding to the pixels in the picture to be presented. Specifically, in one example, the relative position of the second pixel area in the historical picture is the same as the relative position of the first pixel area in the picture to be presented, and the second pixel area is the same size as the first pixel area, so that accurate pixel comparison can be performed in the second pixel area, thereby confirming the pixel A' corresponding to the pixel A in the picture to be presented in the second pixel area. Furthermore, all pixels in the second pixel area are traversed, and the similarity between each pixel and pixel A in the picture to be presented is calculated, for example, the Euclidean distance, cosine similarity, etc. between each pixel and pixel A in the picture to be presented are calculated, and the pixel with the highest similarity to pixel A in the picture to be presented in the second pixel area is selected as pixel A' corresponding to pixel A in the picture to be presented, and the motion information of the corresponding pixel A' is obtained accordingly. Thus, through the above method, the pixel corresponding to the pixel in the picture to be presented is confirmed in each frame of the historical picture acquired, without the need to confirm in the entire historical picture, thereby reducing the amount of calculation.

[0056] After confirming the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented, the motion information of the pixels in the picture to be presented is determined based on the motion information of the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented, thereby more comprehensively acquiring the motion status of each pixel in the picture to be presented, and providing a basis for judging whether there are flickering pixels in the picture to be presented due to irregular motion of pixels according to the motion information.

[0057] Among them, the motion information of the pixel corresponding to the pixel in the picture to be presented in at least one frame of historical picture can be reflected: the motion state of the pixel when the corresponding pixel moves from the position in the previous frame of historical picture of at least one frame of historical picture to the position in at least one frame of historical picture. In a feasible way, the motion information of the pixel corresponding to the pixel in the picture to be presented in at least one frame of historical picture can be realized as the speed vector of the pixel when the corresponding pixel moves from the position in the previous frame of historical picture of at least one frame of historical picture to the position in at least one frame of historical picture, and the speed vector can be used to describe the speed magnitude and speed direction of the pixel movement. For example, for the confirmed 10 frames of historical pictures, the motion information of the pixel corresponding to the pixel in the picture to be presented in the 10th frame of historical picture can be reflected as the speed vector of the pixel when the corresponding pixel moves from the position in the 9th frame of historical picture to the position in the 10th frame of historical picture. The speed vector is a two-dimensional vector, which includes the speed magnitude and direction of the pixel on the x-axis and y-axis.

[0058] In a feasible manner, the motion information of pixels in at least one frame of historical pictures can be obtained based on the method of obtaining the motion information of pixels in the picture to be presented in the method of this embodiment. That is, each historical picture in at least one frame of historical pictures is used as the picture to be presented, and the motion information of pixels in the picture to be presented is obtained. Thus, the motion information of pixels in at least one frame of historical pictures is obtained, and then after determining the pixels in at least one frame of historical pictures corresponding to the pixels in the picture to be presented, the motion information of the corresponding pixels is determined. In a feasible manner, for the first frame of the virtual scene in the time sequence, since there is no historical picture for the picture, before confirming the motion information of the pixels in the first frame of the virtual scene in the time sequence corresponding to the pixels in the picture to be presented, the motion information of the pixels in the first frame of the virtual scene in the time sequence can be preset. Among them, the motion information of the pixels in the first frame of the virtual scene in the time sequence can be obtained based on the experience of those skilled in the art, or through model prediction such as a machine learning model. However, the invention is not limited thereto. In practical applications, if the rendering engine of the virtual scene is connected to an external data source, for example, to a motion capture system, the motion information of the pixels in the first frame of the image in the timing sequence can be set according to the external data source.

[0059] In one feasible manner, weighted processing is performed on the motion information of pixels corresponding to the pixels in the picture to be presented in at least one frame of historical picture according to the timing relationship with the picture to be presented and the first preset weight. For example, weighted processing is performed on the motion information of pixels corresponding to the pixels in the picture to be presented in at least one frame of historical picture according to the timing relationship with the picture to be presented and the first preset weight; and the motion information of the pixels in the picture to be presented is determined according to the weighted processing result.

[0060] Among them, the first preset weight can be appropriately set by those skilled in the art according to actual needs. For example, based on historical experience or statistical data, according to the time sequence relationship with the picture to be presented, for the historical picture closer to the picture to be presented on the time axis, the smaller the change in the pixel motion state corresponding to the pixel in the picture to be presented in the historical picture is, the greater the reference value of the motion information of the corresponding pixel is, and a slightly heavier weight can be set. However, it is not limited to this, and the weight can also be evenly distributed for at least one frame of historical pictures, etc. In a feasible manner, based on the above experience, for example, for three frames of historical pictures (the 1st, 2nd, and 3rd frames) that are adjacent in time sequence to the picture to be presented (the 4th frame), according to the distance from the picture to be presented on the time axis from far to near, the motion information of the corresponding pixels in the three frames of historical pictures is set with weights of 0.2 (the weight of the 1st frame), 0.3 (the weight of the 2nd frame), and 0.5 (the weight of the 3rd frame) in turn. Assuming that the motion information of the pixels in the 1st, 2nd and 3rd frames corresponding to the pixel A in the 4th frame are V1, V2 and V3 respectively, the motion information of the pixel A in the 4th frame can be obtained, that is: 0.2V1+0.3V2+0.5V3, that is, the weighted processing result of the pixels in the 1st, 2nd and 3rd historical frames is used as the motion information of the pixel A in the 4th frame.

[0061] Therefore, according to the reference values ​​of different historical pictures to the picture to be presented, weights are set for the motion information of corresponding pixels in different historical pictures, so that the obtained motion information of pixels in the picture to be presented can be more objective and accurate.

[0062] (II) In-depth analysis of the image

[0063] When the pixel flicker analysis includes depth analysis, it is based on the same principle as the aforementioned method of confirming the motion information of pixels in the picture to be presented, and the depth information of each pixel in the picture to be presented can be obtained, so as to further confirm through the depth information whether there are flickering pixels in the picture of the virtual scene caused by changes in the depth information of the pixels.

[0064] For the image of the virtual scene, after the positional relationship between an object in the image and the virtual camera is confirmed, the depth information of the pixels possessed by the object is also confirmed accordingly. As the positional relationship between the object in the image of the virtual scene and the virtual camera changes, the depth information of the pixels possessed by the object will show a certain change pattern, and if the depth information of the pixel does not conform to the corresponding change pattern, it can be considered that the pixel has flickered.

[0065] In one feasible method, at least one frame of historical picture that is adjacent to the picture to be presented in time sequence is obtained; depth information of pixels in the at least one frame of historical picture that correspond to pixels in the picture to be presented is determined; and based on the depth information of the corresponding pixels, the depth information of the pixels in the picture to be presented is determined.

[0066] The specific implementation method of obtaining at least one frame of historical picture that is adjacent to the picture to be presented in time sequence can refer to the specific implementation method of obtaining at least one frame of historical picture that is adjacent to the picture to be presented in time sequence when performing motion analysis on the picture in (I), which will not be repeated here.

[0067] In at least one frame of historical pictures adjacent to the picture sequence to be presented, under normal circumstances, the change in the depth information of the pixel is relatively small. Therefore, in a feasible way, when obtaining the depth information of the pixel in the picture to be presented, at least one frame of historical pictures adjacent to the picture sequence to be presented can be referred to to obtain more accurate depth information. Thus, the depth information of each pixel in the picture to be presented is obtained more comprehensively, providing a basis for judging whether there are flickering pixels caused by changes in pixel depth information in the picture to be presented according to the depth information. Among them, the number of historical pictures obtained can be set according to actual needs, for example, according to the frame rate when the Unreal Engine renders the virtual scene, that is, the number of pictures generated per second: the number of historical pictures obtained adjacent to the picture sequence to be presented is set to the number of pictures generated within one second according to the current frame rate. However, it is not limited to this, and other setting methods, such as setting based on data statistical results or experimental simulation results, are also applicable to the scheme of the embodiment of the present application.

[0068] Based on at least one frame of historical picture that is adjacent to the picture to be presented in time sequence, the pixels in each frame of historical picture that correspond to the pixels in the picture to be presented and the depth information of the corresponding pixels are determined respectively to confirm the change of the depth information of the pixels in the picture to be presented in time sequence. For example, in 10 consecutive frames of historical pictures, there is a sphere, and the distance between the sphere and the virtual camera gradually decreases. From the pixel dimension, the depth information of the pixels corresponding to the sphere has changed. Because these pixels are all pixels corresponding to the object of the sphere, although their depth information has changed, the correspondence between multiple frames and the overall change of the depth information of a certain pixel can be determined based on the similarity of pixels between adjacent frames. And on this basis, accurate depth information is determined for the pixel corresponding to the pixel in the picture to be presented (such as the 11th frame).

[0069] Based on this, in a feasible manner, pixels in the historical picture corresponding to pixels in the picture to be presented may be confirmed based on the similarity between the pixels.

[0070] In one example, the position of a pixel in a picture to be presented is determined, and a pixel area of ​​a preset size containing the pixel is determined based on the position; and pixels corresponding to the pixels in the picture to be presented are determined from an area corresponding to the pixel area in at least one frame of historical picture.

[0071] In an optional manner, for a certain pixel in the picture to be presented, the position of the pixel is confirmed in the picture to be presented, and a pixel area of ​​a preset size including the pixel can be confirmed in the picture to be presented according to the position of the pixel, and a pixel similar to the pixel is determined from the corresponding same pixel area in the historical picture. In the multi-frame picture of the virtual scene, there is continuity between the pictures that are adjacent in time sequence or close in time axis according to time sequence, and the change of pixel depth information between two frames of pictures that are adjacent in time sequence is small, thereby avoiding changes that excessively affect the user's visual experience in the continuous picture. In the pictures that are adjacent in time sequence or close in time axis according to time sequence, if the pixel depth information changes, the position of the pixel in the above picture is close, the change of the relative position between the pixel and the virtual camera is small, and the change of the position of the pixel in the above picture is within a certain area. Therefore, when confirming the pixel corresponding to the pixel in the picture to be presented in at least one frame of the historical picture, the confirmation is based on the pixel area of ​​the preset size including the pixel in the picture to be presented, and there is no need to confirm in the entire historical picture, which reduces the amount of calculation.

[0072] The specific implementation method of confirming the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented can refer to the specific implementation method of confirming the corresponding pixels when performing motion analysis on the picture in (I), which will not be repeated here.

[0073] Based on this, after confirming the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented, the depth information of the pixels in the picture to be presented is determined according to the depth information of the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented, and the depth information of each pixel in the picture to be presented is more comprehensively acquired, providing a basis for judging whether there are flickering pixels in the picture to be presented due to changes in depth information based on the depth information.

[0074] In a feasible manner, the depth information of pixels in at least one frame of historical pictures can be obtained based on the method of this embodiment, in which the depth information of pixels in the picture to be presented is obtained. That is, each historical picture in at least one frame of historical pictures is used as the picture to be presented, and the depth information of pixels in the picture to be presented is obtained. In this way, the depth information of pixels in at least one frame of historical pictures is obtained, and then after determining the pixels in at least one frame of historical pictures corresponding to the pixels in the picture to be presented, the depth information of the corresponding pixels is determined. In a feasible manner, for the first frame of the virtual scene in the time sequence, since there is no historical picture for this picture, the first frame can be analyzed with the help of image processing algorithms, feature extraction algorithms, etc., and the depth information of pixels in at least one frame of historical pictures can be obtained based on the analysis results.

[0075] In one feasible manner, the depth information of pixels corresponding to the pixels in the picture to be presented in at least one frame of the historical picture is weighted according to the temporal relationship with the picture to be presented and the second preset weight. For example, the depth information of pixels corresponding to the pixels in the picture to be presented in at least one frame of the historical picture is weighted according to the temporal relationship with the picture to be presented and the second preset weight; the depth information of the pixels in the picture to be presented is determined according to the weighted processing result.

[0076] Among them, the second preset weight can be appropriately set by those skilled in the art according to actual needs. For example, it can be set to be the same as the first preset weight, or it can be based on historical experience or statistical data, etc., according to the timing relationship with the picture to be presented. For historical pictures that are closer to the picture to be presented on the timeline, the depth information of the pixels corresponding to the pixels in the picture to be presented in the historical pictures changes less, and the reference value of the depth information of the corresponding pixels is greater, and a slightly heavier weight can be set. However, it is not limited to this, and weights can also be evenly distributed for at least one frame of historical pictures, etc. Weights are set for the depth information of corresponding pixels in different historical pictures according to the reference value of different historical pictures to the picture to be presented, so that the depth information of each pixel in the picture to be presented is more objective and accurate.

[0077] The specific implementation method of weighted processing of depth information of pixels in at least one frame of historical picture corresponding to pixels in the picture to be presented can be referred to the specific implementation method of weighted processing of motion information of pixels in at least one frame of historical picture corresponding to pixels in the picture to be presented when performing motion analysis on the picture in (I), and will not be repeated here.

[0078] (III) Color analysis of the image

[0079] When the pixel flicker analysis includes color analysis, it is based on the same principle as the aforementioned method of confirming the motion information of pixels in the picture to be presented, and the color information of each pixel in the picture to be presented can be obtained, in order to further confirm through the color information whether there are flickering pixels in the picture of the virtual scene caused by changes in the color information of the pixels.

[0080] For an object in the picture, as the position of the light source or the light intensity in the virtual scene changes, the illumination of the object will show a certain change pattern. Correspondingly, the color information of the pixels in the area affected by the illumination will also show the change pattern. If the color information of the pixel does not conform to the corresponding change pattern, it can be considered that the pixel has flickered.

[0081] In one feasible method, at least one frame of historical picture that is adjacent to the picture to be presented in time sequence is obtained; color information of pixels in at least one frame of historical picture that correspond to pixels in the picture to be presented is determined; and based on the color information of the corresponding pixels, the color information of the pixels in the picture to be presented is determined.

[0082] The specific implementation method of obtaining at least one frame of historical picture that is adjacent to the picture to be presented in time sequence can refer to the specific implementation method of obtaining at least one frame of historical picture that is adjacent to the picture to be presented in time sequence when performing motion analysis on the picture in (I), which will not be repeated here.

[0083] In at least one frame of historical pictures adjacent to the picture sequence to be presented, under normal circumstances, the change in the color information of the pixel is relatively small. Therefore, in a feasible way, when obtaining the color information of the pixel in the picture to be presented, at least one frame of historical pictures adjacent to the picture sequence to be presented can be referred to to obtain more accurate color information. Thus, the color information of each pixel in the picture to be presented is obtained more comprehensively, providing a basis for judging whether there are flickering pixels caused by the change of pixel color information in the picture to be presented according to the color information. Among them, the number of historical pictures obtained can be set according to actual needs, for example, according to the frame rate when the Unreal Engine renders the virtual scene, that is, the number of pictures generated per second: the number of historical pictures obtained adjacent to the picture sequence to be presented is set to the number of pictures generated within one second according to the current frame rate. However, it is not limited to this, and other setting methods, such as setting based on data statistical results or experimental simulation results, are also applicable to the scheme of the embodiment of the present application.

[0084] Based on at least one frame of historical picture that is adjacent to the picture to be presented in time sequence, the pixels corresponding to the pixels in the picture to be presented in each frame of historical picture and the color information of the corresponding pixels are determined respectively to confirm the change of the color information of the pixels in the picture to be presented in time sequence. For example, in 10 consecutive frames of historical pictures, there is a sphere, which is illuminated by a light source at a fixed position, and the color of the light emitted by the light source switches between white and yellow at a fixed frequency. From the pixel dimension, the color information of the pixels corresponding to the sphere has changed. Because these pixels are all pixels corresponding to the object of the sphere, although their color information has changed, the corresponding relationship between multiple frames and the overall change of their color information can be determined based on the similarity of pixels between adjacent frames. On this basis, accurate color information is determined for the pixel corresponding to the pixel in the picture to be presented (such as the 11th frame).

[0085] Based on this, in a feasible manner, pixels in the historical picture corresponding to pixels in the picture to be presented may be confirmed based on the similarity between the pixels.

[0086] In one example, the position of a pixel in a picture to be presented is determined, and a pixel area of ​​a preset size containing the pixel is determined based on the position; and pixels corresponding to the pixels in the picture to be presented are determined from an area corresponding to the pixel area in at least one frame of historical picture.

[0087] In an optional manner, for a certain pixel in the picture to be presented, the position of the pixel is confirmed in the picture to be presented, and a pixel area of ​​a preset size including the pixel can be confirmed in the picture to be presented according to the position of the pixel, and pixels similar to the pixel can be determined from the corresponding same pixel area in the historical picture. In multiple frames of a virtual scene, there is continuity between pictures that are adjacent in time sequence or close in time sequence on the time axis. If the position of a pixel between two frames that are adjacent in time sequence changes, the position change is small to avoid changes in the continuous pictures that excessively affect the user's visual experience. Therefore, when confirming a pixel in at least one frame of historical picture corresponding to a pixel in the picture to be presented, the confirmation is performed based on a pixel area of ​​a preset size including the pixel in the picture to be presented, and there is no need to confirm in the entire historical picture, thereby reducing the amount of calculation.

[0088] The specific implementation method of confirming the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented can refer to the specific implementation method of confirming the corresponding pixels when performing motion analysis on the picture in (I), which will not be repeated here.

[0089] Based on this, after confirming the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented, the color information of the pixels in the picture to be presented is determined according to the color information of the pixels in at least one frame of historical picture corresponding to the pixels in the picture to be presented, and the color information of each pixel in the picture to be presented is more comprehensively acquired, providing a basis for judging whether there are flickering pixels in the picture to be presented due to changes in color information based on the color information.

[0090] In a feasible manner, the color information of pixels in at least one frame of historical screen can be obtained based on the method of this embodiment, in which the color information of pixels in the screen to be presented is obtained. That is, each historical screen in at least one frame of historical screen is used as the screen to be presented, and the color information of pixels in the screen to be presented is obtained. Thus, the color information of pixels in at least one frame of historical screen is obtained, and then after determining the pixels in at least one frame of historical screen corresponding to the pixels in the screen to be presented, the color information of the corresponding pixels is determined. In a feasible manner, for the first frame of the virtual scene in the time sequence, since there is no historical screen for this screen, the first frame can be analyzed with the help of image processing algorithms, feature extraction algorithms, etc., and the color information of pixels in at least one frame of historical screen can be obtained based on the analysis results.

[0091] In one feasible manner, weighted processing is performed on the color information of pixels corresponding to the pixels in the picture to be presented in at least one frame of the historical picture according to the timing relationship with the picture to be presented and the third preset weight. For example, weighted processing is performed on the color information of pixels corresponding to the pixels in the picture to be presented in at least one frame of the historical picture according to the timing relationship with the picture to be presented and the third preset weight; and the color information of the pixels in the picture to be presented is determined according to the weighted processing result.

[0092] Among them, the third preset weight can be appropriately set by those skilled in the art according to actual needs. For example, it can be set to be the same as the first preset weight or the second preset weight, or it can be based on historical experience or statistical data, etc., according to the timing relationship with the picture to be presented, for the historical picture that is closer to the picture to be presented on the time axis, the color information of the pixel corresponding to the pixel in the picture to be presented in the historical picture changes less, then the reference value of the color information of the corresponding pixel is greater, and a slightly heavier weight can be set. However, it is not limited to this, and weights can also be evenly distributed for at least one frame of historical pictures, etc. Weights are set for the color information of corresponding pixels in different historical pictures according to the reference value of different historical pictures to the picture to be presented, so that the color information of each pixel in the picture to be presented is more objective and accurate.

[0093] The specific implementation method of weighted processing of color information of pixels in at least one frame of historical picture corresponding to pixels in the picture to be presented can be referred to the specific implementation method of weighted processing of motion information of pixels in at least one frame of historical picture corresponding to pixels in the picture to be presented when performing motion analysis on the picture in (I), and will not be repeated here.

[0094] In one example, before acquiring at least one frame of historical picture that is sequentially adjacent to the picture to be presented, at least one of the following is judged: whether there are pixels without motion information in the picture to be presented; whether there are pixels without depth information in the picture to be presented; whether there are pixels without color information in the picture to be presented; whether there are pixels with a preset transparency in the picture to be presented; if there is a judgment result of yes, then the operation of acquiring at least one frame of historical picture that is sequentially adjacent to the picture to be presented is executed. That is, if any of the judgment results is yes, the operation of acquiring at least one frame of historical picture that is sequentially adjacent to the picture to be presented is executed.

[0095] If the pixels of the image to be presented have at least one of accurate motion information, depth information, and color information, or if at least one of accurate motion information, depth information, and color information can be directly determined based on the image to be presented, flicker analysis can be performed based on this information.

[0096] However, in the process of rendering a virtual scene by a rendering engine, in order to improve performance, some optimization settings in the rendering engine, such as reducing the level of detail, may cause the motion information of some pixels in the image to be presented to be ignored, resulting in the presence of pixels without motion information in the image to be presented. As a result, in the image to be presented, it is impossible to determine whether the pixel has flickering caused by irregular motion based on the motion information.

[0097] When the depth buffer in the rendering engine has low precision, the depth information of some pixels in the image to be rendered may be ignored due to factors such as the depth buffer being unable to record the depth information of all pixels, resulting in the presence of pixels without depth information in the image to be rendered. As a result, in the image to be rendered, it is impossible to determine whether the pixel has flickering caused by changes in depth information based on the depth information.

[0098] Similarly, in the process of rendering a virtual scene by a rendering engine, there may be pixels without color information in the picture to be presented due to insufficient or inaccurate lighting settings in the virtual scene. Therefore, in the picture to be presented, it is impossible to determine whether the pixel has flickering caused by changes in color information based on the color information.

[0099] For objects with high transparency in the virtual scene, the rendering engine usually involves more complex motion information calculations, calculations for sorting the distance relative to the virtual camera, and color calculations when rendering, in order to ensure that the objects with high transparency can be correctly rendered and mixed with other objects. If the pixels of the objects with high transparency cannot be correctly mixed with other objects due to problems such as precision limitations in the rendering process, the motion information, depth information, and color information of the pixels corresponding to the objects with high transparency may be erroneous or missing in the picture to be presented. As a result, in the picture to be presented, it is impossible to determine whether the pixel flickers based on the motion information, depth information, or color information.

[0100] Based on this, it is determined according to the above judgment whether it is necessary to obtain at least one frame of historical picture that is adjacent to the picture to be presented in time sequence, so as to obtain more accurate motion information, depth information or color information of the pixels in the picture to be presented based on the historical picture, so as to provide a more accurate basis for picture-based pixel flicker analysis of the picture to be presented.

[0101] Since there may be abnormalities in the above-mentioned motion information, depth information, and color information, in a feasible manner, for pixels with motion information, depth information, or color information in the picture to be presented, the existing motion information, depth information, or color information can be verified based on the results obtained by the above-mentioned pixel flicker analysis to ensure the accuracy of the motion information, depth information, or color information of the pixels in the picture to be presented. Specifically, the existing motion information, depth information, or color information is compared with the results obtained by the pixel flicker analysis. For example, the existing motion information of the pixel of the picture to be presented is compared with the motion information of the pixel obtained by motion analysis of the picture. If the comparison result exceeds a preset value, it can be determined that the existing motion information of the pixel is unreliable, and the existing motion information of the pixel is replaced with the motion information of the pixel obtained by motion analysis of the picture. In practical applications, the comparison of the existing motion information, depth information, or color information with the results obtained by the pixel flicker analysis can be implemented by a technician in this field in any appropriate manner according to actual needs, such as finding a difference or a quotient.

[0102] Step S204: Based on the result of the pixel flicker analysis, determine the flicker pixels to be processed in the picture.

[0103] Among the pixels corresponding to the screen of the virtual scene, there may be pixels that produce unexpected flickering phenomena, including but not limited to: flickering pixels that do not flicker according to the expected flickering rules, flickering pixels that are expected not to flicker but actually flicker in the screen of the virtual scene, etc. For pixels that produce unexpected flickering phenomena, it is necessary to confirm them as flickering pixels to be processed so that they can be processed in subsequent steps, thereby improving the presentation quality of the screen to be presented. For example, based on the needs of virtual shooting, there is a glass curtain wall in the virtual scene, and the color of the glass curtain wall is alternately displayed as red and blue at a fixed frequency. Then, in the screen of the virtual scene, the pixels constituting the glass curtain wall are expected to change the pixel color between red and blue at the same frequency, thereby generating pixel flickering. However, due to the unstable position of the light source, the color information of the pixels constituting the above-mentioned glass curtain wall changes irregularly, which is inconsistent with the expected color information change rule, that is, an unexpected flickering phenomenon occurs, which affects the quality of the video frame obtained when virtual shooting is performed based on the screen to be presented. Therefore, it can be considered that among the pixels constituting the glass curtain wall, the pixels that have produced unexpected flickering phenomenon are flickering pixels to be processed, and they need to be processed in subsequent steps to improve the presentation quality of the picture to be presented.

[0104] In an example, corresponding to the aforementioned motion analysis, depth analysis, and color analysis, the result of the pixel flicker analysis includes at least one of the following: the result of the motion analysis, the result of the depth analysis, and the result of the color analysis.

[0105] The result of the pixel flicker analysis can reflect the dynamic change characteristics of the pixels in the picture of the virtual scene in at least one of motion, depth, and color. For example, when the result of the pixel flicker analysis includes the result of the motion analysis, that is, when it includes motion information, the result of the pixel flicker analysis can reflect the dynamic change of the motion information of the pixels in the picture of the virtual scene in the entire time sequence. Based on this, the dynamic change characteristics are judged. If at least one of the motion information, depth information, and color information of the pixels in the picture to be presented changes during the process of displaying the picture of the virtual scene from the historical picture to the picture to be presented, and this change does not conform to the expected law when establishing the virtual scene, it can be considered as a flicker pixel to be processed, and it needs to be processed in subsequent steps to improve the presentation quality of the picture to be presented. For example, when establishing the virtual scene, the color information of pixel A is expected to remain unchanged, but the result of the pixel flicker analysis shows that in the 10 consecutive frames, the color information of pixel A has changed in the 5th frame. Based on the result of the pixel flicker analysis, the color information of pixel A does not follow the expected law, then it can be considered that pixel A has an unexpected flicker phenomenon, and pixel A is a flicker pixel to be processed.

[0106] Based on this, this step determines the flicker pixels to be processed in the picture to be presented based on the results of the pixel flicker analysis. Any one of the results of the pixel flicker analysis can achieve the effect of determining the flicker pixels to be processed in the picture to be presented, but using multiple results at the same time can more comprehensively confirm the flicker pixels to be processed in the picture to be presented.

[0107] In one feasible method, based on the results of pixel flicker analysis, a presentation trajectory of pixels in the picture can be obtained; according to the presentation trajectory, flicker pixel detection is performed on the pixels in the picture to obtain real flicker pixels and unexpected flicker pixels, and the unexpected flicker pixels are used as flicker pixels to be processed.

[0108] In some scenarios, there may be real flickering pixels in the picture, such as a flashing warning light or a flashing light source. The flickering of the pixels corresponding to such objects is regular and can be considered as real flickering pixels. Otherwise, they can be considered as unexpected flickering pixels, that is, abnormal flickering pixels that may be caused by different reasons need to be flicker-removed. These pixels can be used as flickering pixels to be processed.

[0109] The pixel presentation trajectory can be considered as a visual display of the dynamic changes of pixels in the virtual scene screen during the process of displaying the historical screen to the screen to be presented. It may be presented in any appropriate form, such as a broken line form, a smoothed curve form, etc.

[0110] For example, when the results of the pixel flicker analysis include the results of the motion analysis, that is, include motion information, the presentation trajectory of the pixels in the picture can be a visual display of the dynamic changes of the pixel motion information. In one feasible method, the motion information of the pixels on each frame is arranged in a time sequence, and a series of data points of the motion information can be obtained. By drawing and connecting these data points, the presentation trajectory of the pixels in the picture can be obtained. For example, there is a sphere in the picture of a virtual scene. In 11 consecutive frames, the sphere moves in a straight line at a uniform speed a, and the pixels corresponding to the sphere also move in a straight line at a uniform speed a. In each frame, the movement speed of the pixels corresponding to the sphere is a. The movement speeds of the pixels corresponding to the sphere in each frame are drawn and connected in a time sequence, and the obtained presentation trajectory is shown in the two-dimensional coordinate system as follows: Figure 3A As shown in the figure, the x-axis is the serial number of the picture frame, and the y-axis is the movement speed of the pixel, in meters per second.

[0111] When the result of the pixel flicker analysis includes the result of the depth analysis, that is, includes the depth information, the presentation trajectory of the pixels in the picture can be a visual display of the dynamic changes of the pixel depth information. In one feasible method, the depth information of the pixels on each frame is arranged in a time sequence, and a series of depth information data points can be obtained. By drawing and connecting these data points, the presentation trajectory of the pixels in the picture can be obtained. For example, there is a sphere in the picture of a virtual scene. In 11 consecutive frames, the sphere reciprocates between a point far away from the virtual camera and a point close to the virtual camera. The depth value of the pixel corresponding to the sphere in each frame changes. The depth information of the pixel corresponding to the sphere in each frame is drawn and connected in a time sequence. The obtained presentation trajectory is shown in the two-dimensional coordinate system as follows. Figure 3B As shown in the figure, the x-axis is the serial number of the picture frame, and the y-axis is the depth value of the pixel, in millimeters.

[0112] When the results of the pixel flicker analysis include the results of the color analysis, that is, include color information, the presentation trajectory of the pixels in the picture can be a visual display of the dynamic changes of the pixel color information. In one feasible method, the color information of the pixels on each frame is arranged in a time sequence, and a series of color information data points can be obtained. By drawing and connecting these data points, the presentation trajectory of the pixels in the picture can be obtained. For example, there is a sphere in the picture of a virtual scene. In 11 consecutive frames, the color of the sphere switches between red and blue, and the color information of the pixels corresponding to the sphere in each frame changes. The color information of the pixels corresponding to the sphere in each frame is drawn and connected in a time sequence. The obtained presentation trajectory is shown in the two-dimensional coordinate system as follows. Figure 3C As shown in the figure, the x-axis is the serial number of the picture frame, and the y-axis is the weighted sum of the three color values ​​of R, G, and B of the pixel. In a feasible method, the weights of the three color values ​​of R, G, and B are set according to the sensitivity of the human eye to different colors. For example, a higher weight is set for a color that the human eye is more sensitive to, and a lower weight is set for a color that the human eye is less sensitive to. Then, when the color information of the pixel changes, the weighted sum can better reflect the sensitivity of the human eye to the pixel flicker caused by the change in the color information of the pixel.

[0113] As mentioned above, in practice, flicker pixels include real flicker pixels and unexpected flicker pixels. Real flicker pixels flicker in the virtual scene according to the expected rules when establishing the virtual scene based on the needs of virtual shooting, so no processing is required. For example, there is a light board in the virtual scene that switches colors once a minute and switches between yellow and white. Correspondingly, the pixels corresponding to the light board switch colors once a minute and switch between yellow and white at the same frequency. At this time, in the virtual scene, the pixels corresponding to the light board flicker, but the pixels corresponding to the light board are flickering according to the presentation trajectory. The detection result is that the pixels corresponding to the light board conform to the corresponding color change rules, that is, they flicker according to the expected rules when establishing the virtual scene. Then, the pixels corresponding to the light board can be identified as real flicker pixels, which do not affect the presentation quality of the picture to be presented, so there is no need to remove them. The dynamic change of the motion information of the unexpected flicker pixels does not conform to the corresponding motion rules, so the unexpected flicker pixels are treated as flicker pixels to be processed to be removed in subsequent steps. For example, there is a sphere in the virtual scene. In 11 consecutive frames, the sphere moves in a straight line at a uniform speed a. At this time, it is expected that the pixels corresponding to the sphere also move in a straight line at a uniform speed a. However, according to the presentation trajectory, flickering pixel detection is performed on the pixels corresponding to the sphere. It is found that after the 5th frame, the speed of the pixels corresponding to the sphere is 2a. The dynamic change of the pixel motion information corresponding to the sphere does not conform to its corresponding motion law, that is, it flickers in accordance with the expected law when the virtual scene is established. The pixels corresponding to the sphere can be determined as unexpected flickering pixels, which affect the presentation quality of the picture to be presented, and thus need to be removed.

[0114] Based on the above principle, flicker pixel detection is performed on the pixels in the picture according to the presentation trajectory. Specifically, by judging whether the dynamic changes of the pixels in the picture conform to the expected rules when establishing the virtual scene, the real flicker pixels and unexpected flicker pixels are obtained, and the unexpected flicker pixels are confirmed as flicker pixels to be processed, so as to be processed later, thereby improving the presentation quality of the picture to be presented.

[0115] When the results of pixel flicker analysis include multiple results of motion analysis, depth analysis, and color analysis, that is, multiple results of motion information, depth information, and color information, the presentation trajectories of pixels in the picture at different levels can be obtained. When flicker pixel detection is performed on the pixels in the picture, these presentation trajectories at different levels can complement each other, thereby more comprehensively confirming the flicker pixels to be processed in the picture to be presented. For example, there is a sphere in 11 consecutive frames, and the depth information of the expected sphere remains unchanged when the virtual scene is established, and the motion information remains unchanged. When flicker pixel detection is performed based on the presentation trajectory obtained by the depth information, it is determined that the depth information of the pixels of the sphere conforms to the expected law, but when flicker pixel detection is performed based on the presentation trajectory obtained by the motion information, it is confirmed that the pixels corresponding to the sphere have undergone a change in motion information in the 5th frame, which does not conform to the expected law. Therefore, the presentation trajectory obtained by the motion information, as a supplement to the presentation trajectory obtained by the depth information, confirms that the pixels of the sphere are unexpected flicker pixels, thereby more comprehensively confirming the flicker pixels to be processed in the picture to be presented.

[0116] In addition, in another feasible manner, a scintillation pixel region to be processed may be determined according to the result of the pixel scintillation analysis; and pixels in the scintillation pixel region to be processed may be determined as scintillation pixels to be processed.

[0117] In the virtual scene, on the one hand, there may be a large number of details and elements, and searching for flickering pixels in the entire picture will consume a lot of computing resources. On the other hand, there may be certain areas that need to be focused on, in which pixels are more likely to flicker. For example, the edges of different areas in the picture are prone to appear jagged, causing pixel flickering, etc. Based on this, the flickering pixel area to be processed can be confirmed in the picture to be presented through human-computer interaction and other forms, and the pixels in the flickering pixel area to be processed are all determined as flickering pixels to be processed, so that more flexible processing can be performed on the pixels in this area in the future, thereby improving processing efficiency and reducing the workload of post-processing.

[0118] Step S206: Perform flicker removal processing on the flicker pixels to be processed, and present the flicker-removed picture on the screen.

[0119] The flicker removal processing of the flicker pixels to be processed may be uniformly removed by a unified method or algorithm. However, it is not limited thereto. In order to perform more accurate flicker removal processing on different flicker pixels to be processed, in a feasible method, flicker removal processing may be performed on the flicker pixels to be processed according to different flicker removal intensities. The flicker removal intensities of different flicker pixels to be processed may be determined based on, for example, flicker removal intensities statistical results of flicker pixels of the same type, or based on historical experience, or based on simulation experiment results.

[0120] Optionally, in a feasible manner, a corresponding flicker removal intensity may be determined from a plurality of different flicker removal intensities according to the presentation trajectory of the flicker pixel to be processed; and flicker removal processing may be performed on the flicker pixel to be processed according to the determined flicker removal intensity.

[0121] In the picture to be presented, the flicker pixels to be processed may have different flicker degrees, which is reflected in the presentation trajectory as the degree to which the motion information, depth information or color information of the flicker pixels to be processed contained in a certain point in the presentation trajectory deviates from the motion information, depth information or color information that should be presented according to the corresponding change law. When flicker removal is performed on the flicker pixels to be processed, for pixels with weaker flicker degrees, a weaker flicker removal intensity should be used for flicker removal processing, and for pixels with stronger flicker degrees, a stronger flicker removal intensity should be used for flicker removal processing. If the processing is performed according to a uniform removal intensity, for pixels with weaker flicker degrees, the picture may be blurred or distorted due to excessive flicker removal intensity, and for pixels with stronger flicker degrees, the flicker phenomenon may not be effectively removed due to excessive flicker removal intensity. Among them, the determination of the flicker degree can be flexibly set by those skilled in the art according to actual needs. For example, multiple deviation ranges can be set from low to high. If it falls into a lower deviation range, it is considered that the flicker degree is weak; if it falls into a higher deviation range, it is considered that the flicker program is strong; if it falls into the deviation range between the two, it is considered that the flicker degree is average. Similar to the above, the specific settings of the deviation range and the strength level can be implemented by those skilled in the art according to actual needs, and the embodiments of the present application are not limited to this.

[0122] Based on this, in a feasible method, for different degrees of flicker, a plurality of different flicker removal intensities are preset, so as to perform different degrees of flicker removal for different degrees of flicker, and optimize the flicker removal effect of the picture to be presented. Exemplarily, three flicker removal intensities are preset, and taking the motion information of the pixels in the picture to be presented as an example, when the degree of deviation of the motion information of the flicker pixel to be processed from the motion information that should be presented according to the corresponding change rule is less than or equal to 5%, that is, compared with the motion information of the previous frame of the picture in the time sequence, the increase or decrease of the motion information of the flicker pixel to be processed is less than or equal to 5%, the flicker degree of the flicker pixel to be processed is slight, and the corresponding flicker removal intensity is the third level. At this time, the flicker pixel to be processed does not affect the user's viewing experience, and the picture details should be retained as much as possible; when the motion information of the flicker pixel to be processed deviates from the motion information that should be presented according to the corresponding change rule by more than 5% and less than or equal to 20%, that is, compared with the motion information of the previous frame of the picture in the time sequence, When the increase or decrease in the motion information of the flickering pixel to be processed is greater than 5% and less than or equal to 20%, the flicker degree of the flickering pixel to be processed is obvious, and the corresponding flicker removal intensity is the second level. At this time, the flickering pixel to be processed may attract the attention of the user, and appropriate flicker removal should be performed, while retaining the details of the picture as much as possible; when the degree of deviation of the motion information of the flickering pixel to be processed from the motion information that should be presented according to the corresponding change law is greater than 20%, that is, compared with the motion information of the previous frame of the picture in the time sequence, the increase or decrease in the motion information of the flickering pixel to be processed is greater than 20%, the flicker degree of the flickering pixel to be processed is serious, and the corresponding flicker removal intensity is the first level. At this time, the flickering pixel to be processed seriously affects the user's viewing and needs to be completely removed.

[0123] Therefore, the corresponding flicker removal intensity can be determined from a plurality of different flicker removal intensities based on the presentation trajectory of the flicker pixel to be processed, thereby removing the flicker phenomenon more effectively.

[0124] In a feasible manner, according to the determined flicker removal intensity, a flicker removal strategy is selected to perform flicker removal processing on flicker pixels to be processed. For example, when the determined flicker removal intensity is level three and the flicker degree of the corresponding flicker pixels to be processed is slight, the image to be presented can be smoothed by calculating the average value of the pixels within a preset range around the flicker pixels to be processed to replace the value of the flicker pixels to be processed so as to achieve the effect of smoothing, or by using image enhancement technology, such as improving the contrast of the image, etc., to retain the details in the image to be presented as much as possible and remove the flicker. When the determined flicker removal intensity is level two and the flicker degree of the corresponding flicker pixels to be processed is obvious, the image to be presented can be adaptively filtered, specifically, the parameters of the filter are dynamically adjusted according to the flicker degree and texture characteristics in the image to be presented, so as to remove the flicker while maintaining good image quality. When the determined flicker removal intensity is level one and the flicker degree of the corresponding flicker pixels to be processed is severe, the flicker pattern of the flicker pixels to be processed can be learned through image processing algorithms such as convolutional neural networks (CNN) and generative adversarial networks (GAN), or the model can be trained through deep learning technology to identify and remove the flicker pixels to be processed, thereby effectively removing the flicker pixels to be processed.

[0125] After the flicker pixels to be processed in the image to be presented are removed by the above method, the image after the flicker removal process is presented on the screen. At this time, there are no flicker pixels in the presented image, but the image quality is relatively high, the image presentation is relatively stable, and the presentation effect is relatively good. On this basis, the quality of the video obtained by virtual shooting based on the image presented on the screen is also higher.

[0126] As can be seen from the above, through the scheme of the embodiment of the present application, in the method of presenting a virtual scene through a screen, or projecting a virtual scene through a screen as a shooting background to realize virtual shooting, a picture to be presented on the screen for presenting the virtual scene is subjected to a picture-based pixel flicker analysis, and based on the result of the pixel flicker analysis, the flicker pixels to be processed in the picture are determined, and then flicker removal processing is performed. As a result, the flicker pixels of the picture are processed before being presented on the screen, so that when the picture is presented on the screen, the picture presented is a picture after flicker removal processing, that is, there are no flicker pixels in the picture, but the picture quality is relatively high, the picture presentation is relatively stable, and the presentation effect is relatively good. On this basis, the quality of the video obtained by virtual shooting based on the picture presented on the screen is also higher.

[0127] Compared to the traditional method of performing flicker removal processing through hardware, the solution of the embodiment of the present application does not require the addition of additional hardware, or the need to modify the hardware, which not only improves the quality of the presented image, but also reduces the implementation cost of the flicker removal processing; compared to the traditional method of performing flicker removal processing through complex rendering algorithms, the implementation of the solution of the embodiment of the present application is simpler, more efficient, and has better versatility.

[0128] Reference Figure 4 , shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device. For example, the electronic device can be implemented as follows Figure 1A , or, as Figure 1B The screen shown in .

[0129] like Figure 4 As shown, the electronic device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0130] in:

[0131] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0132] The communication interface 404 is used to communicate with other electronic devices or servers.

[0133] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in any of the above method embodiments.

[0134] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0135] The processor 402 may be a CPU, a GPU (Graphic Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0136] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0137] The program 410 may include multiple computer instructions. Specifically, the program 410 may enable the processor 402 to execute operations corresponding to the method described in any of the aforementioned method embodiments through the multiple computer instructions.

[0138] The specific implementation of each step in program 410 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0139] The present application also provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above-mentioned multiple method embodiments is implemented. The computer storage medium includes but is not limited to: a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk or a magneto-optical disk, etc.

[0140] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any one of the above-mentioned multiple method embodiments.

[0141] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to image data for collection, data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0142] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0143] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., a random access memory (RAM), a read-only memory (ROM), a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown here.

[0144] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0145] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A picture processing method, comprising: During the virtual shooting process, a picture to be presented on a screen for presenting a virtual scene is subjected to pixel flicker analysis based on the picture; Based on the result of pixel flicker analysis, determining flicker pixels to be processed in the picture; The flicker pixels to be processed are subjected to flicker removal processing, and the picture subjected to flicker removal processing is presented on the screen.

2. The method according to claim 1, wherein: The pixel flicker analysis based on the picture includes at least one of the following: performing motion analysis on the picture, performing depth analysis on the picture, and performing color analysis on the picture.

3. The method according to claim 2, wherein: When the pixel flicker analysis includes the motion analysis, the performing pixel flicker analysis based on the picture to be presented on the screen for presenting the virtual scene comprises: Acquire at least one frame of historical picture that is sequentially adjacent to the picture to be presented; Determine pixels in the at least one historical frame that correspond to pixels in the picture to be presented, and motion information of the corresponding pixels; Based on the motion information of the corresponding pixels, the motion information of the pixels in the picture to be presented is determined.

4. The method according to claim 3, wherein: The determining, based on the motion information of the corresponding pixel, the motion information of the pixel in the picture to be presented includes: performing weighted processing on motion information of pixels in the at least one historical picture frame corresponding to pixels in the picture to be presented according to a temporal relationship with the picture to be presented and a first preset weight; According to the weighted processing result, the motion information of the pixels in the picture to be presented is determined.

5. The method according to claim 2, wherein: When the pixel flicker analysis includes the depth analysis, the performing pixel flicker analysis based on the picture to be presented on the screen for presenting the virtual scene comprises: Acquire at least one frame of historical picture that is sequentially adjacent to the picture to be presented; Determining depth information of pixels in the at least one historical frame corresponding to pixels in the picture to be presented; Based on the depth information of the corresponding pixels, the depth information of the pixels in the picture to be presented is determined.

6. The method according to claim 5, wherein: The determining, based on the depth information of the corresponding pixel, the depth information of the pixel in the picture to be presented includes: performing weighted processing on depth information of pixels in the at least one historical picture frame corresponding to pixels in the picture to be presented according to a temporal relationship with the picture to be presented and a second preset weight; Determine the depth information of the pixels in the picture to be presented according to the weighted processing result.

7. The method according to claim 2, wherein: When the pixel flicker analysis includes the color analysis, the performing pixel flicker analysis based on the picture to be presented on the screen for presenting the virtual scene comprises: Acquire at least one frame of historical picture that is sequentially adjacent to the picture to be presented; Determining color information of pixels in the at least one historical frame corresponding to pixels in the picture to be presented; Based on the color information of the corresponding pixels, the color information of the pixels in the picture to be presented is determined.

8. The method according to claim 7, wherein: The determining, based on the color information of the corresponding pixel, the color information of the pixel in the picture to be presented comprises: performing weighted processing on color information of pixels in the at least one historical picture frame corresponding to pixels in the picture to be presented according to a time sequence relationship with the picture to be presented and a third preset weight; According to the weighted processing result, the color information of the pixels in the picture to be presented is determined.

9. The method according to any one of claims 3 to 8, wherein: Before acquiring at least one frame of historical picture that is sequentially adjacent to the picture to be presented, the method further includes: At least one of the following is judged: whether there are pixels without motion information in the picture to be presented; whether there are pixels without depth information in the picture to be presented; whether there are pixels without color information in the picture to be presented; whether there are pixels with a preset transparency in the picture to be presented; If there is a judgment result of yes, the operation of acquiring at least one frame of historical picture that is sequentially adjacent to the picture to be presented is performed.

10. The method according to any one of claims 3 to 8, wherein: The determining of pixels in the at least one historical frame corresponding to pixels in the picture to be presented includes: Determine the position of a pixel in the picture to be presented, and determine a pixel area of ​​a preset size including the pixel according to the position; From the area corresponding to the pixel area in the at least one frame of historical picture, pixels corresponding to the pixels in the picture to be presented are determined.

11. The method according to claim 1, wherein: The step of determining the flicker pixels to be processed in the picture based on the result of the pixel flicker analysis includes: Based on the result of the pixel flicker analysis, obtaining a presentation trajectory of the pixels in the picture; According to the presentation trajectory, flicker pixel detection is performed on pixels in the picture to obtain real flicker pixels and unexpected flicker pixels, and the unexpected flicker pixels are used as flicker pixels to be processed.

12. The method according to claim 1, wherein: The step of determining the flicker pixels to be processed in the picture based on the result of the pixel flicker analysis includes: Determining a flickering pixel area to be processed according to the result of the pixel flicker analysis; The pixels in the to-be-processed scintillation pixel area are determined as the to-be-processed scintillation pixels.

13. The method according to claim 1, wherein: The step of performing flicker removal processing on the flicker pixels to be processed comprises: According to different flicker removal intensities, flicker removal processing is performed on the flicker pixels to be processed.

14. The method according to claim 13, wherein: The step of performing flicker removal processing on the flicker pixels to be processed according to different flicker removal intensities includes: Determining a corresponding flicker removal intensity from a plurality of different flicker removal intensities according to the presentation trajectory of the flicker pixel to be processed; According to the determined flicker removal intensity, flicker removal processing is performed on the flicker pixel to be processed.

15. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 14.

16. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

17. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to execute operations corresponding to the method according to any one of claims 1 to 14.

Images