Image processing method, device and equipment and computer readable storage medium

Abstract

The invention provides an image processing method, device and equipment and a computer readable storage medium. The method comprises the steps of determining grid coordinates of grids corresponding to pixels in a to-be-displayed image, and determining an adjacent grid set of the grids corresponding to the pixels based on the grid coordinates; acquiring the coverage rate of each adjacent grid in each adjacent grid set, the material information of each pixel and the depth gradient value of each grid and each corresponding adjacent grid; determining a generation score of each pixel based on each piece of material information, the coverage rate of each adjacent grid and the corresponding depth gradient value; generating a first surface element of the to-be-displayed image based on each generated score and a score threshold; and coloring the to-be-displayed image by using the first surface element to obtain a target display image. According to the invention, uniform illumination can be realized, and the display definition and the display effect of the to-be-displayed image are improved.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to an image processing method, device, equipment and computer-readable storage medium. Background Art

[0002] With the emergence of various applications, the application scenarios of image display are becoming more and more diversified. Image display can present application scenarios more vividly. Taking games as an example, when the player selects the palace scene, the terminal will display the palace scene accordingly.

[0003] When the related technology generates the surface elements of the palace image, it only considers the coverage of the adjacent surface elements, but does not consider other influencing factors, which causes non-uniform illumination of the palace image, making the palace presented by the terminal blurred and affecting the effect of the image palace display. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus, computer device, computer program product, and computer-readable storage medium, which can achieve uniform illumination and improve the display effect of the image to be displayed.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides an image processing method, the method comprising:

[0007] Determine the grid coordinates of the grid corresponding to each pixel in the image to be displayed, and determine the adjacent grid set of the grid corresponding to each pixel based on each grid coordinate;

[0008] Obtaining the coverage of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value between each grid and the corresponding adjacent grid;

[0009] Determining a generation score of each pixel based on each material information, a coverage rate of each adjacent grid and a corresponding depth gradient value;

[0010] generating a first surface element of the image to be displayed based on each generated score and a score threshold;

[0011] The image to be displayed is colored using the first surface element to obtain a target display image.

[0012] The present application provides an image processing device, the device comprising:

[0013] A first determination module is used to determine the grid coordinates of the grid corresponding to each pixel in the image to be displayed, and determine the adjacent grid set of the grid corresponding to each pixel based on each grid coordinate;

[0014] A first acquisition module is used to acquire the coverage of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value between each grid and the corresponding adjacent grid;

[0015] A second determination module, configured to determine a generation score of each pixel based on each material information, a coverage rate of each adjacent grid and a corresponding depth gradient value;

[0016] A generating module, configured to generate a first surface element of the image to be displayed based on each generating score and a score threshold;

[0017] A coloring module is used to color the image to be displayed using the first surface element to obtain a target display image.

[0018] An embodiment of the present application provides a computer device, including:

[0019] A memory for storing computer executable instructions;

[0020] The processor is used to implement the image processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0021] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the image processing method provided in the embodiment of the present application when executed by a processor.

[0022] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the image processing method provided by the embodiment of the present application is implemented.

[0023] The embodiments of the present application have the following beneficial effects:

[0024] In the embodiment of the present application, the adjacent grid set of each pixel corresponding to the grid in the image to be displayed is first determined, and then the coverage rate of each adjacent grid in each adjacent grid set, the material information of each pixel and the depth gradient value of each grid and the corresponding adjacent grid are obtained; then, based on each material information, the coverage rate of each adjacent grid and the corresponding depth gradient value, the generation score of each pixel is determined, so that the generation score of each pixel is not only associated with the coverage rate of the adjacent grid, but also associated with the material information of the pixel and the depth gradient value of the adjacent grid; then, based on each generation score and the score threshold, the first surface element of the image to be displayed is generated, so that the generation of the first surface element is associated with the coverage rate of the adjacent grid, the material information of the pixel and the depth gradient value of the adjacent grid, and uniform illumination can be achieved through the first surface element; finally, the image to be displayed is colored using the first surface element to obtain the target display image. That is, during the coloring process, not only the coverage rate factor of the adjacent grid is taken into account, but also the material information of the pixel and the depth gradient value factor of the adjacent grid are taken into account, so that the changes in material properties and depth can be reflected during coloring, and finally the display clarity and display effect of the image to be displayed are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic representation of surface elements in the related art;

[0026] Figure 2 is a display schematic diagram of a display interface obtained through relevant technologies;

[0027] Figure 3 is a schematic diagram of a network architecture of an image processing system 100 provided in an embodiment of the present application;

[0028] Figure 4 is a structural diagram of a server 400 provided in an embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of an implementation flow of the image processing method provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of an implementation flow of determining grid coordinates provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of an implementation process of determining coverage provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of an implementation flow of determining a depth gradient value provided in an embodiment of the present application;

[0033] Fig. 9A schematic diagram of an implementation flow of determining a generated score provided in an embodiment of the present application;

[0034] Fig.10 A schematic diagram of an implementation flow of generating a first surface element provided in an embodiment of the present application;

[0035] Fig.11 A schematic diagram of an implementation process for determining candidate attributes provided in an embodiment of the present application;

[0036] Fig.12 A schematic diagram of an implementation process of obtaining a target display image provided in an embodiment of the present application;

[0037] Fig.13 A schematic diagram showing a target display image obtained by the image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0039] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0041] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0042] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0043] 1) Surfel is a data structure used to describe a small area on a large surface. The surface element contains the position, normal and radius of the small area, which is used to accumulate irradiance and is used as a micro point light source at a certain point in the lighting scene. Figure 1 A schematic diagram representing surface elements.

[0044] 2) Material: It is a data structure that describes the surface material, usually including albedo, metallicity, roughness and self-luminescence.

[0045] In order to better understand the image processing method provided in the embodiment of the present application, the method for generating surface elements in the related art and the existing shortcomings are first explained.

[0046] Related technologies include generating surface elements based on screen space, updating surface elements by traversing surface element containers, tracing rays by traversing ray buffers, integrating by traversing surface element containers, and performing pixel shading based on screen space.

[0047] To generate surface elements based on screen space, first, divide the screen space into 16*16 rectangular areas, check the number of surface elements inside each rectangular area, and if the number of surface elements is too small, generate new surface elements according to the geometry buffer (G-Buffer) attributes. Then, update the acceleration structure, and when a surface element is generated, find the small area (cells) it covers, and add the surface element to the surface element list of these small areas.

[0048] For updating surface elements by traversing the surface element container, first, the world position and world normal are updated according to the transform ID, local position and local normal. Then, the surface elements are recycled, and the current surface elements are determined by heuristics whether they need to be removed. If they need to be removed, they are recycled; finally, ray generation is performed, and the number of rays to be generated is determined according to the properties such as the irradiance convergence degree of the surface elements. Assuming that the number of rays is n, n positions in the ray buffer are occupied in advance, where the position can be represented by the sum of the ray offset and the ray count.

[0049] For tracing rays by traversing the ray buffer, first, ray guiding is used to obtain the position (pos) and normal according to the surface element identification number, and the direction and probability density function (pdf) of the ray are generated according to importance sampling. Then, ray tracing is used to collect one indirect illumination and two to infinite indirect illumination through ray tracing to obtain the radiance result, and the radiance result is divided by the probability density function to obtain the radiance.

[0050] For integration by traversing the surface element container, first, access the n rays corresponding to the surface element in the ray buffer, average the illumination results of the n rays, and get the illumination of the surface element in this frame; then, perform an estimator to evaluate the convergence of the illumination based on the difference between the long-term average and the short-term average; then, perform temporal filtering to determine the temporal mixing weight based on the convergence; finally, perform irradiance sharing to determine whether to enable it based on the convergence; if enabled, query the surrounding surface elements and mix the illumination of the surrounding surface elements.

[0051] For pixel shading based on screen space, first, reconstruct the world coordinate position of the pixel, find the small area where the pixel is located, and obtain the list of surface elements in the small area, where the surface elements in the list of surface elements are very close to the pixel. Then, traverse these surface elements and weigh the weights according to the normal and position of the surface elements, and weight and mix them to get the illumination of the pixel. Finally, share (shading) based on the illumination.

[0052] When generating surface elements through the above-mentioned related technologies, only the coverage rate of the adjacent surface elements of a certain point in the world space is considered, and other factors are not considered, and the final result is as follows Figure 2 See the display interface shown in Figure 2 ,The related technologies will cause problems of non-uniform illumination and unclear display, affecting the image display effect.

[0053] The embodiments of the present application provide an image processing method, apparatus, computer device, computer-readable storage medium, and computer program product, which can solve the problems of non-uniform illumination and poor display effects. The following describes an exemplary application of the computer device provided by the embodiments of the present application. The computer device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smart phones, smart speakers, smart watches, smart TVs, and vehicle-mounted terminals. The following describes an exemplary application when the computer device is implemented as a server.

[0054] See also Figure 3 , Figure 3 It is a schematic diagram of the network architecture of the image processing system 100 provided in an embodiment of the present application. To support an image processing application, the terminal 200 is connected to the server 400 via the network 300. The network 300 may be a wide area network or a local area network, or a combination of the two.

[0055] The terminal 200 is used to collect the image to be displayed and send the image to be displayed to the server 400. The server 400 determines the grid coordinates of the grid corresponding to each pixel in the image to be displayed, and determines the adjacent grid set of the grid corresponding to each pixel based on each grid coordinate; the server 400 obtains the coverage rate of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value of each grid and the corresponding adjacent grid; the server 400 determines the generation score of each pixel based on each material information, the coverage rate of each adjacent grid and the corresponding depth gradient value; the server 400 generates the first surface element of the image to be displayed based on each generation score and the score threshold; the server 400 uses the first surface element to color the image to be displayed to obtain the target display image. The server 400 returns the target display image to the terminal 200 so that the terminal 200 displays the target display image.

[0056] In some embodiments, the terminal 200 is used to collect the image to be displayed and obtain the material information of each pixel in the image to be displayed; the terminal 200 also sends a request to obtain the coverage rate and depth gradient value to the server 400, and the server 400 determines the grid coordinates of the grid corresponding to each pixel in the image to be displayed based on the acquisition request, and based on each grid coordinate, determines the adjacent grid set of the grid corresponding to each pixel, obtains the coverage rate of each adjacent grid in each adjacent grid set and the depth gradient value of each grid and the corresponding adjacent grid, and returns the coverage rate and depth gradient value to the terminal 200; the terminal 200 determines the generation score of each pixel based on each material information, the coverage rate of each adjacent grid and the corresponding depth gradient value; the terminal 200 generates the first surface element of the image to be displayed based on each generation score and the score threshold; the terminal 200 uses the first surface element to color the image to be displayed to obtain and display the target display image.

[0057] In the embodiment of the present application, taking the generation of the first surface element by the server 400 as an example, the server 400 makes the generation of the first surface element associated with the coverage of the adjacent grids, the material information of the pixels, and the depth gradient value of the adjacent grids, and can achieve uniform illumination through the first surface element. Based on this, when the first surface element is used for coloring, not only the coverage factor of the adjacent grids is taken into account, but also the material information of the pixels and the depth gradient value factor of the adjacent grids are taken into account, so that the changes in material properties and depth can be reflected during coloring, and finally the display clarity and display effect of the image to be displayed are improved.

[0058] In some embodiments, the server 400 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal 200 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car-mounted smart terminal, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0059] See also Figure 4 , Figure 4 A schematic diagram of the structure of the server 400 provided in the embodiment of the present application, Figure 4The server 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the server 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 4 Various buses are labeled as bus system 440 .

[0060] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0061] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0062] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0063] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0064] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0065] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0066] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0067] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0068] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0069] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 4 An image processing device 455 stored in a memory 450 is shown, which may be software in the form of a program or a plug-in, and includes the following software modules: a first determination module 4551, a first acquisition module 4552, a second determination module 4553, a generation module 4554, and a coloring module 4555. These modules are logical, and thus may be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0070] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (Application Specific Integ rated Circuit, ASIC), DSP, programmable logic device (Programmable Logic Device, PLD), complex programmable logic device (Complex Programmable Logic Device, CPLD), field programmable gate array (Field-Programmable Gate Array, FPGA) or other electronic components.

[0071] The image processing method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the terminal or server provided in the embodiment of the present application.

[0072] The following describes the image processing method provided by the embodiment of the present application. As mentioned above, the computer device implementing the image processing method of the embodiment of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0073] It should be noted that the following examples of image processing methods are described using a game scene as an example. Those skilled in the art, based on their understanding of the following, may apply the image method provided in the embodiments of the present application to image processing in other scenes.

[0074] See also Figure 5 , Figure 5 This is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application. The method is applied to a computer device. Figure 5 The steps shown are used to illustrate the image processing method provided by the embodiment of the present application. Figure 5 The execution subject is the server.

[0075] In step S101, the grid coordinates of the grid corresponding to each pixel in the image to be displayed are determined, and based on each grid coordinate, the adjacent grid set of the grid corresponding to each pixel is determined.

[0076] In some embodiments, see Figure 6 , Figure 5 The step S101 of "determining the grid coordinates of the grid corresponding to each pixel in the image to be displayed" can be implemented by the following steps S1011 to S1013, which are described in detail below.

[0077] In step S1011, a geometric buffer corresponding to the image to be displayed is obtained.

[0078] In some embodiments, the geometry buffer stores data such as depth values, surface normals, texture coordinates, and material properties for each pixel in the image to be displayed. The data stored in the geometry buffer can be used for image processing, image rendering, etc., such as deferred shading, so that lighting calculations can be performed efficiently. The geometry buffer is a data structure used in computer graphics to store geometric information of a three-dimensional scene. The geometry buffer is typically used in real-time rendering applications, such as video games or virtual reality experiences, to effectively determine lighting and other visual effects.

[0079] In some embodiments, when the terminal acquires the image to be displayed, the geometric buffer corresponding to the image to be displayed is obtained at the same time as the image to be displayed is obtained. Correspondingly, when the server acquires the image to be displayed, the geometric buffer corresponding to the image to be displayed is also acquired. The image to be displayed may be an image in a game scene, an image displayed in a virtual reality scene, a photographed landscape, etc.

[0080] In step S1012, the depth value of each pixel in the to-be-displayed image is obtained from the geometry buffer, and the world coordinates of each pixel are determined based on the pixel position and each depth value of each pixel.

[0081] In some embodiments, for each pixel, a depth value corresponding to the pixel position may be obtained from the geometry buffer based on the pixel position of each pixel, thereby obtaining the depth value of each pixel.

[0082] In some embodiments, the pixel position and the corresponding depth value of each pixel are used to construct the screen space coordinates of each pixel. Assuming that the pixel position is (x, y) and the depth value is d, the obtained screen space coordinates are (x, y, d). Next, the projection matrix, view matrix and model matrix are obtained, the product matrix of the projection matrix, view matrix and model matrix is ​​determined, and then the inverse matrix of the product matrix is ​​multiplied by the screen space coordinates of each pixel to obtain the world coordinates of each pixel. That is, the screen space coordinates are converted to world space coordinates through the projection matrix, view matrix and model matrix.

[0083] In step S1013, the world coordinates of each pixel are converted into grid coordinates of the grid corresponding to each pixel.

[0084] In some embodiments, the world coordinates of each pixel can be converted into grid coordinates by a grid conversion function. For example, the grid conversion function can be a GetGridCoordinate() function. The world coordinates of each pixel can be substituted into the grid conversion function, and the output of the grid conversion function is the grid coordinates of the grid corresponding to each pixel.

[0085] In the embodiment of the present application, based on each grid coordinate, an adjacent grid set adjacent to the grid corresponding to each pixel is determined, and the adjacent grid set refers to the grids adjacent to the grid coordinates in the three-dimensional space. The adjacent grid set includes 27 adjacent grids.

[0086] Through step S101, the depth value of each pixel can be determined first, and then the screen space coordinates and world coordinates of each pixel can be determined in turn; then, the world coordinates of each pixel are converted into grid coordinates; finally, based on each grid coordinate, the adjacent grid set of the grid corresponding to each pixel is determined. In this way, the adjacent grid set corresponding to each pixel can be determined quickly and accurately, so as to determine the generation score of each pixel through the characteristics of the adjacent grid set.

[0087] Continue to see Figure 5 , and the description continues with step S101 above.

[0088] In step S102, the coverage rate of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value between each grid and each corresponding adjacent grid are obtained.

[0089] In some embodiments, before executing the above step S102, it is necessary to first determine the coverage rate of each adjacent grid in each adjacent grid set, see Figure 7 , the coverage rate of each adjacent grid in each adjacent grid set can be determined by following steps S001A to S005A, which are described in detail below.

[0090] In step S001A, each world coordinate corresponding to each pixel, each normal of each pixel and each corresponding second surface element of each adjacent grid are obtained.

[0091] In the embodiment of the present application, the second surface element includes a position, a radius, and a normal.

[0092] In some embodiments, the world coordinates of each pixel can be determined by the above step S1012. Since the geometry buffer stores the surface normal of each pixel, that is, the geometry buffer stores the normal of each pixel, based on this, the normal of each pixel can be obtained from the geometry buffer.

[0093] In the embodiment of the present application, when the grid coordinates of the grid corresponding to each pixel are determined, the second surface elements of each adjacent grid corresponding to each pixel are also obtained simultaneously, and the second surface elements include position, radius and normal.

[0094] In step S002A, a first value of each adjacent grid is determined based on each world coordinate, a position of each adjacent grid and a corresponding radius.

[0095] In some embodiments, the positions of each adjacent grid may be each adjacent grid coordinate. Taking one pixel as an example, the difference between the world coordinate of the pixel and the coordinates of each adjacent grid corresponding to the pixel may be determined first, and then each difference may be divided by the radius of each corresponding adjacent grid to obtain the first value of each adjacent grid corresponding to the pixel.

[0096] In some embodiments, the first value of each adjacent grid corresponding to each pixel may be determined in a similar manner as described above.

[0097] In step S003A, based on each normal line and the corresponding normal line of each adjacent mesh, a second value of each adjacent mesh is determined.

[0098] In some embodiments, taking one pixel as an example, the dot product of the normal of the pixel and the normals of each adjacent grid corresponding to the pixel may be determined first, and the dot product result may be determined as the second value of each adjacent grid corresponding to the pixel.

[0099] In some embodiments, the second value of each adjacent grid corresponding to each pixel may be determined in a similar manner as described above.

[0100] In step S004A, occlusion information between each world coordinate and the position of each corresponding adjacent grid is obtained, and the third value of each adjacent grid is determined based on each occlusion information.

[0101] In some embodiments, taking one pixel as an example, the middle area between the world coordinates of the pixel and the coordinates of each adjacent grid corresponding to the pixel can be determined first, and then the object information contained in each middle area can be read, and the occlusion information of the occluder in the middle area can be determined based on each object information, and finally the third value of each adjacent grid can be determined based on each occluder information. Among them, the occluder information can be a numerical value, and the size of the numerical value can represent the size of the occluder. When the occluder information represents that the occluder is larger, the third value is smaller; when the occluder information represents that the occluder is smaller, the third value is larger. Exemplarily, when the occluder information is 5, it represents an occluder with a volume of 5 square centimeters, and the corresponding third value can be 10; when the occluder information is 10, it represents an occluder with a volume of 10 square centimeters, and the corresponding third value can be 2.

[0102] In some embodiments, the third value of each adjacent grid corresponding to each pixel may be determined in a similar manner as described above.

[0103] In step S005A, the coverage rate of each adjacent grid is determined based on the first value, the corresponding second value and the corresponding third value of each adjacent grid.

[0104] In some embodiments, taking an adjacent grid corresponding to a pixel as an example, the first value of the adjacent grid is firstly added to the second value of the adjacent grid to obtain the cumulative sum of the adjacent grid; then the cumulative sum of the adjacent grid is multiplied by the third value of the adjacent grid, and the multiplication result is the coverage rate of the adjacent grid. The coverage rates of other adjacent grids corresponding to the pixel can be determined in a similar manner.

[0105] In some embodiments, the coverage of each adjacent grid corresponding to each pixel may be determined in a manner similar to that described above.

[0106] In some embodiments, before executing the above step S102, it is necessary to determine the depth gradient value of each grid and each corresponding adjacent grid, see Figure 8 , the depth gradient values ​​of each grid and its corresponding adjacent grids can be determined by following steps S001B to S004B, which are described in detail below.

[0107] In step S001B, the grid corresponding to each pixel is determined.

[0108] In some embodiments, the grid coordinates corresponding to each pixel may be determined through the above step S1013. The grid coordinates are used to indicate the position of the grid, which is equivalent to determining the grid corresponding to each pixel.

[0109] In step S002B, the depth value of each grid and the depth value of each adjacent grid are obtained, and based on the depth value of each grid and the depth value of each corresponding adjacent grid, the depth value variation between each grid and each corresponding adjacent grid is determined.

[0110] In some embodiments, the depth value of each pixel may be determined as the depth value of the grid corresponding to each pixel. Taking a grid corresponding to a pixel as an example, the difference between the depth value of the grid and the depth value of the grid adjacent to the grid is determined as the depth value variation between the grid and the corresponding adjacent grids.

[0111] In some embodiments, the depth value variation between each grid and each corresponding adjacent grid may be determined by the above method.

[0112] In step S003B, the distance between each grid and each corresponding adjacent grid is determined.

[0113] In some embodiments, the distance between each grid and each corresponding adjacent grid may be determined based on the grid coordinates of each grid in combination with Euclidean distance, Manhattan distance, and the like.

[0114] In step S004B, the depth gradient value between each grid and each corresponding adjacent grid is determined based on the depth value variation and the corresponding distance corresponding to each grid.

[0115] In some embodiments, taking an adjacent grid corresponding to a grid as an example, the depth gradient value between the grid and the adjacent grid is obtained by dividing the depth value change between the grid and the adjacent grid by the distance between the grid and the adjacent grid. The depth gradient value between the grid and each corresponding adjacent grid can be determined by the above method.

[0116] In some embodiments, since the geometry buffer stores the material information of each pixel, based on this, the material information of each pixel can be obtained from the geometry buffer. For example, the material information can be metal information corresponding to metal material, tile information corresponding to tile material, cotton information corresponding to cotton material, etc.

[0117] In some embodiments, the above steps S001A to S005A may be performed before the above steps S001B to S004B, and the above steps S001A to S005A may also be performed after the above steps S001B to S004B. In addition, the above steps S001A to S005A and the above steps S001B to S004B may also be performed simultaneously.

[0118] Through step S102, not only the coverage of each adjacent grid is obtained, but also the material information of each pixel and the depth gradient value of each adjacent grid are obtained. Since different materials have different luminous properties, and the light will change with the change of depth value, the present application can obtain comprehensive and complete information affecting the brightness of light.

[0119] Continue to see Figure 5 , and the description continues with step S102 above.

[0120] In step S103, the generation score of each pixel is determined based on each material information, the coverage of each adjacent grid and the corresponding depth gradient value.

[0121] In some embodiments, see Fig. 9 , Figure 5 The illustrated step S103 can be implemented by following the steps S1031 to S1033, which are described in detail below.

[0122] In step S1031 , based on each material information and each depth gradient value, the importance value of each adjacent mesh is determined.

[0123] In some embodiments, the self-luminous value of the material corresponding to each pixel may be determined based on each material information; and then the sum of each self-luminous value and each corresponding depth gradient value may be determined as the important value of each adjacent grid.

[0124] In some embodiments, different materials correspond to different self-luminous values, and the mapping relationship between the material and the corresponding self-luminous value can be stored in a mapping relationship table in advance. Based on this, the self-luminous value corresponding to the material can be obtained from the mapping relationship table based on the material information of each pixel; then, the self-luminous value and the corresponding depth gradient value are accumulated, and the accumulated sum is determined as the important value of each adjacent grid. Among them, the self-luminous value can be represented by a numerical value, and the larger the numerical value, the stronger the self-luminous ability of the material. For example, the self-luminous value of the turned-on lamp can be 10, the self-luminous value of the metal material can be 4, and the self-luminous value of the cotton material can be 2.

[0125] In step S1032, the difference between the importance value of each adjacent grid and the corresponding coverage rate is determined as the score of each adjacent grid.

[0126] In some embodiments, the score of each adjacent grid is obtained by subtracting the corresponding coverage from the importance value. For example, taking an adjacent grid of a pixel as an example, assuming that the importance value of the adjacent grid is 25 and the coverage is 12, the score of the adjacent grid is 13.

[0127] In step S1033, the sum of the scores of the adjacent grids corresponding to each pixel is determined as the generation score of each pixel.

[0128] In some embodiments, for each pixel, the scores of the adjacent grids corresponding to the pixel are accumulated, and the accumulated sum is determined as the generated score of the pixel.

[0129] Exemplarily, assuming that the scores of the adjacent grids corresponding to a pixel are 1 to 27, the generated score of the pixel is the cumulative sum of 1 to 27, that is, the generated score of the pixel is 378.

[0130] Through step S103, the generation score of each pixel is determined by the material information, the coverage rate of each adjacent grid and the depth gradient value, so that the three together affect the generation score of the corresponding pixel. Since the material information, coverage rate and depth gradient value are the main factors affecting the brightness, the generation score fully considers the comprehensive and complete factors affecting the brightness.

[0131] Continue to see Figure 5 , and the description continues with step S103 above.

[0132] In step S104, a first surface element of the image to be displayed is generated based on the respective generated scores and the score threshold.

[0133] In some embodiments, see Fig.10 , Figure 5 The illustrated step S104 can be implemented by following the steps S1041 to S1045, which are described in detail below.

[0134] In step S1041 , when there is a target generation score greater than the score threshold among the generation scores, the score threshold is updated.

[0135] In the embodiment of the present application, the score threshold refers to the current score threshold. As long as there is a target generation score greater than the score threshold, the score threshold is updated once to obtain an updated score threshold. The updated score threshold is used as the new score threshold for comparison with the generation score next time. If there is a target generation score, the score threshold is updated once; if there are multiple target generation scores, the score threshold is updated multiple times, and the updated score threshold takes effect in real time. If each generation score is less than or equal to the score threshold, the current score threshold remains unchanged, that is, there is no need to update the score threshold. Among them, the initial score threshold can be set to 0.

[0136] When updating the score threshold, the score threshold may be added to the preset score to obtain the updated score threshold, or the score threshold and the preset factor may be multiplied to determine the updated score threshold, where the preset score is a positive number and the preset factor is a number greater than 1.

[0137] In step S1042, candidate attribute values ​​are determined based on the target generation score.

[0138] In some embodiments, see Fig.11 , Fig.10 The illustrated step S1042 can be implemented by following the steps S421 to S423, which are described in detail below.

[0139] In step S421, the target pixel corresponding to the target generation score is determined.

[0140] In some embodiments, the target generation score may carry an identifier of a corresponding pixel, and the target pixel corresponding to the target generation score may be determined by the pixel identifier, wherein the pixel identifier may be the position coordinate of the pixel.

[0141] In step S422, the world coordinates of the target pixel, the normal line of the target pixel, and a preset radius are obtained.

[0142] In some embodiments, the world coordinates of the target pixel may be determined by referring to the method for determining the world coordinates of each pixel in step S1012. In addition, since the geometry buffer stores the surface normals of each pixel, that is, the geometry buffer stores the normals of each pixel, based on this, the normal corresponding to the target pixel may be obtained from the geometry buffer. The preset radius is a value preset in advance, and for example, the preset radius may be 2 mm, 3 mm, 4 mm, etc.

[0143] In step S423, the world coordinates, the normal line and the preset radius are determined as candidate attribute values.

[0144] In the embodiment of the present application, the world coordinates, the corresponding normal and the preset radius of each target pixel are determined as candidate attribute values. The number of candidate attribute values ​​corresponds to the number of target pixels. For example, when there are 50 target pixels, 50 candidate attribute values ​​can be determined.

[0145] Continue to see Fig.10 , and the description continues with step S1042 above.

[0146] In step S1043, a random number corresponding to the target generation score is generated based on a preset random function.

[0147] In the embodiment of the present application, the number of random numbers is the same as the number of target generation scores.

[0148] In the embodiment of the present application, each time a target generation score is determined, a random number is generated by a preset random function, wherein the preset random function may be a rand function, an srand function, etc. If there are multiple target generation scores in each generation score, multiple random numbers corresponding to the target generation scores are generated. For example, when there are 50 target generation scores, 50 random numbers may be determined.

[0149] In step S1044, when there is a target random number greater than a set threshold among the random numbers, a selected generation score corresponding to the target random number is determined from the target generation scores.

[0150] In the embodiment of the present application, the set threshold may be a value set in advance. For example, the set threshold may be 0.4, 0.5, 0.6, etc.

[0151] In an embodiment of the present application, a random number greater than a set threshold is determined as a target random number, and a target generation score corresponding to the target random number is also determined from the target generation score, which is recorded as the selected generation score.

[0152] In some embodiments, if all random numbers are less than or equal to a set threshold, the process ends directly.

[0153] In step S1045, a first surface element is generated based on the candidate attribute value selected to generate a score.

[0154] In the embodiment of the present application, based on the pixel identifier corresponding to the selected generation score, the candidate attribute value of the selected generation score can be determined from the candidate attribute values ​​of all target generation scores; and the first surface element is generated based on the candidate attribute value of the selected generation score. That is, the world coordinates of the candidate attribute value of the selected generation score are determined as the position of the first surface element, the normal of the candidate attribute value of the selected generation score is determined as the normal of the first surface element, and the preset radius is determined as the radius of the first surface element.

[0155] Through step S104, when the generated score is greater than the score threshold and the random number is greater than the set threshold, the first surface element is generated based on the candidate attribute value of the selected generated score, that is, the first surface element is generated when the self-luminous value of the material is larger and the depth gradient value is larger, so that the first surface element can truly reflect the changes in the self-luminous value of the material and the pixel depth value through the first surface element, that is, the first surface element is associated with the material and the depth gradient value. In addition, the first surface element can also be randomly generated based on the random number, so as to improve the uniformity of the first surface element in the target display image.

[0156] Continue to see Figure 5 , and the description continues with step S104 above.

[0157] In step S105, the image to be displayed is colored using the first surface element to obtain a target display image.

[0158] In some embodiments, the first surface element includes normal direction information, position information, and material attribute information, see Fig.12 , Figure 5 The illustrated step S105 can be implemented by following the steps S1051 to S1053, which are described in detail below.

[0159] In step S1051, the initial brightness value of the image to be displayed and a preset illumination model are obtained.

[0160] In the embodiment of the present application, the image to be displayed corresponds to a brightness value, which is recorded as the initial brightness value. The initial brightness value can be obtained at the same time as the image to be displayed, wherein the initial brightness value includes the brightness value of each pixel in the image to be displayed. Brightness, also known as luminous efficiency, refers to the brightness of a surface, that is, the luminous flux emitted per unit solid angle per unit surface area of ​​the orthographic projection on a plane perpendicular to the light transmission direction of the light source.

[0161] In some embodiments, the preset illumination model may be a Phong illumination model, a Lambert illumination model, a Brin-Phong illumination model, etc. The illumination model is a computer model that simulates the physical process of light illumination in nature based on relevant laws of optics. The illumination model can take into account the direct illumination effect of the light source on the surface of the object.

[0162] In step S1052, the initial brightness value, normal direction information, position information and material attribute information are superimposed using the illumination model to obtain a superimposed brightness value.

[0163] In the embodiment of the present application, the superposition processing is performed in units of pixels, and the brightness of the pixel corresponding to the first surface element can be determined based on the normal direction information, position information and material attribute information. Based on this, on the basis of the initial brightness of each pixel, the brightness is superimposed in combination with the set weight to obtain the superimposed brightness value.

[0164] In some embodiments, for a pixel without the first surface element, the initial brightness value of the pixel is directly determined as the brightness value of the pixel after superposition.

[0165] In step S1053, the initial brightness value of the image to be displayed is updated using the superimposed brightness value to obtain a target display image.

[0166] In the embodiment of the present application, the initial brightness value of the image to be displayed is updated to the brightness value after superposition, and the image after the brightness value is updated is determined as the target display image. The target display image is finally displayed, and the brightness value presented by the target display image is the brightness value after superposition, which can show more details, thereby improving the display effect.

[0167] Through step S105, the image to be displayed is colored using the generated first surface element, that is, the initial brightness value of the image to be displayed is updated, and the initial brightness value is updated to the superimposed brightness value, so as to obtain the target display image. Since more details, textures, light and dark features are superimposed by the first surface element, the target display image can show more image features and can restore the original scene more clearly and realistically, thereby improving the display effect.

[0168] Through the above steps S101 to S105, the generation of the first surface element is associated with the coverage of the adjacent grids, the material information of the pixels, and the depth gradient value of the adjacent grids, and uniform illumination can be achieved through the first surface element. Based on this, when the first surface element is used for coloring, not only the coverage factor of the adjacent grids is taken into account, but also the material information of the pixels and the depth gradient value factor of the adjacent grids are taken into account, so that the changes in material properties and depth can be reflected during coloring, and finally the display effect of the image to be displayed is improved.

[0169] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0170] The embodiment of the present application proposes a method for generating surface elements, which uses and introduces material self-luminous information and the depth gradient value of the current pixel to jointly determine the important value, thereby affecting the generation probability of the surface elements.

[0171] In some embodiments, for a rectangular area of ​​16*16 pixels, first, the world space normal is obtained from the geometry buffer, denoted as N, and the depth value of the pixel is also obtained from the geometry buffer, denoted as d. Next, the point (x, y, d) in the screen space is denoted as Ps, and the product matrix (denoted as P*V*M) using the projection matrix (denoted as P), the view matrix (denoted as V) and the model matrix (denoted as M) is determined by multiplication, and then the inverse matrix of the product matrix is ​​obtained; Ps is converted to a world space point by the inverse matrix, denoted as Pw. Then, Pw is converted to a grid coordinate by a conversion function, denoted as gridcoordinate, where the conversion function can be a GetGridCoordinate() function, and for the 27 adjacent grids of gridcoordinate, the surface elements of each adjacent grid are obtained, denoted as adjacent surface elements. Afterwards, the coverage (C, Coverage) of the 27 adjacent surface elements to Pw is checked. Wherein, C can be determined by the following formula (1); and the importance (I, Importance) is also determined by formula (2). Finally, combined with formula (3), the surface point generation score (SGS, surfel_generation_score) of Pw is obtained by I and C.

[0172] C=(Contribution1+Contribution2)*Contribution3 Formula (1);

[0173] Wherein, C is the coverage rate of adjacent surface elements to the world coordinates, Contribution1 corresponds to the first value in other embodiments, Contribution2 corresponds to the second value in other embodiments, and Contribution3 corresponds to the third value in other embodiments.

[0174] I=Contribution4+Contribution5 formula (2);

[0175] Among them, I corresponds to the important value in other embodiments, Contribution4 corresponds to the material information in other embodiments, and Contribution5 corresponds to the depth gradient value in other embodiments.

[0176] SGS=IC Formula (3);

[0177] Here, SGS corresponds to the generated score in other embodiments.

[0178] In some embodiments, after obtaining SGS, it is determined whether SGS is greater than a score threshold, which can be recorded as max_surfel_generation_score. If SGS is greater than the score threshold, max_surfel_generation_score is updated, and the new position is set to Pw, the new normal is set to N, the new radius is set to 2, and the new position, new normal and new radius are determined as candidate surface element values. Among them, the new position can be recorded as Pnew, the new normal can be recorded as Nnew, and the new radius can be recorded as Radiusnew.

[0179] In some embodiments, all adjacent surface elements of a pixel are iterated. A random number is also generated, and it is checked whether the random number is greater than a preset threshold. If the random number is greater than the preset threshold, Pnew, Nnew, and Rnew are used to generate a new surface element.

[0180] The embodiment of the present application focuses on determining Contribution 4 and Contribution 5, and determining I based on Contribution 4 and Contribution 5. Among them, Contribution 4 is mainly used for self-luminous materials. When the material associated with the pixel is self-luminous, the chance of generating surface elements should be increased, because each point of an object of self-luminous material is a luminous illumination point. Based on this, more surface elements should be placed on the pixel. Conribution 5 mainly acts on surfaces with large curvature changes. Since the surface element is a plane with a radius, the flatter the curvature of the surface where the pixel is located, only one surface element is needed to approximate it. If the curvature of the surface where the pixel is located is less flat and the curvature of the surface changes more, more surface elements are needed.

[0181] The surface element generation method of the related technology only considers the coverage of adjacent grids in the world space, and lacks consideration of other factors. The embodiment of the present application provides a "signal" from the material data of the object so that the coverage process can determine where denser surfing should be generated, and comprehensively controls the density of surface elements so that the lighting of the rendered scene will be more uniform.

[0182] In some embodiments, see Fig.13 Through the image processing method provided in the embodiment of the present application, the following can be obtained: Fig.13 The target display image shown is shown. Compared with the related art Figure 2 , obtained in the embodiment of the present application Fig.13 Objects in the scene appear bright and clear, improving the overall display effect of the scene.

[0183] It can be understood that in the embodiments of the present application, data related to the image to be displayed, material information, etc. are involved, and the collection, use and processing of such relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0184] The following is a description of an exemplary structure of the image processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 4 As shown, the software modules stored in the image processing device 455 of the memory 450 may include:

[0185] A first determination module 4551 is used to determine the grid coordinates of the grid corresponding to each pixel in the image to be displayed, and determine the adjacent grid set of the grid corresponding to each pixel based on each grid coordinate;

[0186] A first acquisition module 4552 is used to acquire the coverage of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value between each grid and the corresponding adjacent grid;

[0187] A second determination module 4553, configured to determine a generation score of each pixel based on each material information, a coverage rate of each adjacent grid and a corresponding depth gradient value;

[0188] A generating module 4554, configured to generate a first surface element of the image to be displayed based on each generation score and a score threshold;

[0189] The coloring module 4555 is used to use the first surface element to color the image to be displayed to obtain a target display image.

[0190] In some embodiments, the second determining module 4553 is further configured to:

[0191] Determining the importance value of each adjacent grid based on each material information and each depth gradient value;

[0192] Determine the difference between the importance value of each adjacent grid and the corresponding coverage rate as the score of each adjacent grid;

[0193] The sum of the scores of the adjacent grids corresponding to the pixels is determined as the generation score of the pixels.

[0194] In some embodiments, the second determining module 4553 is further configured to:

[0195] Determine the self-luminous value of the material corresponding to each pixel based on the information of each material;

[0196] The sum of each self-luminous value and the corresponding depth gradient value is determined as the important value of each adjacent grid.

[0197] In some embodiments, the image processing device 455 further includes:

[0198] A third determination module is used to determine the grid corresponding to each pixel;

[0199] A fourth determination module, configured to obtain a depth value of each grid and a depth value of each adjacent grid, and determine a depth value variation between each grid and each corresponding adjacent grid based on the depth value of each grid and the depth value of each corresponding adjacent grid;

[0200] A fifth determining module, used to determine the distance between each grid and each corresponding adjacent grid;

[0201] The sixth determination module is used to determine the depth gradient value between each grid and each corresponding adjacent grid based on the depth value variation and the corresponding distance corresponding to each grid.

[0202] In some embodiments, the generating module 4554 is further used to:

[0203] When there is a target generation score greater than the score threshold among the generation scores, updating the score threshold;

[0204] determining candidate attribute values ​​based on the target generation score;

[0205] Generating random numbers corresponding to the target generation scores based on a preset random function, wherein the number of the random numbers is the same as the number of the target generation scores;

[0206] When there is a target random number greater than a set threshold value among the random numbers, determining a selected generation score corresponding to the target random number from the target generation score;

[0207] The first surface element is generated based on the candidate attribute value selected to generate a score.

[0208] In some embodiments, the generating module 4554 is further used to:

[0209] Determine a target pixel corresponding to the target generation score;

[0210] Obtaining the world coordinates of the target pixel, the normal of the target pixel, and a preset radius;

[0211] The world coordinates, the normal line and the preset radius are determined as candidate attribute values.

[0212] In some embodiments, the first determining module 4551 is further configured to:

[0213] Obtaining a geometric buffer corresponding to the image to be displayed;

[0214] Acquire a depth value of each pixel in the to-be-displayed image from the geometry buffer, and determine a world coordinate of each pixel based on a pixel position of each pixel and each depth value;

[0215] The world coordinates of each pixel are converted into grid coordinates of the grid corresponding to each pixel.

[0216] In some embodiments, the image processing device 455 further includes:

[0217] A second acquisition module, used for acquiring each world coordinate corresponding to each pixel, each normal of each pixel and a second surface element of each corresponding adjacent grid, wherein the second surface element includes a position, a radius and a normal;

[0218] a seventh determination module, configured to determine a first value of each adjacent grid based on each world coordinate, a position of each adjacent grid and a corresponding radius;

[0219] An eighth determination module, configured to determine a second value of each adjacent grid based on each normal line and a normal line of each corresponding adjacent grid;

[0220] A ninth determination module, configured to obtain occlusion information between each world coordinate and a position of each corresponding adjacent grid, and determine a third value of each adjacent grid based on each occlusion information;

[0221] The tenth determining module is used to determine the coverage rate of each adjacent grid based on the first value, the corresponding second value and the corresponding third value of each adjacent grid.

[0222] In some embodiments, the first surface element includes normal direction information, position information and material attribute information, and the shading module 4555 is further used to:

[0223] Obtaining the initial brightness value of the image to be displayed and a preset illumination model;

[0224] Using the illumination model, the initial light brightness value, the normal direction information, the position information and the material attribute information are superimposed to obtain a superimposed light brightness value;

[0225] The initial brightness value of the image to be displayed is updated using the superimposed brightness value to obtain a target display image.

[0226] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above-mentioned image processing method of the embodiment of the present application.

[0227] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the image processing method provided by the present application embodiment, for example, Figure 5 The image processing method is shown.

[0228] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0229] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0230] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0231] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0232] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Determine the grid coordinates of the grid corresponding to each pixel in the image to be displayed, and determine the adjacent grid set of the grid corresponding to each pixel based on each grid coordinate; Obtaining the coverage of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value between each grid and the corresponding adjacent grid; Determining a generation score of each pixel based on each material information, a coverage rate of each adjacent grid and a corresponding depth gradient value; generating a first surface element of the image to be displayed based on each generated score and a score threshold; The image to be displayed is colored using the first surface element to obtain a target display image.

2. The method according to claim 1, characterized in that The determining, based on each material information, the coverage of each adjacent grid and the corresponding depth gradient value, the generation score of each pixel comprises: Determining the importance value of each adjacent grid based on each material information and each depth gradient value; Determine the difference between the importance value of each adjacent grid and the corresponding coverage rate as the score of each adjacent grid; The sum of the scores of the adjacent grids corresponding to the pixels is determined as the generation score of the pixels.

3. The method according to claim 2, characterized in that The determining the importance value of each adjacent grid based on each material information and each depth gradient value includes: Determine the self-luminous value of the material corresponding to each pixel based on the information of each material; The sum of each self-luminous value and the corresponding depth gradient value is determined as the important value of each adjacent grid.

4. The method according to claim 1, characterized in that The method further comprises: Determine the grid corresponding to each pixel; Acquire the depth value of each grid and the depth value of each adjacent grid, and determine the depth value change between each grid and each corresponding adjacent grid based on the depth value of each grid and the depth value of each corresponding adjacent grid; Determine the distance between each grid and each corresponding adjacent grid; Determine the depth gradient value between each grid and each corresponding adjacent grid based on the depth value variation and the corresponding distance corresponding to each grid.

5. The method according to any one of claims 1 to 4, characterized in that The step of generating the first surface element of the image to be displayed based on the respective generated scores and the score thresholds comprises: When there is a target generation score greater than the score threshold among the generation scores, updating the score threshold; determining candidate attribute values ​​based on the target generation score; Generating random numbers corresponding to the target generation scores based on a preset random function, wherein the number of the random numbers is the same as the number of the target generation scores; When there is a target random number greater than a set threshold value among the random numbers, determining a selected generation score corresponding to the target random number from the target generation score; The first surface element is generated based on the candidate attribute value selected to generate a score.

6. The method according to claim 5, characterized in that The determining of the candidate attribute value based on the target generation score comprises: Determine a target pixel corresponding to the target generation score; Obtaining the world coordinates of the target pixel, the normal of the target pixel, and a preset radius; The world coordinates, the normal line and the preset radius are determined as candidate attribute values.

7. The method according to any one of claims 1 to 4, characterized in that The step of determining the grid coordinates of the grid corresponding to each pixel in the image to be displayed includes: Obtaining a geometric buffer corresponding to the image to be displayed; Acquire a depth value of each pixel in the to-be-displayed image from the geometry buffer, and determine a world coordinate of each pixel based on a pixel position of each pixel and each depth value; The world coordinates of each pixel are converted into grid coordinates of the grid corresponding to each pixel.

8. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire each world coordinate corresponding to each pixel, each normal of each pixel and each corresponding second surface element of each adjacent grid, wherein the second surface element includes a position, a radius and a normal; Determine a first value of each adjacent grid based on each world coordinate, a position of each adjacent grid and a corresponding radius; Determine a second value of each adjacent mesh based on each normal and a corresponding normal of each adjacent mesh; Obtaining occlusion information between each world coordinate and a corresponding position of each adjacent grid, and determining a third value of each adjacent grid based on each occlusion information; Based on the first value, the corresponding second value and the corresponding third value of each adjacent grid, the coverage rate of each adjacent grid is determined.

9. The method according to any one of claims 1 to 4, characterized in that The first surface element includes normal direction information, position information and material attribute information, and the coloring process of the image to be displayed using the first surface element to obtain a target display image includes: Obtaining the initial brightness value of the image to be displayed and a preset illumination model; Using the illumination model, the initial light brightness value, the normal direction information, the position information and the material attribute information are superimposed to obtain a superimposed light brightness value; The initial brightness value of the image to be displayed is updated using the superimposed brightness value to obtain a target display image.

10. An image processing device, characterized in that: The device comprises: A first determination module is used to determine the grid coordinates of the grid corresponding to each pixel in the image to be displayed, and determine the adjacent grid set of the grid corresponding to each pixel based on each grid coordinate; A first acquisition module is used to acquire the coverage of each adjacent grid in each adjacent grid set, the material information of each pixel, and the depth gradient value between each grid and the corresponding adjacent grid; A second determination module, configured to determine a generation score of each pixel based on each material information, a coverage rate of each adjacent grid and a corresponding depth gradient value; A generating module, configured to generate a first surface element of the image to be displayed based on each generation score and a score threshold; A coloring module is used to color the image to be displayed using the first surface element to obtain a target display image.

11. A computer device, characterized in that: The computer device comprises: A memory for storing computer executable instructions; A processor, configured to implement the image processing method according to any one of claims 1 to 9 when executing the computer executable instructions stored in the memory.

12. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the image processing method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instructions are executed by a processor, the image processing method according to any one of claims 1 to 9 is implemented.

Images