Image processing method and system, image processing device

By building a virtual enclosure and point light source, the light parameter distribution information is generated, and the problem of long-term lighting simulation in three-dimensional real-time rendering is solved, and fast rendering and efficient image processing is achieved, which is suitable for low-performance devices.

CN113313808BActive Publication Date: 2024-11-29ALIBABA (SHENZHEN) TECH CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202010880726.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-27
Publication Date
2024-11-29
Estimated Expiration
2040-08-27

AI Technical Summary

Technical Problem

In three-dimensional real-time rendering, lighting simulation calculations take a long time and are difficult to meet real-time requirements. In particular, high dynamic range image generation and graphics processor programmability solutions are not suitable for low-performance devices.

Method used

By constructing a virtual enclosure for the luminous object in the target image, determining the sampling point lighting information and virtual point light source positions on the virtual enclosure, generating optical parameter distribution information, using this information to quickly render the image, or generating high dynamic range images for use in Render.

Benefits of technology

It greatly improves image rendering speed, reduces calculation amount and time-consuming, and is suitable for low-performance devices, without the need for camera to acquire multiple exposed images, simplifying the lighting calculation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113313808B_ABST
    Figure CN113313808B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide an image processing method and system, and an image processing device. The method includes: obtaining a virtual bounding volume of a luminous object in a target image; determining the illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; determining the position of a virtual point light source corresponding to the luminous object; determining the light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points; the light parameter distribution information is used to render the target image. By adopting the technical solution provided in the embodiments of the present application, the light parameter distribution information of the luminous object can be obtained in a relatively simple manner, with a small amount of calculation, and using the light parameter distribution information in real-time rendering can avoid the time-consuming problem of illumination calculation and improve the rendering rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method and system, and an image processing device. Background Art

[0002] The main contradiction in 3D real-time rendering is the image rendering quality and rendering speed. In the real-time rendering process, in order to generate a realistic image, it is necessary to simulate the lighting, but the simulation of lighting is a very time-consuming task and often cannot meet real-time requirements.

[0003] At present, in order to improve authenticity and real-time performance, high dynamic range images are often used as environmental backgrounds to provide lighting; or the programmability of graphics processing units (GPUs) is used to achieve the mapping of environmental lighting information. However, high dynamic range images need to be synthesized by collecting multiple exposure images through the camera; using the programmability of graphics processing units (GPUs) consumes a lot of system performance and is difficult to apply to low-performance devices. Summary of the invention

[0004] The present application provides an image processing method and system, and an image processing method that solve the above problems or at least partially solve the above problems.

[0005] In one embodiment of the present application, an image processing method is provided. The method comprises:

[0006] Obtain a virtual bounding volume of a luminous object in a target image;

[0007] Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0008] Determine the position of the virtual point light source corresponding to the luminous object;

[0009] Determining light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points;

[0010] The light parameter distribution information is used to render the target image.

[0011] In another embodiment of the present application, an image processing method is provided. The method comprises:

[0012] In response to a rendering request for a target image, obtaining light parameter distribution information corresponding to the target image;

[0013] Calculating illumination information corresponding to a plurality of pixels in the target image using the light parameter distribution information;

[0014] Rendering the target image according to illumination information corresponding to a plurality of pixels in the target image;

[0015] The light parameter distribution information includes the virtual point light source position corresponding to the luminous object in the target image and the lighting information corresponding to multiple sampling points on the virtual bounding volume of the luminous object.

[0016] In another embodiment of the present application, an image processing method is provided. The method comprises:

[0017] Obtain a virtual bounding volume of a luminous object in a target image;

[0018] Determining illumination information of a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0019] Generate a high dynamic range image according to illumination information of a plurality of sampling points on the virtual bounding volume;

[0020] The high dynamic range image is used to render the target image.

[0021] In one embodiment of the present application, an image processing device is provided. The image processing device includes: a processor, a memory and a display; wherein:

[0022] A processor is used to obtain a virtual bounding volume of a luminous object in a target image; determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; determine a virtual point light source position corresponding to the luminous object; and determine light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points;

[0023] A memory, used to store the optical parameter distribution information;

[0024] The processor is further configured to obtain the light parameter distribution information in response to a rendering request for the target object, and perform rendering processing on the target image;

[0025] A display is used to display the target image after rendering.

[0026] In one embodiment of the present application, an image processing device is provided. The image processing device includes: a processor, a memory and a display, wherein:

[0027] A processor, configured to obtain a virtual bounding volume of a luminous object in a target image; determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; and generate a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume;

[0028] A memory, used for storing the high dynamic range image;

[0029] The processor is further configured to obtain the high dynamic range image in response to a rendering request for the target object, and perform rendering processing on the target image;

[0030] A display is used to display the target image after rendering.

[0031] In another embodiment of the present application, an image processing method is provided. The method comprises:

[0032] Get a three-dimensional image of the space;

[0033] For the luminous object in the three-dimensional image of the space, configuring a virtual bounding volume;

[0034] Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0035] A high dynamic range image is generated according to the illumination information of a plurality of sampling points on the virtual bounding volume.

[0036] In another embodiment of the present application, an image processing device is provided. The image processing device includes:

[0037] A processor, configured to obtain a three-dimensional spatial image; configure a virtual bounding volume for a luminous object in the three-dimensional spatial image; determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; and generate a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume;

[0038] A memory, used for storing the high dynamic range image;

[0039] The processor is further configured to obtain the high dynamic range image in response to a user's rendering request for the spatial three-dimensional image; and render the spatial three-dimensional image using the high dynamic range image;

[0040] A display is used to display the three-dimensional image of the space after rendering.

[0041] A technical solution provided by an embodiment of the present application obtains a virtual bounding volume of a luminous object in a target image; then determines the lighting information corresponding to multiple sampling points on the virtual bounding volume based on the luminous information of the luminous object; and obtains light parameter distribution information based on the determined virtual point light source position corresponding to the luminous object and the lighting information corresponding to the multiple sampling points, so as to use the light parameter distribution information to achieve the purpose of quickly rendering the target image; the entire solution converts the luminous object in the target image into a virtual point light source, and performs lighting calculations in the lighting mode of a point light source, with a small amount of calculation and a short time consumption, which can greatly improve the image rendering rate.

[0042] Another technical solution provided by the embodiment of the present application is to obtain a virtual bounding volume of a luminous object in a target image, and then determine the illumination information of multiple sampling points on the virtual bounding volume based on the luminous information of the luminous object, and then generate a high dynamic range image based on the illumination information of multiple sampling points on the virtual bounding volume for rendering the target image. In the prior art, a camera collects multiple exposure images and synthesizes them to obtain a high dynamic range image, while in the solution provided by this embodiment, the high dynamic range image is obtained by using the virtual bounding volume of the luminous object and the luminous information of the luminous object, without the participation of data acquisition equipment such as cameras, and is not limited by the performance of actual hardware equipment. When the high dynamic range image is used for real-time rendering, the amount of calculation is small and the time consumption is short, which can greatly improve the image rendering rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of an image processing method provided in one embodiment of the present application;

[0045] Figure 2 An exemplary schematic diagram of calculating illumination information corresponding to a sampling point on a virtual bounding volume provided by an embodiment of the present application;

[0046] Figure 3 An example diagram of a scattered light diagram provided by an embodiment of the present application;

[0047] Figure 4 A schematic diagram of calculating the light intensity corresponding to a pixel point in a target image provided by an embodiment of the present application;

[0048] Figure 5A flowchart of an image processing method provided by another embodiment of the present application;

[0049] Figure 6 A flowchart of an image processing method provided in yet another embodiment of the present application;

[0050] Figure 7 A flowchart of an image processing method provided in yet another embodiment of the present application;

[0051] Figure 8 A schematic diagram of the interface operation and image processing process provided in an embodiment of the present application;

[0052] Fig. 9 A block diagram of the principle structure of an image processing device provided in one embodiment of the present application;

[0053] Fig.10 A structural block diagram of an image processing system provided in one embodiment of the present application;

[0054] Fig.11 A schematic diagram showing the principle of implementing an image processing process by an image processing system provided in an embodiment of the present application;

[0055] Fig.12 A structural block diagram of an image processing device provided in one embodiment of the present application;

[0056] Fig.13 A structural block diagram of an image processing device provided by another embodiment of the present application;

[0057] Fig.14 A schematic diagram of the structure of an image processing device provided in yet another embodiment of the present application;

[0058] Fig.15 A schematic diagram of the structure of an image processing device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0059] Virtual Reality (VR) is a new high-tech technology that has emerged in recent years. It uses computer simulation to generate a virtual world in three-dimensional space, providing users with simulations of vision, hearing, touch and other senses, allowing users to observe things in three-dimensional space in a timely and unlimited manner as if they were in the scene. At present, virtual reality technology has been widely used in various industries (such as architectural decoration and virtual reality games). In the process of virtual reality implementation, real-time rendering technology is often needed to generate images with a strong sense of reality to enhance the user experience. However, the realism of the rendered image depends largely on the simulation calculation of the illumination. In the calculation process, in order to simulate the real environment illumination to the greatest extent, it is necessary to consider not only the effect of direct illumination from the light source, but also indirect illumination factors such as reflection and refraction between scene objects. This is a very time-consuming task, so it often cannot meet the real-time requirements.

[0060] In the prior art, in order to improve authenticity and real-time performance, high dynamic range images can be used as environmental backgrounds to provide environmental lighting information. However, the use of high dynamic range images requires synthesizing high dynamic range images through multiple images before performing lighting calculations, which requires a large amount of calculations and is not very universal for mobile devices. In addition, the programmability of a graphics processing unit (GPU) can also be used to achieve the mapping of environmental lighting information. For example, in a virtual scene of interior design, when a self-luminous object (such as a desk lamp) is the light source in the scene, the self-luminous object can be simplified in a voxelized manner and stored in a storage medium, and finally the voxels are cone tracked during rendering. However, in this way, the voxels will take up a large amount of memory, and the creation and query of voxels will consume GPU time, and the error of the luminous intensity of the voxels will also be large.

[0061] To this end, the embodiment of the present application proposes a solution for constructing a virtual bounding volume for the self-luminous object in the target image to obtain the light parameter distribution information corresponding to the self-luminous object, so as to render the target image using the light parameter distribution information. In order to enable personnel in the technical field to better understand the solution of the present application, the technical solution in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application.

[0062] In some processes described in the specification, claims and the above-mentioned figures of this application, multiple operations appearing in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types. The term "or / and" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist, for example: A or / and B, indicating that A can exist alone, A and B can exist at the same time, and B can exist alone; the character " / " in this application generally indicates that the objects associated before and after are in an "or" relationship. In addition, the following embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making any creative work shall fall within the scope of protection of this application.

[0063] Before describing the solutions provided by the embodiments of the present application, the relevant terms involved in the present application are briefly described.

[0064] Voxelization of self-luminous objects: refers to cutting a self-luminous object into several small stereo pixels, and using each stereo pixel as a light source to generate lighting.

[0065] Direct lighting: The lighting effect produced when light from a light source first hits an object.

[0066] Indirect lighting: The lighting effect produced when the light from a light source is reflected multiple times before hitting an object. For example, the only light source in a room is the sunlight coming in from the window, but the whole room is illuminated. The part illuminated by the sunlight is direct lighting, and the rest is so-called indirect lighting; for another example, when a flashlight shines on the ground, the area outside the projection point of the light (such as the wall) will also be illuminated, which is indirect lighting.

[0067] High Dynamic Range (HDR): Dynamic range refers to the ratio of the "brightest" to the "darkest" contained in the image, that is, the number of grayscale levels from the "brightest" to the "darkest" in the image. The larger the dynamic range, the richer the levels that can be represented and the wider the color space included. High dynamic range (HDR) means that a very high ratio can be achieved from the "brightest" to the "darkest". A high dynamic range image (HDR image for short) is an image that can show a very high dynamic range. In the application of HDR images in image-based lighting, the environment map at a certain point in the scene is obtained, and then the corresponding radiance map is obtained through the high dynamic range recovery algorithm. The light source distribution is obtained in the radiance map, so that the lighting can be calculated.

[0068] Rendering refers to the technology of projecting the model in a three-dimensional scene into a digital image in two dimensions according to the set environment, lighting, material and rendering parameters; while realistic rendering refers to the rendering technology that makes the rendered image as close as possible to the image captured and processed by a real camera.

[0069] Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0070] 101. Acquire a virtual bounding volume of a luminous object in a target image;

[0071] 102. Determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0072] 103. Determine the position of the virtual point light source corresponding to the luminous object;

[0073] 104. Determine light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points;

[0074] The light parameter distribution information is used to render the target image.

[0075] In the above 101, the target image may be a three-dimensional image or a two-dimensional image in a virtual reality scene. The target image may be created artificially using corresponding software, or may be automatically generated by a machine based on relevant data information. Alternatively, the target image is generated by a device with certain data processing capabilities (which may be the executor of the method of this embodiment or other devices) based on the data collected by the collection device after data is collected by the collection device, such as a camera.

[0076] When rendering a target image in an interior design virtual reality scene, it can be divided into outdoor lighting rendering and indoor lighting rendering, wherein the outdoor lighting rendering refers to rendering using outdoor natural lighting (such as sunlight), and the indoor lighting rendering refers to rendering using direct lighting or indirect lighting generated by self-luminous objects such as electric lamps, candles, and televisions in indoor scenes. The application scenario of this embodiment can be indoor light rendering, and the target image is rendered by using the luminous information of the self-luminous object. Specifically, in order to quickly and accurately obtain the lighting information that the luminous object in the target image can provide for the target image, the self-luminous object can be simplified in the form of a virtual enclosing body to obtain a virtual point light source corresponding to the self-luminous object, so as to render the target image using the lighting information of the virtual point light source. That is, a virtual enclosing body is designed to wrap the luminous object, and the luminous object can be located at the center of the virtual enclosing body. The light emitted by the luminous object is reflected on the inner wall of the virtual enclosing body, and a plurality of sampling points on the virtual enclosing body are sampled to obtain a distribution information (or a distribution map) reflecting the lighting situation. Based on this, the above 101 “obtaining a virtual bounding volume of the luminous object in the target image” may specifically include:

[0077] 1011. Obtaining position information and size information of the light-emitting object;

[0078] 1012. Determine the center point of the virtual bounding volume according to the position information;

[0079] 1013. Determine contour data of the virtual bounding volume according to the size information;

[0080] 1014. Obtain the virtual bounding volume according to the center point and the contour data.

[0081] In a specific implementation, the position information and size information of the luminous object can be stored in a storage medium. When the user retrieves the position information and size information of the luminous object to perform the above steps, the user can input the identifier of the luminous object or select the luminous object from the target image through the interactive methods provided by the display interface, such as mouse, keyboard, voice, etc. After the user completes the input or selection, the execution subject of the method of this embodiment will obtain the position information and size information of the luminous object from the corresponding storage medium; and then the center point of the virtual enclosing volume can be determined according to the obtained position information, and the contour data of the virtual enclosing volume can be determined according to the obtained size information. For example, the center of mass of the luminous object determined according to the position information can be used as the center point of the virtual enclosing volume, and then the virtual enclosing volume can be obtained according to the center point and the contour data.

[0082] After determining the virtual bounding volume corresponding to the luminous object, in order to obtain the lighting information corresponding to multiple sampling points on the virtual bounding volume, it is also necessary to configure the corresponding diffuse reflection attribute and / or pure white attribute for the virtual bounding volume, so as to calculate the lighting information of any sampling point on the virtual bounding volume according to the reflected light. That is, after the above step 1014, the following steps may also be included:

[0083] 1015. Configure a diffuse reflection attribute and / or a pure white attribute for the virtual bounding volume.

[0084] In specific implementation, the diffuse reflection property of the virtual enclosure is related to the diffuse reflection coefficient. The reflection property of the virtual enclosure can be adjusted by setting the diffuse reflection coefficient. If the diffuse reflection coefficient value is close to 1, it means that the reflected light is close to the incident light intensity. If the diffuse reflection coefficient is close to 0, it means that the surface of the virtual enclosure absorbs most of the incident light. Since the intensity of the reflected light determines the brightness of the surface of the object, a material with a diffuse reflection coefficient close to 1 can be configured for the virtual enclosure. Since the color of the virtual enclosure will affect the reflectivity, the whiter the color, the higher the reflectivity. In order to make the virtual enclosure have a higher reflectivity, its color attribute is usually configured as a pure white attribute, so that the lighting information corresponding to multiple sampling points on the virtual enclosure can be obtained more realistic.

[0085] In the above, the virtual bounding volume can be a bounding box composed of multiple faces, such as a cube, a cuboid, etc., or a bounding sphere composed of one face, or other shapes, which are not specifically limited in this application. Taking into account the actual lighting effect, the virtual bounding volume that can be used in this application is a virtual bounding sphere. Based on this, in the case where the virtual bounding volume is a virtual bounding sphere, an implementable solution of the above step 1013 "determining the contour data of the virtual bounding volume according to the size information" is: determining the maximum outer envelope size according to the size information; determining the sphere radius according to the maximum outer envelope size; wherein the sphere radius is greater than the maximum outer envelope size.

[0086] In the above step 102, the virtual bounding volume sampling technology of offline rendering can be used. If the virtual bounding volume is a sphere, the illumination information reflected on the inner wall of the virtual bounding volume by the luminous information of the luminous object is sampled by using the hemispherical sampling technology of offline rendering.

[0087] Specifically, the lighting information may include but is not limited to: lighting intensity, lighting color, etc.

[0088] In an achievable technical solution, the above step 102 of “determining the illumination information corresponding to any sampling point on the virtual bounding volume according to the luminous information of the luminous object” may specifically include:

[0089] 1021. Calculate the light intensity of the luminous light along a direction to a sampling point on the virtual bounding volume based on the luminous information of the luminous object;

[0090] 1022. Use the direction and the light intensity as illumination information corresponding to the sampling point.

[0091] In practical applications, affected by the material of the object, the scattering phenomenon that occurs when the light emitted by the light source contacts the object usually includes refraction and reflection. The light refracted into the object will continue to intersect with the particles inside the object, and refraction and reflection will also occur. A part of it will be re-emitted from the surface of the object, and the re-emitted light will also affect the light intensity on the surface of the object. Therefore, in order to obtain more realistic lighting, when calculating the light intensity of a sampling point on the virtual bounding volume, it is necessary to combine the light intensity generated by the above two scattering directions at the sampling point. For ease of distinction, in the calculation process, diffuse reflection will be used to represent how the surface of the virtual bounding volume reflects light, while specular reflection will represent how much light is refracted, absorbed, and scattered out of the surface.

[0092] For an example, see Figure 2 As shown, the virtual enclosing volume corresponding to the luminous object 01 is the virtual enclosing sphere 02, and only part of the surface of the virtual enclosing sphere 02 is shown here. When the light ray l emitted by the luminous object 01 shines on point P on the inner surface of the virtual enclosing sphere 02, the change in light intensity at point P will be determined by the diffuse reflection light and the specular reflection light. Among them, the light intensity generated by the diffuse reflection light at point P can be calculated by the angle between the incident light ray l and the normal line of the virtual enclosing sphere surface, which can be calculated by formula (1):

[0093] I Diffuse =K d *I l *cosθ,cosθ=N·L (1)

[0094] Among them, K d is the diffuse reflection coefficient of the object surface, I l is the light intensity of the incident light l, N is the unit normal vector of the incident point P, and L is the unit vector from the incident point P to the luminous object.

[0095] The light intensity of the specular reflected light at point P can be calculated using formula (2):

[0096] I Specular =K s I l (R.V) α (2)

[0097] Among them, K Sis the high light reflection coefficient, R is the reflected light obtained by the reflection law, V is the vector from the surface to the human eye, α is the reflection constant, the larger α is, the higher I specular As the attenuation speed is faster. Then, the light intensity at point P can be expressed as: I p =I Diffuse +I Specular The illumination information corresponding to point P can include the light intensity I at point P. p And the direction S of the luminous light l reaching point P (that is, the opposite direction of the unit vector L).

[0098] The same method can be used to calculate the illumination information corresponding to any sampling point on the surface of the virtual enclosing sphere 02 corresponding to the luminous object. The illumination information of the sampling points on the surface of the virtual enclosing sphere can be understood as the spherical distribution of the virtual point light source corresponding to the luminous object, which is the idea of ​​converting the luminous object into a point light source proposed in this embodiment. Since the reverse extension lines of the directions corresponding to the light rays emitted outward by the virtual point light source will intersect with the center point of the virtual enclosing sphere, the direction of any pixel point in the target image relative to the center point of the virtual enclosing sphere can also be used as the direction of any pixel point in the target image relative to the virtual point light source. Based on this, one implementable solution for the above 103 "determining the position of the virtual point light source corresponding to the luminous object" is to use the position of the center point of the virtual enclosing body in the target image as the position of the virtual point light source.

[0099] In the above 104, when determining the light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the multiple sampling points, considering that the symmetrical sampling points in the upper and lower hemispheres of the virtual enclosing sphere have the same light intensity, when determining the light parameter distribution information, it is only necessary to sample the hemispherical area of ​​the virtual enclosing sphere. Alternatively, the characteristics of the luminous object, such as a desk lamp placed on a desktop, only the upper hemisphere of the virtual enclosing sphere needs to be sampled. Alternatively, the luminous object is a ceiling lamp, and only the lower hemisphere of the virtual enclosing sphere needs to be sampled. Specifically, in the process of sampling the hemispherical surface of the virtual enclosing sphere, the spherical surface can be decomposed into regions using a method similar to the longitude and latitude coordinate segmentation, and then a certain number of samples are taken for each sub-region, and the illumination information of each sampling point is calculated, so as to obtain the illumination information of multiple sampling points on the hemispherical surface.

[0100] The technical solution provided by this embodiment obtains a virtual bounding box of a luminous object in a target image; then determines the lighting information corresponding to a plurality of sampling points on the virtual bounding box based on the luminous information of the luminous object; and obtains light parameter distribution information based on the determined virtual point light source position corresponding to the luminous object and the lighting information corresponding to the plurality of sampling points, so as to achieve the purpose of quickly rendering the target image by using the light parameter distribution information; the entire solution converts the luminous object in the target image into a virtual point light source, and performs lighting calculations in the lighting manner of a point light source, with a small amount of calculation and a short time consumption, which can greatly improve the image rendering rate.

[0101] Furthermore, the image processing method provided in this embodiment may also include:

[0102] 105a. Encode the optical parameter distribution information to compress it to obtain compressed data;

[0103] 105b. Store the compressed data.

[0104] In a specific implementation, in order to make the light parameter distribution information more efficiently stored and transmitted, the light parameter distribution information can be compressed using RGBE, OpenEXR, LogLuv and other encoding methods to obtain corresponding compressed data, and the compressed data is stored in the corresponding storage medium in the form of a longitude and latitude HDR image, so as to facilitate calling when rendering the target image. Among them, the storage format of the HDR image can be Latitude or Longitude format, which can provide a one-to-one mapping from spherical coordinates.

[0105] Furthermore, the image processing method provided in this embodiment may also include:

[0106] 106a. In response to a rendering request for the target image, obtain the light parameter distribution information;

[0107] 106b. Calculate illumination information of a plurality of pixels in the target image using the light parameter distribution information;

[0108] 106c. Render the target object according to the illumination information corresponding to the plurality of pixels in the target image.

[0109] In the above 106a, after receiving the rendering request sent by the user for the target object, the light parameter distribution information for rendering the target image can be obtained from the storage medium, and the corresponding scattered illumination map and / or high light reflection map (such as Figure 3As shown in the figure), the resolved scatter image and / or highlight image can be used as a lookup table of the illumination value corresponding to any pixel in the target image, so that the amount of illumination calculation can be reduced during rendering, which can speed up the rendering speed. 1 (φ,θ) to find the illumination value I N1 (N 1 ).

[0110] In the above step 106b, “using the light parameter distribution information to calculate the light information corresponding to any pixel point in the target image” may specifically include:

[0111] S1. Determine direction and distance according to the position of the virtual point light source and the position of a pixel in the target image;

[0112] S2. Based on the light parameter distribution information, obtaining the light intensity corresponding to the direction;

[0113] S3. Calculate the light intensity corresponding to the pixel point according to the distance and the light intensity corresponding to the direction.

[0114] For an example, see Figure 4 As shown, the target image contains an object 03; a pixel 4 on the object 03; and the light intensity corresponding to the pixel 4 is calculated. Assume that, Figure 4 The position of the virtual point light source o in the target image corresponding coordinate system is (x 0 ,y 0 , z 0 ); The position of pixel 4 in the target image corresponding coordinate system is (x 4 ,y 4 , z 4 ); therefore, the direction and distance can be determined based on the position of the virtual point light source o and the position of the pixel point 4. The direction can be characterized as: the angle between the line connecting the virtual point light source o and the pixel point 4 and the coordinate axis and / or coordinate plane in the coordinate system corresponding to the target image. For example, Figure 4 The direction of the line connecting the virtual point light source o and the pixel point 4 can be represented by two angles, namely the angle θ between the line and the normal N1, and the angle between the line and the coordinate plane. The distance between the virtual point light source o and the pixel point 4 can be directly calculated using the coordinate values ​​of the two points.

[0115] It can be seen from the above content that the light parameter distribution information includes: direction and the light intensity corresponding to the direction. Therefore, after determining the direction of the line connecting the virtual point light source o and the pixel point 4, the light intensity corresponding to the direction of the line connecting the virtual point light source o and the pixel point 4 can be obtained based on the light parameter distribution information. After obtaining the light intensity, the light intensity corresponding to the pixel point 4 is calculated based on the distance between the virtual point light source o and the pixel point 4. The light intensity will attenuate as the distance increases, so the preset attenuation formula can be used to calculate the light intensity corresponding to the pixel point 4. The attenuation formula is not specifically limited in this embodiment, and most existing rendering engines contain their own attenuation formulas.

[0116] The above steps 106b and 106c can be implemented by starting a rendering engine. Specifically, for the rendering process of the target image, reference can be made to the corresponding content in the prior art, and this embodiment does not specifically limit this. This embodiment focuses on providing a solution for converting the luminous object in the target image into a point light source to obtain light parameter distribution information that can accelerate the image rendering efficiency. The light parameter distribution information can be used by the corresponding rendering engine to render the target image.

[0117] It is also necessary to add that in this embodiment: if the luminous object in the target image is a fluorescent lamp, the light emitted by it does not need to consider the color of the light, and only the light intensity is concerned; that is, the light parameter distribution information includes the direction and the light intensity corresponding to the direction. However, when the luminous object is a colored lamp with a lampshade, in addition to the light intensity, the light color must also be considered; therefore, the light parameter distribution information must include: direction, the light intensity corresponding to the direction, and the light color corresponding to the direction.

[0118] When the light parameter distribution information includes the light color corresponding to the direction, the light color corresponding to the direction needs to be obtained in the above step S2, so that the light color can also be taken into consideration in the subsequent rendering process of the target image.

[0119] The process of obtaining the light parameter distribution information through steps 101 to 104 in this embodiment can be understood as a preprocessing stage, which is an offline stage, that is, the part where the extraction is completed. When real-time rendering of the image is required, the light parameter distribution information is retrieved and the image is rendered. The amount of calculation is small and the time consumption is short, which can greatly improve the image rendering rate. The following embodiment corresponds to the technical solution from the perspective of implementing image rendering from the perspective of calling the light parameter distribution information. Specifically, Figure 5 FIG. 2 is a flow chart of an image processing method provided by another embodiment of the present application. Figure 5 As shown, the method comprises the following steps:

[0120] 201. In response to a rendering request for a target object, obtain light parameter distribution information corresponding to a target image;

[0121] 202. Calculate illumination information corresponding to a plurality of pixel points in the target image using the light parameter distribution information;

[0122] 203. Rendering the target object according to illumination information corresponding to a plurality of pixels in the target image;

[0123] The light parameter distribution information includes the virtual point light source position corresponding to the luminous object in the target image and the lighting information corresponding to multiple sampling points on the virtual bounding volume of the luminous object.

[0124] The above 202 “using the light parameter distribution information to calculate the light information corresponding to any pixel in the target image” may specifically include:

[0125] 2021. Determine a direction and a distance according to a position of the virtual point light source and a position of a pixel in the target image;

[0126] 2022. Obtaining light intensity corresponding to the direction based on the light parameter distribution information;

[0127] 2022. Calculate the light intensity corresponding to the pixel point according to the distance and the light intensity corresponding to the direction.

[0128] For the specific implementation of the above 201 to 203, please refer to the corresponding parts in other embodiments, which will not be repeated here.

[0129] The technical solution provided in this embodiment responds to a rendering request for a target object and renders the target image based on the light parameter distribution information corresponding to the acquired target image. Since the light parameter distribution information is composed of the virtual point light source position corresponding to the luminous object in the target image and the lighting information corresponding to multiple sampling points on the virtual bounding volume of the luminous object, the lighting information corresponding to any pixel point in the target image can be calculated by the lighting method of the virtual point light source. This lighting calculation method has a small amount of calculation, is short in time, and can greatly improve the rendering rate.

[0130] Furthermore, the image processing method provided in this embodiment may also include the following steps:

[0131] 204a, obtaining a virtual bounding volume of the luminous object in the target image;

[0132] Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0133] 204b, determining the position of the virtual point light source corresponding to the luminous object;

[0134] 204c. Determine light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the multiple sampling points.

[0135] The above-mentioned 204a “obtaining a virtual bounding volume of the luminous object in the target image” may specifically include:

[0136] Acquire the position information and size information of the light-emitting object;

[0137] Determining the center point of the virtual bounding volume according to the position information;

[0138] Determining contour data of the virtual bounding volume according to the size information;

[0139] The virtual bounding volume is obtained according to the center point and the contour data.

[0140] Furthermore, after step 204, the method may further include:

[0141] 204a. Configure diffuse reflection attribute and pure white attribute for the virtual bounding volume.

[0142] In the above, the virtual bounding volume can be a bounding box composed of multiple faces, such as a cube, a cuboid, etc., or a bounding sphere composed of one face, or other shapes, which are not specifically limited in this application. Considering the actual lighting effect, the virtual bounding volume preferably used in this application is a bounding sphere. Based on this, in the case where the virtual bounding volume is a virtual bounding sphere, an achievable solution for the above "determining the contour data of the virtual bounding volume according to the size information" is:

[0143] Determining a maximum outer envelope size according to the size information;

[0144] The spherical radius is determined according to the maximum outer envelope size; wherein the spherical radius is greater than the maximum outer envelope size.

[0145] It should be noted here that: for the contents of each step not fully described in detail in the method provided in this embodiment, reference can be made to the corresponding contents in the above embodiments, which will not be repeated here. In addition, in addition to the above steps, the method provided in this embodiment may also include other parts or all of the steps in the above embodiments, which can be specifically referred to the corresponding contents in the above embodiments, which will not be repeated here.

[0146] It should also be added that the light parameter distribution information determined by the solution provided in this embodiment can be reused when rendering a type of scene. For example, in the field of virtual house design applications, design users can use three-dimensional applications (such as Figure 8) to create a virtual space with the same size and room type as the target physical house. Then, use the tools provided by the 3D application (such as sofa module, wallpaper module, table and chair module, lamp module, etc.) to add furniture, lamps, wallpaper or wall paint, etc. to the virtual space by dragging, inputting parameters, etc. Figure 8 The table lamp shown in the figure is the light-emitting object in this embodiment. The scheme provided in this embodiment is used to determine the light parameter distribution information; during this rendering, the light parameter distribution information can be used to render the virtual space. Figure 8 The table lamp shown in the figure is only the other furniture, wallpaper, etc. that have changed. At this time, when rendering the virtual space, there is no need to execute the steps of the method described in this embodiment. The light parameter distribution information generated last time can be directly reused to render the virtual space designed this time. That is, the method provided in this embodiment can also include the following steps:

[0147] 205. Store the identifier of the light-emitting object in association with the light parameter distribution information.

[0148] Furthermore, the method provided in this embodiment may also include:

[0149] 206. In response to a rendering request for the image to be rendered, obtain an identifier of a luminous object in the image to be rendered;

[0150] 207. Using the identifier of the luminous object in the image to be rendered, query whether light parameter distribution information associated with the identifier is stored;

[0151] 208. When optical parameter distribution information corresponding to the identifier is stored, acquiring the optical parameter distribution information;

[0152] 209. Render the image to be rendered using the light parameter distribution information.

[0153] In specific implementation, the technical solution provided in this embodiment may be an optional solution. For example, a 3D application has a selection control, and a user can touch the selection control to select whether to use the solution provided in this embodiment for rendering. That is, the method provided in this embodiment may also include the following steps:

[0154] 210. In response to a selection operation of the user, determine a rendering mode selected by the user;

[0155] 211. When the rendering mode selected by the user is the first mode, trigger the step of acquiring a virtual bounding volume of the luminous object in the target image, so as to execute subsequent steps until the light parameter distribution information is determined;

[0156] 212. When the rendering mode selected by the user is the second mode, call a rendering algorithm corresponding to the second mode, and use the rendering algorithm to complete rendering of the target image.

[0157] As mentioned above, the rendering algorithm corresponding to the second method may be other existing algorithms except the scheme of this embodiment, such as simplifying the self-luminous object in a voxelized manner. Rendering is a method of performing cone tracking on voxels, and this embodiment does not make any specific limitation on this.

[0158] Figure 6 FIG. 2 is a flow chart of an image processing method provided by another embodiment of the present application. Figure 6 As shown, the method includes:

[0159] 301. Acquire a virtual bounding volume of a luminous object in a target image;

[0160] 302. Determine illumination information of a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0161] 303. Generate a high dynamic range image according to illumination information of a plurality of sampling points on the virtual bounding volume;

[0162] The high dynamic range image is used to render the target image.

[0163] In the above 301, “obtaining a virtual bounding volume of the luminous object in the target image” may specifically include:

[0164] 3011. Obtaining position information and size information of the light-emitting object;

[0165] 3012. Determine the center point of the virtual bounding volume according to the position information;

[0166] 3013. Determine contour data of the virtual bounding volume according to the size information;

[0167] 3014. Obtain the virtual bounding volume according to the center point and the contour data.

[0168] Furthermore, after the above step 301, the above method may further include the following steps:

[0169] A diffuse reflection attribute and / or a pure white attribute is configured for the virtual bounding volume.

[0170] In an achievable technical solution, the virtual enclosing volume may be a virtual sphere; accordingly, an achievable solution for the above-mentioned "determining the contour data of the virtual enclosing volume according to the size information" is: determining the maximum outer envelope size according to the size information; determining the spherical radius according to the maximum outer envelope size; wherein the spherical radius is greater than the maximum outer envelope size.

[0171] Furthermore, the above 302 "determining the lighting information corresponding to any sampling point on the virtual enclosure according to the luminous information of the luminous object" may specifically include: calculating the light intensity of the luminous light along a direction to a sampling point on the virtual enclosure based on the luminous information of the luminous object; and using the direction and the light intensity as the lighting information corresponding to the sampling point.

[0172] In the above 303, the high dynamic range image (HDR image for short) can be obtained by rendering and calculation based on the illumination information of multiple sampling points on the virtual enclosing volume. In the prior art, the HDR image is obtained by collecting multiple images with different exposure levels by a camera, and then synthesizing the multiple images with different exposure levels. The present embodiment obtains the HDR image by rendering based on the illumination information of multiple sampling points on the virtual enclosing volume. With the HDR image, the HDR mapping technology used in the prior art can be used to complete the rendering process of the target image. Using the HDR image to render the target image in real time has a small amount of calculation and a short time consumption, which can greatly improve the rendering rate.

[0173] It should be noted here that the rendering process of the HDR image in this embodiment can be simply understood as the following process: using the sampling technology of offline rendering, in the correct, refer to the corresponding content in the prior art.

[0174] It should be noted here that: for the contents of each step not fully described in detail in the method provided in this embodiment, reference can be made to the corresponding contents in the above embodiments, which will not be repeated here. In addition, in addition to the above steps, the method provided in this embodiment may also include other parts or all of the steps in the above embodiments, which can be specifically referred to the corresponding contents in the above embodiments, which will not be repeated here.

[0175] The target image in the above embodiments may be a three-dimensional image, a two-dimensional image, etc., and this application does not specifically limit this. The following takes the target image as a spatial three-dimensional image as an example to illustrate the technical solution provided by this application. The spatial three-dimensional image can be designed by a user through a three-dimensional application (or software) through operations such as mapping. For example, in the field of virtual house design applications, a design user can create a virtual space with the same size and room type as the target physical house through the tools provided by the three-dimensional application, and then use the tools provided by the three-dimensional application (such as sofa modules, wallpaper modules, table and chair modules, lamp modules, etc.) to add furniture, lamps, wallpaper or wall paint, etc. in the virtual space through operations such as dragging and parameter input. After completing the decoration design, the designer can click on the rendering control provided by the application to render the spatial three-dimensional image. The solution provided in this embodiment is only for the lighting rendering of the spatial three-dimensional image, and the rendering of the relevant texture, etc. is not involved in this embodiment.

[0176] Figure 7 FIG. 2 is a flow chart of an image processing method provided by another embodiment of the present application. Figure 7 As shown, the method includes:

[0177] 401. Obtain a three-dimensional spatial image;

[0178] 402. Configure a virtual bounding volume for the luminous object in the three-dimensional spatial image;

[0179] 403. Determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0180] 404. Generate a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume.

[0181] See also Figure 8 As shown in the figure, users can use the tools provided in the client application APP to arrange various home decoration modules, such as sofas, tables, lamps, etc., in a pre-designed three-dimensional space by dragging and dropping. Users can render the entire room after designing multiple rooms, or trigger rendering after completing the home decoration design in a local area of ​​the room. For example, users can click on Figure 8 Render control in to trigger the rendering process.

[0182] In an achievable technical solution, the above step 402 of “configuring a virtual bounding volume for the luminous object in the three-dimensional spatial image” may specifically include:

[0183] 4021. Obtaining position information and size information of the light-emitting object;

[0184] 4022. Determine the center point of the virtual bounding volume according to the position information;

[0185] 4023. Determine contour data of the virtual bounding volume according to the size information;

[0186] 4024. Obtain the virtual bounding volume according to the center point and the contour data.

[0187] Furthermore, the method provided in this embodiment includes the following steps:

[0188] 405. In response to a user's rendering request for the spatial three-dimensional image, acquiring the high dynamic range image;

[0189] 406. Render the spatial three-dimensional image using the high dynamic range image.

[0190] It should be noted here that: for the contents of each step not fully described in detail in the method provided in this embodiment, reference can be made to the corresponding contents in the above embodiments, which will not be repeated here. In addition, in addition to the above steps, the method provided in this embodiment may also include other parts or all of the steps in the above embodiments, which can be specifically referred to the corresponding contents in the above embodiments, which will not be repeated here.

[0191] The image processing methods provided in the above embodiments can be implemented by a single machine, for example, by an image processing device, which can be a desktop computer, a laptop computer, a tablet computer, a smart phone, a smart wearable device, etc. with certain image processing capabilities, and this embodiment does not specifically limit this. Fig. 9 As shown, the image processing device provided in this embodiment includes: a processor 1102, a memory 1101, a display 1105, etc. Among them,

[0192] Processor 1102 is used to obtain a virtual bounding volume of a luminous object in a target image; determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; determine a virtual point light source position corresponding to the luminous object; and determine light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points;

[0193] Memory 1101, used to store the optical parameter distribution information;

[0194] The processor 1102 is further configured to obtain the light parameter distribution information in response to a rendering request for the target object, and perform rendering processing on the target image;

[0195] The display 1105 is used to display the target image after rendering.

[0196] Specifically, in addition to the above functions, the processor may also implement other functions, for details, please refer to the description of the above embodiments.

[0197] Further, if Fig. 9 As shown, the image processing device also includes: a communication component 1103, a power supply component 1104 and other components. Fig. 9 Only some components are shown schematically, which does not mean that the image processing device only includes Fig. 9 Components shown.

[0198] Another embodiment of the present application further provides an image processing device. The structure of the image processing device is as follows: Fig. 9 The difference is that the image processing device in this embodiment includes some components whose functions are different from those in the above embodiments. Specifically, the image processing device in this embodiment includes:

[0199] A processor, configured to obtain a virtual bounding volume of a luminous object in a target image; determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; and generate a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume;

[0200] A memory, used for storing the high dynamic range image;

[0201] The processor is further configured to obtain the high dynamic range image in response to a rendering request for the target object, and perform rendering processing on the target image;

[0202] A display is used to display the target image after rendering.

[0203] Similarly, the processor described in this embodiment may realize other functions in addition to the above functions, and the details may refer to the description of the previous embodiments.

[0204] Another embodiment of the present application provides an image processing device. The structure of the image processing device is as follows: Fig. 9 The difference is that the image processing device in this embodiment includes some components whose functions are different from those in the above embodiments. Specifically, the image processing device in this embodiment includes:

[0205] A processor, configured to obtain a three-dimensional spatial image; configure a virtual bounding volume for a luminous object in the three-dimensional spatial image; determine illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; and generate a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume;

[0206] A memory, used for storing the high dynamic range image;

[0207] The processor is further configured to obtain the high dynamic range image in response to a user's rendering request for the spatial three-dimensional image; and render the spatial three-dimensional image using the high dynamic range image;

[0208] A display is used to display the three-dimensional image of the space after rendering.

[0209] Similarly, the processor described in this embodiment may realize other functions in addition to the above functions, and the details may refer to the description of the previous embodiments.

[0210] Alternatively, the image processing methods provided in the above embodiments may be applied to Fig.10 In the system architecture environment composed of the client 100 and the server 200 shown in FIG. Fig.10 As shown, the server 200 is connected to the client 100 through a network, and the network includes but is not limited to: a wide area network, a metropolitan area network or a local area network. The client can be any terminal device such as a desktop computer, a tablet computer, a mobile phone, a smart wearable device, etc.; the server can be a common server, a server cluster, a cloud or a virtual server, etc., which is not specifically limited in the embodiment of the present application. The image processing method of the embodiment of the present application can be executed by the server, can also be executed by the client, and can also be executed partially by the server and partially by the client.

[0211] like Fig.11 In the embodiment shown, the generation of the high dynamic range image is performed by the server 200. The rendering process of rendering the target image using the high dynamic range image can be performed by the client 100. The user can design a three-dimensional space through the client 100, such as virtual home decoration design. The target image obtained after the design is completed is as shown in FIG. Fig.11 The spatial three-dimensional image in the target image is sent to the server 200. The server 200 may be a cloud server based on cloud computing. The server 200 determines the luminous object (i.e. Fig.11 In the case of a central desk lamp, a virtual enclosure corresponding to the luminous object can be obtained based on the position information and size information of the luminous object. The virtual enclosure will generate illumination information on the surface due to the influence of the luminous information of the luminous object. The illumination information of multiple sampling points on the virtual enclosure can be determined by calculation. A high dynamic range image can be generated based on the illumination information of multiple sampling points on the virtual enclosure; the high dynamic range image is sent to the client 100; the client 100 can render the target image based on the received high dynamic range graphics. The whole scheme is simple and efficient to implement, and the obtained high dynamic range image has high precision; and when the high dynamic range image is used for real-time rendering, the illumination calculation can be performed in the form of a virtual point light source, with a small amount of calculation and short time consumption, which can greatly improve the rendering rate.

[0212] It should be noted that the above application scenarios are only provided to facilitate understanding of the spirit and principle of the present application, and the technical solutions provided by the above embodiments are not limited in this respect. On the contrary, the technical solutions provided by the above embodiments can be applied to any applicable scenario.

[0213] Fig.12 FIG. 1 shows a structural block diagram of an image processing device provided by an embodiment of the present application. Fig.12 As shown, the image processing device includes: an acquisition module 601 and a determination module 602; wherein,

[0214] An acquisition module 601 is used to acquire a virtual bounding volume of a luminous object in a target image;

[0215] Determination module 602 is used to determine the lighting information corresponding to multiple sampling points on the virtual bounding volume based on the luminous information of the luminous object; it is also used to determine the virtual point light source position corresponding to the luminous object; and it is also used to determine the light parameter distribution information based on the virtual point light source position and the lighting information corresponding to the multiple sampling points; wherein the light parameter distribution information is used to render the target image.

[0216] The technical solution provided by this embodiment obtains a virtual bounding box of a luminous object in a target image; then determines the lighting information corresponding to a plurality of sampling points on the virtual bounding box based on the luminous information of the luminous object; and obtains light parameter distribution information based on the determined virtual point light source position corresponding to the luminous object and the lighting information corresponding to the plurality of sampling points, so as to achieve the purpose of quickly rendering the target image by using the light parameter distribution information; the entire solution converts the luminous object in the target image into a virtual point light source, and performs lighting calculations in the lighting manner of a point light source, with a small amount of calculation and a short time consumption, which can greatly improve the image rendering rate.

[0217] Furthermore, the acquisition module 601 is specifically used for:

[0218] Acquire the position information and size information of the light-emitting object;

[0219] Determining the center point of the virtual bounding volume according to the position information;

[0220] Determining contour data of the virtual bounding volume according to the size information;

[0221] The virtual bounding volume is obtained according to the center point and the contour data.

[0222] Furthermore, the above device may also include:

[0223] The configuration module 603 is used to configure diffuse reflection properties and / or pure white properties for the virtual bounding volume.

[0224] Furthermore, the virtual bounding volume is a virtual sphere; and the acquisition module 601 is further specifically used to: determine the maximum outer envelope size according to the size information; determine the sphere radius according to the maximum outer envelope size; wherein the sphere radius is greater than the maximum outer envelope size.

[0225] Furthermore, the determination module 602 is specifically configured to: use the position of the center point of the virtual bounding volume in the target image as the position of the virtual point light source.

[0226] Furthermore, the determination module 602 is also specifically used to: calculate the light intensity of the luminous light along a direction to a sampling point on the virtual bounding volume based on the luminous information of the luminous object; and use the direction and the light intensity as the lighting information corresponding to the sampling point.

[0227] Furthermore, the above device may also include:

[0228] The storage module 603 is used to encode the optical parameter distribution information to compress it to obtain compressed data; and store the compressed data.

[0229] Furthermore, the above device may also include:

[0230] A response module, configured to obtain the light parameter distribution information in response to a rendering request for the target image;

[0231] A calculation module, used to calculate the illumination information of a plurality of pixel points in the target image using the light parameter distribution information;

[0232] The rendering module is used to render the target object according to the illumination information corresponding to a plurality of pixels in the target image.

[0233] Furthermore, the calculation module is specifically used to: determine the direction and distance based on the position of the virtual point light source and the position of a pixel point in the target image; obtain the light intensity corresponding to the direction based on the light parameter distribution information; and calculate the light intensity corresponding to the pixel point based on the distance and the light intensity corresponding to the direction.

[0234] Furthermore, the device provided in this embodiment may also include a selection module. The selection module is used to:

[0235] In response to a selection operation by the user, determining a rendering mode selected by the user;

[0236] When the rendering mode selected by the user is the first mode, triggering the step of acquiring a virtual bounding volume of the luminous object in the target image, so as to execute subsequent steps until the light parameter distribution information is determined;

[0237] When the rendering mode selected by the user is the second mode, a rendering algorithm corresponding to the second mode is called, and the rendering of the target image is completed by using the rendering algorithm.

[0238] It should be noted here that: the image processing device provided in this embodiment can achieve the above Figure 1 The technical solution described in the embodiment of the image processing method shown in the figure, the specific implementation principle of each module or unit can be referred to the above Figure 1 The corresponding contents in the illustrated embodiment of the image processing method will not be repeated here.

[0239] Fig.13 FIG. 2 shows a structural block diagram of an image processing device provided by another embodiment of the present application. Fig.13 As shown, the image processing device includes: a response module 701, a calculation module 702 and a rendering module 703; wherein,

[0240] A response module 701 is used to obtain the light parameter distribution information in response to a rendering request for the target image;

[0241] A calculation module 702 is used to calculate the illumination information of a plurality of pixel points in the target image using the light parameter distribution information;

[0242] The rendering module 703 is used to render the target object according to the illumination information corresponding to the multiple pixels in the target image.

[0243] The technical solution provided by this embodiment obtains the virtual bounding volume of the luminous object in the target image, and then determines the illumination information of multiple sampling points on the virtual bounding volume according to the luminous information of the luminous object, and then generates a high dynamic range image according to the illumination information of multiple sampling points on the virtual bounding volume for rendering the target image. In the prior art, a camera collects multiple exposure images and synthesizes them to obtain a high dynamic range image, while the high dynamic range image in the solution provided by this embodiment is obtained by using the virtual bounding volume of the luminous object and the luminous information of the luminous object, without the participation of data acquisition equipment such as cameras, and is not limited by the performance of actual hardware equipment. When the high dynamic range image is used for real-time rendering, the amount of calculation is small and the time consumption is short, which can greatly improve the image rendering rate.

[0244] Furthermore, the calculation module 702 is specifically used to: determine the direction and distance according to the position of the virtual point light source and the position of a pixel point in the target image; obtain the light intensity corresponding to the direction based on the light parameter distribution information; and calculate the light intensity corresponding to the pixel point according to the distance and the light intensity corresponding to the direction.

[0245] Furthermore, the above device may also include:

[0246] An acquisition module, used for acquiring a virtual bounding volume of a luminous object in a target image;

[0247] A determination module is used to determine the lighting information corresponding to multiple sampling points on the virtual bounding volume based on the luminous information of the luminous object; it is also used to determine the virtual point light source position corresponding to the luminous object; and it is also used to determine the light parameter distribution information based on the virtual point light source position and the lighting information corresponding to the multiple sampling points; wherein the light parameter distribution information is used to render the target image.

[0248] Furthermore, the acquisition module is specifically used to: acquire position information and size information of the luminous object; determine the center point of the virtual enclosing volume according to the position information; determine contour data of the virtual enclosing volume according to the size information; and obtain the virtual enclosing volume according to the center point and the contour data.

[0249] Furthermore, the above device may also include:

[0250] A configuration module is used to configure diffuse reflection properties and / or pure white properties for the virtual bounding volume.

[0251] Furthermore, the virtual bounding volume is a virtual sphere; and

[0252] The acquisition module is further specifically used to: determine a maximum outer envelope size according to the size information; determine a spherical radius according to the maximum outer envelope size; wherein the spherical radius is greater than the maximum outer envelope size.

[0253] It should be noted here that: the image processing device provided in this embodiment can achieve the above Figure 5 The technical solution described in the embodiment of the image processing method shown in the figure, the specific implementation principle of each module or unit can be referred to the above Figure 5 The corresponding contents in the illustrated embodiment of the image processing method will not be repeated here.

[0254] Fig.14 FIG. 2 shows a structural block diagram of an image processing device provided by another embodiment of the present application. Fig.14 As shown, the image processing device includes: an acquisition module 801, a determination module 802 and a generation module 803; wherein,

[0255] An acquisition module 801 is used to acquire a virtual bounding volume of a luminous object in a target image;

[0256] A determination module 802 is used to determine illumination information of a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object;

[0257] A generating module 803, configured to generate a high dynamic range image according to illumination information of a plurality of sampling points on the virtual bounding volume;

[0258] The high dynamic range image is used to render the target image.

[0259] The technical solution provided in this embodiment obtains the virtual bounding volume of the luminous object in the target image, and then determines the illumination information of multiple sampling points on the virtual bounding volume according to the luminous information of the luminous object, and then generates a high dynamic range image according to the illumination information of multiple sampling points on the virtual bounding volume for rendering the target image. The whole solution is simple and efficient to implement, and can obtain a high dynamic range image with high precision; and when using the high dynamic range image for real-time rendering, the calculation amount is small and the time consumption is short, which can greatly improve the rendering rate.

[0260] Furthermore, the acquisition module 801 is specifically used for:

[0261] Acquire the position information and size information of the light-emitting object;

[0262] Determining the center point of the virtual bounding volume according to the position information;

[0263] Determining contour data of the virtual bounding volume according to the size information;

[0264] The virtual bounding volume is obtained according to the center point and the contour data.

[0265] Furthermore, the above device further comprises:

[0266] The configuration module is used to configure diffuse reflection properties and pure white properties for the virtual bounding volume.

[0267] Furthermore, the virtual bounding volume is a virtual sphere; and the acquisition module 801 is further specifically used to: determine the maximum outer envelope size according to the size information; determine the sphere radius according to the maximum outer envelope size; wherein the sphere radius is greater than the maximum outer envelope size.

[0268] Furthermore, the determination module 802 is specifically used to: calculate the light intensity of the luminous light along a direction to a sampling point on the virtual bounding volume based on the luminous information of the luminous object; and use the direction and the light intensity as the lighting information corresponding to the sampling point.

[0269] It should be noted here that: the image processing device provided in this embodiment can achieve the above Figure 6 The technical solution described in the data processing method embodiment shown in the figure, the specific implementation principle of each module or unit can be found in the above Figure 6 The corresponding contents in the illustrated embodiment of the image processing method will not be repeated here.

[0270] Fig.15 FIG. 2 shows a structural block diagram of an image processing device provided by another embodiment of the present application. Fig.15 As shown, the image processing device includes: an acquisition module 901, a configuration module 902, a determination module 903 and a generation module 904. The acquisition module 901 is used to acquire a three-dimensional spatial image. The configuration module 902 is used to configure a virtual bounding volume for a luminous object in the three-dimensional spatial image. The determination module 903 is used to determine the illumination information corresponding to multiple sampling points on the virtual bounding volume according to the luminous information of the luminous object. The generation module 904 is used to generate a high dynamic range image according to the illumination information of multiple sampling points on the virtual bounding volume.

[0271] Furthermore, the configuration module 902 is specifically used to: obtain the position information and size information of the luminous object; determine the center point of the virtual enclosing volume according to the position information; determine the contour data of the virtual enclosing volume according to the size information; and obtain the virtual enclosing volume according to the center point and the contour data.

[0272] Furthermore, the above device may also include a response module and a processing module. The response module is used to respond to a user's rendering request for the spatial three-dimensional image and obtain the high dynamic range image; the processing module is used to render the spatial three-dimensional image using the high dynamic range image.

[0273] It should be noted here that: the image processing device provided in this embodiment can achieve the above Figure 7 The technical solution described in the data processing method embodiment shown in the figure, the specific implementation principle of each module or unit can be found in the above Figure 7 The corresponding contents in the illustrated embodiment of the image processing method will not be repeated here.

[0274] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a computer, the steps or functions of the image processing method provided in the above embodiments can be implemented.

[0275] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0276] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0277] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: include: Obtaining position information and size information of the luminous object in the target image; Determine the center point of the virtual bounding volume according to the position information; Determining contour data of the virtual bounding volume according to the size information; Obtaining the virtual bounding volume according to the center point and the contour data; Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; Determine the position of the virtual point light source corresponding to the luminous object; Determining light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points; The light parameter distribution information is used to render the target image.

2. The method according to claim 1, characterized in that Also includes: A diffuse reflection attribute and / or a pure white attribute is configured for the virtual bounding volume.

3. The method according to claim 1, characterized in that The virtual enclosing volume is a virtual sphere; as well as Determining the outline data of the virtual bounding volume according to the size information includes: Determining a maximum outer envelope size according to the size information; The spherical radius is determined according to the maximum outer envelope size; wherein the spherical radius is greater than the maximum outer envelope size.

4. The method according to claim 1, characterized in that: Determining the position of the virtual point light source corresponding to the luminous object includes: The position of the center point of the virtual bounding volume in the target image is used as the position of the virtual point light source.

5. The method according to any one of claims 1 to 4, characterized in that Determining illumination information corresponding to any sampling point on the virtual bounding volume according to the luminous information of the luminous object includes: Based on the luminous information of the luminous object, calculating the light intensity of the luminous light along a direction to a sampling point on the virtual bounding volume; The direction and the light intensity are used as the illumination information corresponding to the sampling point.

6. The method according to any one of claims 1 to 4, characterized in that: Also includes: Encoding the optical parameter distribution information to compress it to obtain compressed data; The compressed data is stored.

7. The method according to any one of claims 1 to 4, characterized in that: Also includes: In response to a rendering request for the target image, acquiring the light parameter distribution information; Calculating illumination information corresponding to a plurality of pixels in the target image using the light parameter distribution information; The target object is rendered according to the illumination information corresponding to the plurality of pixels in the target image.

8. The method according to claim 7, characterized in that Calculating illumination information corresponding to any pixel point in the target image using the light parameter distribution information includes: Determining direction and distance according to the position of the virtual point light source and the position of a pixel in the target image; Based on the light parameter distribution information, obtaining the light intensity corresponding to the direction; The light intensity corresponding to the pixel point is calculated according to the distance and the light intensity corresponding to the direction.

9. The method according to any one of claims 1 to 4, characterized in that Also includes: In response to a selection operation by the user, determining a rendering mode selected by the user; When the rendering mode selected by the user is the first mode, triggering the step of acquiring a virtual bounding volume of the luminous object in the target image, so as to execute subsequent steps until the light parameter distribution information is determined; When the rendering mode selected by the user is the second mode, a rendering algorithm corresponding to the second mode is called, and the rendering of the target image is completed by using the rendering algorithm.

10. An image processing method, characterized in that: include: In response to a rendering request for a target image, obtaining light parameter distribution information corresponding to the target image; Calculating illumination information corresponding to a plurality of pixels in the target image using the light parameter distribution information; Rendering the target image according to illumination information corresponding to a plurality of pixels in the target image; The light parameter distribution information includes the virtual point light source position corresponding to the luminous object in the target image, and the illumination information corresponding to multiple sampling points on the virtual bounding volume of the luminous object; The virtual bounding volume is obtained based on the center point of the virtual bounding volume determined based on the position information of the light-emitting object and the outline data of the virtual bounding volume determined based on the size information of the light-emitting object.

11. The method according to claim 10, characterized in that Calculating illumination information corresponding to any pixel point in the target image using the light parameter distribution information includes: Determining direction and distance according to the position of the virtual point light source and the position of a pixel in the target image; Based on the light parameter distribution information, obtaining the light intensity corresponding to the direction; The light intensity corresponding to the pixel point is calculated according to the distance and the light intensity corresponding to the direction.

12. The method according to claim 10 or 11, characterized in that: Also includes: Obtain a virtual bounding volume of a luminous object in a target image; Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; Determine the position of the virtual point light source corresponding to the luminous object; Light parameter distribution information is determined based on the virtual point light source position and the illumination information corresponding to the multiple sampling points.

13. The method according to claim 12, characterized in that Get the virtual bounding volume of the luminous object in the target image, including: Acquire the position information and size information of the light-emitting object; Determining the center point of the virtual bounding volume according to the position information; Determining contour data of the virtual bounding volume according to the size information; The virtual bounding volume is obtained according to the center point and the contour data.

14. The method according to claim 13, characterized in that Also includes: A diffuse reflection attribute and / or a pure white attribute is configured for the virtual bounding volume.

15. The method according to claim 13, characterized in that The virtual enclosing volume is a virtual sphere; as well as Determining the outline data of the virtual bounding volume according to the size information includes: Determining a maximum outer envelope size according to the size information; The spherical radius is determined according to the maximum outer envelope size; wherein the spherical radius is greater than the maximum outer envelope size.

16. An image processing method, characterized in that: include: Obtaining position information and size information of the luminous object in the target image; Determine the center point of the virtual bounding volume according to the position information; Determining contour data of the virtual bounding volume according to the size information; Obtaining the virtual bounding volume according to the center point and the contour data; Determining illumination information of a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; Generate a high dynamic range image according to illumination information of a plurality of sampling points on the virtual bounding volume; The high dynamic range image is used to render the target image.

17. The method according to claim 16, characterized in that Also includes: A diffuse reflection attribute and / or a pure white attribute is configured for the virtual bounding volume.

18. The method according to claim 16, characterized in that The virtual enclosing volume is a virtual sphere; as well as Determining the outline data of the virtual bounding volume according to the size information includes: Determining a maximum outer envelope size according to the size information; The spherical radius is determined according to the maximum outer envelope size; wherein the spherical radius is greater than the maximum outer envelope size.

19. The method according to any one of claims 16 to 18, characterized in that Also includes: In response to a rendering request for a target image triggered by a user, acquiring the high dynamic range image; The target image is rendered using the high dynamic range image.

20. An image processing device, characterized in that: include: A processor, configured to obtain position information and size information of a luminous object in a target image; and determine a center point of a virtual bounding volume according to the position information; Determine the outline data of the virtual bounding volume according to the size information; obtain the virtual bounding volume according to the center point and the outline data; Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; Determine a virtual point light source position corresponding to the luminous object; determine light parameter distribution information based on the virtual point light source position and the illumination information corresponding to the plurality of sampling points; A memory, used to store the optical parameter distribution information; The processor is further configured to obtain the light parameter distribution information in response to a rendering request for a target object, and perform rendering processing on the target image; A display is used to display the target image after rendering.

21. An image processing device, characterized in that: include: A processor, configured to obtain position information and size information of a luminous object in a target image; and determine a center point of a virtual bounding volume according to the position information; Determine the outline data of the virtual bounding volume according to the size information; obtain the virtual bounding volume according to the center point and the outline data; Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; generating a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume; A memory, used for storing the high dynamic range image; The processor is further configured to obtain the high dynamic range image in response to a rendering request for a target object, and perform rendering processing on the target image; A display is used to display the target image after rendering.

22. An image processing method, characterized in that: include: Get a three-dimensional image of the space; Acquire position information and size information of the luminous object in the three-dimensional spatial image; Determine the center point of the virtual bounding volume according to the position information; Determining contour data of the virtual bounding volume according to the size information; Obtaining the virtual bounding volume according to the center point and the contour data; Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; A high dynamic range image is generated according to the illumination information of a plurality of sampling points on the virtual bounding volume.

23. The method according to claim 22, characterized in that Also includes: In response to a user's rendering request for the spatial three-dimensional image, acquiring the high dynamic range image; The high dynamic range image is used to render the three-dimensional spatial image.

24. An image processing device, characterized in that: include: A processor, for obtaining a three-dimensional image of the space; Acquire the position information and size information of the luminous object in the three-dimensional spatial image; determine the center point of the virtual bounding volume according to the position information; Determine the outline data of the virtual bounding volume according to the size information; obtain the virtual bounding volume according to the center point and the outline data; Determining illumination information corresponding to a plurality of sampling points on the virtual bounding volume according to the luminous information of the luminous object; generating a high dynamic range image according to the illumination information of the plurality of sampling points on the virtual bounding volume; A memory, used for storing the high dynamic range image; The processor is further configured to obtain the high dynamic range image in response to a user's rendering request for the spatial three-dimensional image; and render the spatial three-dimensional image using the high dynamic range image; A display is used to display the three-dimensional image of the space after rendering.

Citation Information

Patent Citations

  • Illumination and shade drafting method based on transition light label drawing

    CN101271587A

  • Multiple-light source rendering method based on virtual spherical light sources

    CN104200512A

  • Light and shadow rendering method, device and electronic device for virtual object in panoramic video

    CN109064544A

  • Virtual light source processing method and device, medium and electronic equipment

    CN111260766A

  • Image processing apparatus and method

    KR1020120004227A