Image rendering method and device based on depth-of-field estimation, equipment and storage medium

Abstract

The invention relates to the technical field of image rendering, and particularly discloses an image rendering method and device based on depth-of-field estimation, equipment and a storage medium. Acquiring a scene file submitted by a user, and analyzing the scene file to obtain field depth information of a scene; performing depth-of-field estimation based on the depth-of-field information to obtain a depth-of-field estimation result; determining a depth-of-field calculation strategy of each pixel point in the image to be rendered according to the depth-of-field estimation result, and performing depth-of-field calculation based on the depth-of-field calculation strategy to obtain depth-of-field data of each pixel point; and performing image rendering on the to-be-rendered image based on the depth-of-field data to obtain a target rendered image. The depth-of-field estimation is carried out according to the depth-of-field information, and the depth-of-field calculation strategy of each pixel point is determined according to the depth-of-field estimation result, so that the calculation strategies are diversified, targeted depth-of-field calculation can be carried out on each pixel point, accurate calculation of all pixel points is not needed, the calculation amount is reduced, and the rendering efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image rendering technology, and in particular to an image rendering method, device, equipment and storage medium based on depth of field estimation. Background Art

[0002] Offline rendering is a commonly used technique in computer graphics to generate high-quality, realistic images and animations. Depth of field is one of the important parameters in offline rendering, simulating the effect of a clear foreground and blurred background when the front and rear cameras have different focal points. However, in traditional offline rendering methods, depth of field generation requires complex calculations for each pixel based on information such as camera parameters and object distance, resulting in long rendering time and huge amount of calculation, which in turn affects the efficiency of offline rendering. Therefore, how to improve rendering efficiency has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides an image rendering method, apparatus, device and storage medium based on depth of field estimation to improve rendering efficiency.

[0004] In a first aspect, the present application provides an image rendering method based on depth of field estimation, the method comprising:

[0005] Obtaining a scene file submitted by a user, analyzing the scene file, and obtaining depth of field information of the scene;

[0006] Performing depth of field estimation based on the depth of field information to obtain a depth of field estimation result;

[0007] Determine a depth of field calculation strategy for each pixel in the image to be rendered according to the depth of field estimation result, and perform depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel;

[0008] The image to be rendered is rendered based on the depth of field data to obtain a target rendered image.

[0009] In a second aspect, the present application further provides an image rendering device based on depth of field estimation, the device comprising:

[0010] A depth of field information acquisition module is used to acquire a scene file submitted by a user, analyze the scene file, and acquire the depth of field information of the scene;

[0011] A depth of field estimation result obtaining module, used to perform depth of field estimation based on the depth of field information to obtain a depth of field estimation result;

[0012] A depth of field data acquisition module is used to determine a depth of field calculation strategy for each pixel point in the image to be rendered according to the depth of field estimation result, and perform depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel point;

[0013] The rendering image obtaining module is used to perform image rendering on the image to be rendered based on the depth of field data to obtain a target rendering image.

[0014] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the image rendering method based on depth of field estimation as described above when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image rendering method based on depth of field estimation as described above.

[0016] The present application discloses an image rendering method, device, equipment and storage medium based on depth of field estimation, which obtains a scene file submitted by a user, analyzes the scene file, and obtains the depth of field information of the scene; performs depth of field estimation based on the depth of field information to obtain a depth of field estimation result; determines the depth of field calculation strategy for each pixel in the image to be rendered based on the depth of field estimation result, and performs depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel; performs image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image. The present application performs depth of field estimation based on depth of field information, and determines the depth of field calculation strategy for each pixel based on the depth of field estimation result, so that the calculation strategy is diversified, and targeted depth of field calculation can be performed on each pixel, without the need to perform precise calculations on all pixels, thereby reducing the amount of calculations and improving rendering efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of an image rendering method based on depth of field estimation provided by the first embodiment of the present application;

[0019] Figure 2 is a schematic flow chart of an image rendering method based on depth of field estimation provided by the second embodiment of the present application;

[0020] Figure 3 A schematic block diagram of an image rendering device based on depth of field estimation provided in an embodiment of the present application;

[0021] Figure 4 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0024] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] The embodiments of the present application provide an image rendering method, device, equipment and storage medium based on depth of field estimation. Among them, the image rendering method based on depth of field estimation can be applied to a server, and the depth of field is estimated according to the depth of field information, and the depth of field calculation strategy of each pixel is determined according to the depth of field estimation result, so that the calculation strategy is diversified, and the depth of field calculation can be performed on each pixel point in a targeted manner, without the need to perform precise calculations on all pixels, thereby reducing the amount of calculation and improving rendering efficiency. Among them, the server can be an independent server or a server cluster.

[0027] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0028] See also Figure 1 , Figure 1 This is a schematic flow chart of an image rendering method based on depth of field estimation provided by an embodiment of the present application. The image rendering method based on depth of field estimation can be applied to a server, and is used to perform depth of field estimation based on depth of field information, and determine the depth of field calculation strategy for each pixel point based on the depth of field estimation result, so that the calculation strategy is diversified, and targeted depth of field calculation can be performed on each pixel point, without the need to perform precise calculation on all pixel points, thereby reducing the amount of calculation and improving rendering efficiency.

[0029] like Figure 1 As shown, the image rendering method based on depth of field estimation specifically includes steps S101 to S104.

[0030] S101, obtaining a scene file submitted by a user, analyzing the scene file, and obtaining depth information of the scene;

[0031] In one embodiment, a scene file uploaded and submitted by a user is received through a user interface, including original scene images of at least two camera perspectives, camera parameters, and other data. The user interface may also receive rendering parameters configured by the user, including rendering resolution, time limit, and the like.

[0032] In one embodiment, when a scene file is acquired, the scene file is analyzed to extract the depth information of each element (such as an object, background, etc.) in the scene.

[0033] In one embodiment, the depth of field information may include information such as a depth map, a depth of field range, and a focus area.

[0034] Furthermore, the analyzing the scene file to obtain the depth of field information of the scene includes: based on a preset feature recognition model, identifying the scene in the scene file to obtain scene element feature information; based on a preset stereo matching algorithm, the scene element feature information and the scene file, performing stereo matching to obtain a disparity map; based on the disparity map and camera parameter information in the scene file, calculating to obtain the depth of field information.

[0035] In this embodiment, the preset feature recognition model can be a model based on deep learning or traditional image processing technology (such as a convolutional neural network model), which is used to extract feature information of elements in the scene.

[0036] Scene element feature information may include object edges, corners, textures, and other features.

[0037] In one embodiment, the feature recognition model is used to recognize the scene, identify and classify the elements and element feature information in the scene, and at the same time, the depth information of the elements is extracted in combination with the stereo matching method to obtain the depth information of the scene.

[0038] The stereo matching algorithm estimates the depth of objects in the scene by comparing scene images from two perspectives in the scene file. Specifically, feature points are extracted based on the feature information of scene elements in the scene images from two perspectives. These feature points can be corners, edges, or other significant visual features. The image is stereo rectified, and the rectified image can ensure that the search for corresponding points is limited to the same row, simplifying the matching process. For each pixel on the reference image (one of the scene images from two perspectives), the matching cost with the corresponding point on the target image (the other image in the scene images from two perspectives other than the reference image) is calculated. All matching costs are aggregated to obtain the cumulative cost of a point on the reference image at the disparity d. Based on the aggregated cumulative cost, different stereo matching algorithms (such as block matching, semi-global matching, or graph cut) are used to estimate the optimal disparity value for each pixel to obtain a disparity map.

[0039] In one embodiment, the disparity map records the position offset of each pixel in the scene relative to the reference point. The disparity map is a preliminary form of the depth map, in which the disparity value is inversely proportional to the depth value, that is, the larger the disparity, the smaller the depth.

[0040] In one embodiment, the depth of field information is calculated using the disparity map and the camera parameter information (such as focal length, aperture size, etc.) in the scene file. Specifically, after obtaining the disparity map, the depth information of each pixel can be calculated according to the principle of triangulation. The calculation formula is Z = (f*b) / d, where Z is the depth value, f is the focal length of the camera, b is the distance between the centers of the two cameras (baseline), and d is the disparity value. The calculated depth information is integrated to obtain the depth of field information of the entire scene, including the depth map, depth of field range, focus area, etc., to provide the necessary data for the next step of depth of field estimation and image rendering.

[0041] S102, performing depth of field estimation based on the depth of field information to obtain a depth of field estimation result;

[0042] Furthermore, the depth of field estimation is performed based on the depth of field information to obtain the depth of field estimation result, including: obtaining a preset depth of field estimation model; based on the depth of field estimation model, analyzing the depth of field information to obtain an estimated blur degree for each pixel, and based on the estimated blur degree, generating a depth map as the depth of field estimation result.

[0043] In one embodiment, the depth of field estimation model is a combination of a physical model and a deep learning model. The physical model predicts the depth of field based on optical principles, such as camera parameters, including focal length, aperture size, etc. The deep learning-based model predicts the depth of field by learning a large amount of image data.

[0044] The depth estimation model can adjust the blur level of the object according to the distance between the object and the camera, thereby estimating the real depth of field effect and obtaining the depth estimation result. Using the constructed depth estimation model, the depth value of each pixel in the scene can be estimated.

[0045] In a specific embodiment, depth of field estimation includes depth of field range estimation, blur degree estimation and depth of field map generation. Specifically, the depth of field estimation model is used to analyze the depth of field information, calculate the foreground depth of field (near depth of field) and background depth (far depth of field) of the scene, and obtain the depth of field range estimation result. Then, the depth of field estimation model is used to predict the blur degree of each pixel based on the depth of field information and camera parameters of each pixel. Finally, a depth of field map is generated based on the estimated blur degree of each pixel to obtain the depth of field estimation result. Each pixel value in the depth of field map represents the blur degree of the point.

[0046] Furthermore, obtaining a preset depth of field estimation model includes: constructing an initial model based on camera parameter information in the scene file; obtaining historical depth of field information and a historical depth of field map, and training the initial model based on the historical depth of field information and the historical depth of field map to obtain the depth of field estimation model.

[0047] In one embodiment, an initial model may be constructed based on camera parameters. The initial model may be a deep learning framework. The initial model is then trained using historical depth of field information and historical depth of field maps.

[0048] In one embodiment, during the model training process, historical depth of field information is used as training data, and historical depth of field maps are used as verification data. The model is trained through an optimization algorithm and a loss function (such as mean square error) to obtain a depth of field estimation model.

[0049] In one embodiment, during the training process, the estimated depth map obtained by the model based on the historical depth information is compared with the historical depth map to obtain the estimated accuracy of the model. When the estimated accuracy of the model reaches a preset accuracy threshold, the training is terminated and the model at this time is used as the depth estimation model. The accuracy threshold can be freely set by the user according to actual needs.

[0050] In the above embodiment, depth of field information is used to estimate depth of field and obtain a preliminary depth of field estimation result. Based on this estimation result, different depth of field calculation measurements can be formulated for each pixel point. There is no need to perform accurate depth of field calculation on all pixels, thereby improving rendering efficiency while ensuring the quality of the rendering results.

[0051] S103, determining a depth of field calculation strategy for each pixel in the image to be rendered according to the depth of field estimation result, and performing depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel;

[0052] In one embodiment, according to the depth estimation result, key pixels and common pixels in the image are identified, depth calculation strategies of the key pixels and common pixels are obtained respectively, and the final depth calculation strategy is generated by combining the depth calculation strategies of the key pixels and common pixels.

[0053] According to the depth of field calculation strategy corresponding to each pixel point, the depth of field calculation is performed on each pixel point to obtain the depth of field data corresponding to each pixel point.

[0054] S104: Perform image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image.

[0055] Furthermore, the performing image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image includes: dividing the image to be rendered according to the depth of field calculation strategy corresponding to each pixel point, and classifying the depth of field data to obtain at least one area to be rendered and a depth of field data set and resource requirements corresponding to each area to be rendered; obtaining the load condition of at least one preset processing node, and allocating a processing node to each area to be rendered based on the load condition and the resource requirements, so as to perform image rendering on each area to be rendered based on each processing node and the depth of field data set to obtain the target rendered image.

[0056] In one embodiment, according to the depth calculation strategy corresponding to each pixel point, adjacent pixels with the same depth calculation strategy are divided into the same area to be rendered, and the depth data corresponding to the pixels in the area are divided into the same depth data set.

[0057] In one embodiment, the resource requirements of each area to be rendered are obtained. Specifically, the scene geometric complexity and material complexity of the area to be rendered are analyzed, and a table is looked up based on the geometric complexity and material complexity to determine the resource requirements required for rendering each area to be rendered. The resource requirements corresponding to each geometric complexity and material complexity can be determined based on historical complexity and historical resource requirements.

[0058] In one embodiment, the current load of each processing node is collected, including key performance indicators such as CPU and GPU usage, memory usage, storage I / O performance, etc. Specifically, it can be obtained in real time through monitoring tools to ensure that rendering tasks can be reasonably allocated to the node with the best performance.

[0059] Dynamically allocate processing nodes to each area to be rendered based on the load of the processing nodes and the resource requirements of the area to be rendered. You can use an optimization algorithm for task scheduling to ensure load balancing and avoid overloading some nodes while leaving other nodes idle.

[0060] When a processing node receives an assigned area to be rendered, the processing node will start to execute the assigned rendering task and use the corresponding depth of field data set to render images of each area to be rendered. Specifically, based on the depth of field data set of the area to be rendered and the semantic segmentation map of the object in the area to be rendered, a depth of field algorithm is applied to generate a depth of field result image of the area to be rendered, and the generated depth of field result image is synthesized with the color image of the area to be rendered to generate a depth of field rendering image of the area to be rendered.

[0061] After all processing nodes complete their assigned rendering tasks, the system will collect the depth of field rendering images of all nodes and synthesize them to form the final target rendering image. Multiple processing nodes can simultaneously process the rendering tasks of their assigned areas to be rendered, achieving parallel processing and improving rendering efficiency.

[0062] In one embodiment, post-processing operations are performed on the generated rendered image, including image enhancement, image compression, image filtering, etc., to optimize rendering quality and efficiency.

[0063] The above-mentioned embodiments provide an image rendering method, apparatus, device and storage medium based on depth of field estimation, which obtains a scene file submitted by a user, analyzes the scene file, and obtains the depth of field information of the scene; performs depth of field estimation based on the depth of field information to obtain a depth of field estimation result; determines the depth of field calculation strategy for each pixel in the image to be rendered based on the depth of field estimation result, and performs depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel; performs image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image. The present application performs depth of field estimation based on depth of field information, and determines the depth of field calculation strategy for each pixel based on the depth of field estimation result, so that the calculation strategy is diversified, and targeted depth of field calculation can be performed on each pixel, without the need to perform precise calculations on all pixels, thereby reducing the amount of calculations and improving rendering efficiency.

[0064] See also Figure 2 , Figure 2 This is a schematic flow chart of an image rendering method based on depth of field estimation provided by an embodiment of the present application. The image rendering method based on depth of field estimation can be applied to a server to determine the depth of field calculation strategy for each pixel point according to the depth of field estimation result, so that the calculation strategy is diversified, and targeted depth of field calculation can be performed on each pixel point without the need to perform precise calculation on all pixel points, thereby reducing the amount of calculation and improving rendering efficiency.

[0065] like Figure 2 As shown, the image rendering method based on depth of field estimation specifically includes steps S201 to S204.

[0066] S201, obtaining an estimated blur degree of each pixel in the image to be rendered based on the depth of field estimation result;

[0067] S202, determining key pixels and common pixels in each pixel of the image to be rendered based on the estimated blur degree;

[0068] S203, respectively obtaining a first depth of field calculation strategy and a second depth of field calculation strategy corresponding to the key pixel point and the ordinary pixel point;

[0069] S204: Generate the depth of field calculation strategy based on the first depth of field calculation strategy and the second depth of field calculation strategy.

[0070] In one embodiment, the depth of field estimation result includes the blur degree of each pixel and the estimated depth of field effect. Therefore, the blur degree of each pixel can be directly obtained from the depth of field estimation result.

[0071] In one embodiment, pixels are classified into key pixels and common pixels according to the blur degree of the pixels and a preset blur degree threshold. Specifically, pixels whose blur degree is greater than or equal to the blur degree threshold are regarded as common pixels, and pixels whose blur degree is less than the blur degree threshold are regarded as key pixels. Key pixels require less blur processing to maintain important features of the image, while common pixels require more blur processing according to their blur degree.

[0072] In another embodiment, the blur degree of the pixels may be combined with their positions in the image to determine key pixels and common pixels. For example, pixels at the edge, corner, or specific semantic area that reach a preset blur degree threshold may be used as key pixels, and the remaining pixels may be used as common pixels.

[0073] In one embodiment, different depth of field calculation strategies are formulated for key pixels and ordinary pixels. For key pixels, a higher depth of field calculation accuracy is used to ensure rendering quality; while for ordinary pixels, a lower depth of field calculation accuracy is used to reduce the amount of calculation and improve rendering efficiency.

[0074] The first depth of field calculation strategy corresponding to the key pixel point and the second depth of field calculation strategy corresponding to the common pixel point are combined into a unified depth of field calculation strategy.

[0075] Furthermore, the first depth of field calculation strategy and the second depth of field calculation strategy corresponding to the key pixel points and the ordinary pixel points are obtained respectively, including: classifying the key pixel points according to a preset first blur degree range set and the blur degree of each of the key pixel points, and obtaining the depth of field calculation parameters corresponding to each first blur degree range as the depth of field calculation parameters of the key pixel points of each category, to generate the first depth of field calculation strategy; classifying the ordinary pixel points according to a preset second blur degree range set and the blur degree of each of the ordinary pixel points, and obtaining the depth of field calculation parameters corresponding to each second blur degree range as the depth of field calculation parameters of the ordinary pixel points of each category, to generate the second depth of field calculation strategy.

[0076] In one embodiment, the preset blur level range represents different levels of blur levels, such as slight blur, medium blur and severe blur. When the blur levels of pixels belong to the same blur level range, these pixels are classified into the same category and use the same depth of field calculation parameters.

[0077] In one embodiment, for each first blur level range, corresponding depth of field calculation parameters are set. The depth of field calculation parameters may include the type and strength of the blur filter and the depth of field calculation accuracy, etc. The depth of field calculation accuracy may be freely set by the user according to actual needs.

[0078] A first depth of field calculation strategy is generated according to the blur degree of the key pixel and the corresponding depth of field calculation parameter. The first depth of field calculation strategy performs accurate depth of field calculation on the key pixel so that the key pixel remains relatively clear in the final image.

[0079] In one embodiment, for each second blur level range, depth of field calculation parameters applicable to common pixels are set.

[0080] The second depth of field calculation strategy is generated according to the blur degree of the common pixel and the corresponding depth of field calculation parameters. The second depth of field calculation strategy performs common depth of field calculation and blur processing on the common pixel, or directly uses the estimated value corresponding to the pixel in the depth of field estimation result when the depth of field calculation accuracy in the depth of field calculation strategy is less than the preset accuracy threshold, so as to reduce the amount of calculation and improve rendering efficiency.

[0081] In one embodiment, the first depth of field calculation strategy and the second depth of field calculation strategy are combined to form a comprehensive depth of field calculation strategy. According to the generated depth of field calculation strategy, a corresponding blurring process is applied to each pixel in the image.

[0082] In the above embodiment, the depth of field calculation strategy for each pixel point can be determined according to the depth of field estimation result, so that the calculation strategy is diversified and targeted depth of field calculation can be performed on each pixel point without the need to accurately calculate all pixels, thereby reducing the amount of calculation and improving rendering efficiency.

[0083] See also Figure 3 , Figure 3 The embodiment of the present application provides a schematic block diagram of an image rendering device based on depth of field estimation, wherein the image rendering device based on depth of field estimation is used to execute the aforementioned image rendering method based on depth of field estimation. The image rendering device based on depth of field estimation can be configured on a server.

[0084] like Figure 3 As shown, the image rendering device 300 based on depth of field estimation includes:

[0085] The depth information acquisition module 301 is used to acquire a scene file submitted by a user, analyze the scene file, and acquire the depth information of the scene;

[0086] A depth of field estimation result obtaining module 302 is used to perform depth of field estimation based on the depth of field information to obtain a depth of field estimation result;

[0087] A depth of field data acquisition module 303 is used to determine a depth of field calculation strategy for each pixel in the image to be rendered according to the depth of field estimation result, and perform depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel;

[0088] The rendering image obtaining module 304 is used to perform image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image.

[0089] Furthermore, the depth of field data acquisition module includes:

[0090] A distance acquisition unit, used for acquiring an estimated blur degree of each pixel in the image to be rendered based on the depth of field estimation result;

[0091] A pixel point division unit, used for determining key pixel points and common pixel points in each pixel point of the image to be rendered based on the estimated blur degree;

[0092] A strategy acquisition unit, used to respectively acquire a first depth of field calculation strategy and a second depth of field calculation strategy corresponding to the key pixel point and the ordinary pixel point;

[0093] A depth of field calculation strategy generating unit is used to generate the depth of field calculation strategy based on the first depth of field calculation strategy and the second depth of field calculation strategy.

[0094] Furthermore, the strategy acquisition unit includes:

[0095] a first depth of field calculation measurement generation subunit, configured to classify the key pixels according to a preset first blur degree range set and a blur degree of each key pixel, and obtain depth of field calculation parameters corresponding to each first blur degree range as depth of field calculation parameters of key pixels of each category, and generate the first depth of field calculation strategy;

[0096] The second depth of field calculation measurement generation subunit is used to classify the ordinary pixels according to a preset second blur degree range set and the blur degree of each ordinary pixel, and obtain the depth of field calculation parameters corresponding to each second blur degree range as the depth of field calculation parameters of the ordinary pixels of each category, and generate the second depth of field calculation strategy.

[0097] Furthermore, the depth of field estimation result obtaining module includes:

[0098] A depth of field estimation model acquisition unit, used to acquire a preset depth of field estimation model;

[0099] The depth of field estimation result obtaining unit is used to analyze the depth of field information based on the depth of field estimation model, obtain the estimated blur degree of each pixel point, and generate a depth map based on the estimated blur degree as the depth of field estimation result.

[0100] Furthermore, the depth of field estimation model acquisition unit includes:

[0101] An initial model building subunit, used to build an initial model based on the camera parameter information in the scene file;

[0102] The depth estimation model acquisition subunit is used to obtain historical depth information and historical depth maps, and train the initial model based on the historical depth information and the historical depth map to obtain the depth estimation model.

[0103] Furthermore, the depth of field information obtaining module includes:

[0104] A feature information obtaining unit, used to identify the scene in the scene file based on a preset feature recognition model to obtain scene element feature information;

[0105] A disparity map obtaining unit, configured to perform stereo matching based on a preset stereo matching algorithm, the scene element feature information and the scene file to obtain a disparity map;

[0106] A depth of field information calculation unit is used to calculate and obtain the depth of field information based on the disparity map and the camera parameter information in the scene file.

[0107] Furthermore, performing image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image includes:

[0108] An image region division unit is used to divide the image to be rendered according to the depth of field calculation strategy corresponding to each pixel point, and classify the depth of field data to obtain at least one region to be rendered and a depth of field data set and resource requirements corresponding to each of the regions to be rendered;

[0109] An image rendering unit is used to obtain the load condition of at least one preset processing node, and based on the load condition and the resource requirement, allocate a processing node to each of the areas to be rendered, so as to perform image rendering on each of the areas to be rendered based on each of the processing nodes and the depth of field data set, to obtain the target rendered image.

[0110] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0111] The above-mentioned device can be implemented in the form of a computer program. Figure 4 Runs on the computer device shown.

[0112] See also Figure 4 , Figure 4 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0113] See also Figure 4 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0114] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any image rendering method based on depth of field estimation.

[0115] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0116] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any image rendering method based on depth of field estimation.

[0117] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0119] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0120] Obtaining a scene file submitted by a user, analyzing the scene file, and obtaining depth of field information of the scene;

[0121] Performing depth of field estimation based on the depth of field information to obtain a depth of field estimation result;

[0122] Determine a depth of field calculation strategy for each pixel in the image to be rendered according to the depth of field estimation result, and perform depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel;

[0123] The image to be rendered is rendered based on the depth of field data to obtain a target rendered image.

[0124] In one embodiment, when determining the depth of field calculation strategy for each pixel point in the image to be rendered according to the depth of field estimation result, the processor is used to implement:

[0125] Based on the depth of field estimation result, obtaining an estimated blur degree of each pixel in the image to be rendered;

[0126] Based on the estimated blur degree, determining key pixels and common pixels in each pixel of the image to be rendered;

[0127] Respectively obtain a first depth of field calculation strategy and a second depth of field calculation strategy corresponding to the key pixel point and the ordinary pixel point;

[0128] The depth of field calculation strategy is generated based on the first depth of field calculation strategy and the second depth of field calculation strategy.

[0129] In one embodiment, when implementing the first depth of field calculation strategy and the second depth of field calculation strategy corresponding to the key pixel point and the ordinary pixel point respectively, the processor is used to implement:

[0130] According to a preset first blur degree range set and a blur degree of each of the key pixel points, the key pixel points are classified, and depth of field calculation parameters corresponding to each first blur degree range are obtained as depth of field calculation parameters of key pixel points of each category, to generate the first depth of field calculation strategy;

[0131] According to a preset second blur degree range set and the blur degree of each of the ordinary pixels, the ordinary pixels are classified, and the depth of field calculation parameters corresponding to each second blur degree range are obtained as the depth of field calculation parameters of the ordinary pixels of each category to generate the second depth of field calculation strategy.

[0132] In one embodiment, when the processor estimates the depth of field based on the depth of field information and obtains the depth of field estimation result, it is used to implement:

[0133] Obtain a preset depth of field estimation model;

[0134] Based on the depth of field estimation model, the depth of field information is analyzed to obtain an estimated blur degree of each pixel, and based on the estimated blur degree, a depth map is generated as the depth of field estimation result.

[0135] In one embodiment, when the processor implements obtaining a preset depth of field estimation model, it is used to implement:

[0136] Building an initial model based on the camera parameter information in the scene file;

[0137] Historical depth of field information and a historical depth of field map are obtained, and the initial model is trained based on the historical depth of field information and the historical depth of field map to obtain the depth of field estimation model.

[0138] In one embodiment, when the processor analyzes the scene file to obtain the depth information of the scene, it is used to implement:

[0139] Based on a preset feature recognition model, the scene in the scene file is recognized to obtain scene element feature information;

[0140] Based on a preset stereo matching algorithm, the scene element feature information and the scene file, stereo matching is performed to obtain a disparity map;

[0141] The depth of field information is obtained by calculation based on the disparity map and the camera parameter information in the scene file.

[0142] In one embodiment, when the processor performs image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image, the processor is used to implement:

[0143] According to the depth of field calculation strategy corresponding to each pixel point, the image to be rendered is divided, and the depth of field data is classified to obtain at least one area to be rendered and a depth of field data set and resource requirements corresponding to each area to be rendered;

[0144] Obtain the load condition of at least one preset processing node, and allocate a processing node to each of the areas to be rendered based on the load condition and the resource requirements, so as to perform image rendering on each of the areas to be rendered based on each of the processing nodes and the depth of field data set to obtain the target rendered image.

[0145] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any image rendering method based on depth of field estimation provided in an embodiment of the present application.

[0146] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.

[0147] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An image rendering method based on depth of field estimation, characterized in that: include: Obtaining a scene file submitted by a user, analyzing the scene file, and obtaining depth of field information of the scene; Performing depth of field estimation based on the depth of field information to obtain a depth of field estimation result; Determine a depth of field calculation strategy for each pixel in the image to be rendered according to the depth of field estimation result, and perform depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel; The image to be rendered is rendered based on the depth of field data to obtain a target rendered image.

2. The image rendering method based on depth of field estimation according to claim 1, characterized in that: Determining the depth of field calculation strategy for each pixel in the image to be rendered according to the depth of field estimation result includes: Based on the depth of field estimation result, obtaining an estimated blur degree of each pixel in the image to be rendered; Based on the estimated blur degree, determining key pixels and common pixels in each pixel of the image to be rendered; Respectively obtain a first depth of field calculation strategy and a second depth of field calculation strategy corresponding to the key pixel point and the ordinary pixel point; The depth of field calculation strategy is generated based on the first depth of field calculation strategy and the second depth of field calculation strategy.

3. The image rendering method based on depth of field estimation according to claim 2, characterized in that: The first depth of field calculation strategy and the second depth of field calculation strategy corresponding to the key pixel point and the ordinary pixel point are respectively obtained, including: According to a preset first blur degree range set and a blur degree of each of the key pixel points, the key pixel points are classified, and depth of field calculation parameters corresponding to each first blur degree range are obtained as depth of field calculation parameters of key pixel points of each category, to generate the first depth of field calculation strategy; According to a preset second blur degree range set and the blur degree of each of the ordinary pixels, the ordinary pixels are classified, and the depth of field calculation parameters corresponding to each second blur degree range are obtained as the depth of field calculation parameters of the ordinary pixels of each category to generate the second depth of field calculation strategy.

4. The image rendering method based on depth of field estimation according to claim 1, characterized in that: The performing depth of field estimation based on the depth of field information to obtain a depth of field estimation result includes: Obtain a preset depth of field estimation model; Based on the depth of field estimation model, the depth of field information is analyzed to obtain an estimated blur degree of each pixel, and based on the estimated blur degree, a depth map is generated as the depth of field estimation result.

5. The image rendering method based on depth of field estimation according to claim 4, characterized in that: The step of obtaining a preset depth of field estimation model includes: Building an initial model based on the camera parameter information in the scene file; Historical depth of field information and a historical depth of field map are obtained, and the initial model is trained based on the historical depth of field information and the historical depth of field map to obtain the depth of field estimation model.

6. The image rendering method based on depth of field estimation according to claim 1, characterized in that: The step of analyzing the scene file to obtain depth information of the scene includes: Based on a preset feature recognition model, the scene in the scene file is recognized to obtain scene element feature information; Based on a preset stereo matching algorithm, the scene element feature information and the scene file, stereo matching is performed to obtain a disparity map; The depth of field information is obtained by calculation based on the disparity map and the camera parameter information in the scene file.

7. The image rendering method based on depth of field estimation according to any one of claims 1 to 6, characterized in that: The performing image rendering on the image to be rendered based on the depth of field data to obtain a target rendered image includes: According to the depth of field calculation strategy corresponding to each pixel point, the image to be rendered is divided, and the depth of field data is classified to obtain at least one area to be rendered and a depth of field data set and resource requirements corresponding to each area to be rendered; Obtain the load condition of at least one preset processing node, and allocate a processing node to each of the areas to be rendered based on the load condition and the resource requirements, so as to perform image rendering on each of the areas to be rendered based on each of the processing nodes and the depth of field data set to obtain the target rendered image.

8. An image rendering device based on depth of field estimation, characterized in that: include: A depth of field information acquisition module is used to acquire a scene file submitted by a user, analyze the scene file, and acquire the depth of field information of the scene; A depth of field estimation result obtaining module, used to perform depth of field estimation based on the depth of field information to obtain a depth of field estimation result; A depth of field data acquisition module is used to determine a depth of field calculation strategy for each pixel point in the image to be rendered according to the depth of field estimation result, and perform depth of field calculation based on the depth of field calculation strategy to obtain depth of field data for each pixel point; The rendering image obtaining module is used to perform image rendering on the image to be rendered based on the depth of field data to obtain a target rendering image.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the image rendering method based on depth of field estimation according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image rendering method based on depth of field estimation according to any one of claims 1 to 7.

Images