Vehicle 3D model lighting rendering method, system, electronic device and medium

Abstract

The application provides a vehicle 3D model light rendering method, system, electronic equipment and medium, and belongs to the technical field of vehicles. The method comprises the following steps: acquiring an aerial panoramic image collected by a surround-view camera; processing the aerial panoramic image to obtain a binary mask for marking a shadow area of a vehicle; performing main light source parameter estimation based on a vehicle 3D model, the aerial panoramic image and the binary mask to obtain a main light source parameter set; determining an ambient light parameter based on the aerial panoramic image and the binary mask; and performing 3D vehicle model rendering based on the main light source parameter set, the ambient light parameter, the vehicle 3D model and the aerial panoramic image. The application uses the shadow projected by the vehicle itself as an information source for light perception, realizes closed-loop light perception and can adapt to complex scenes. Through accurate light source direction calculation and full parameter estimation, the application provides a mathematical basis for generating correct shadow projection and mirror reflection effects, and significantly improves the realism of rendering.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a method, system, electronic device, and medium for lighting and rendering a 3D vehicle model. Background Technology

[0002] With the increasing popularity of automotive intelligence and the "third space" concept, 3D human-machine interfaces (HMIs) have become an important direction for enhancing user immersion and digital luxury. In in-vehicle surround view (AVM) systems, using 3D vehicle models to replace traditional 2D vehicle icons can provide users with a more intuitive and realistic visual experience. However, existing technologies in this field mainly suffer from problems such as static lighting models and a lack of realistic reflection of local environments. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art, and proposes a method, system, electronic device and medium for lighting rendering of vehicle 3D models.

[0004] In a first aspect, embodiments of the present invention provide a method for lighting and rendering a 3D vehicle model, comprising:

[0005] Acquire bird's-eye view panoramic images captured by surround-view cameras;

[0006] The bird's-eye view panoramic image is processed to obtain a binary mask that identifies the shadow areas of the vehicles;

[0007] Based on the vehicle 3D model, the main light source parameters are estimated using the bird's-eye view panoramic image and the binarized mask to obtain the main light source parameter set;

[0008] Ambient light parameters are determined based on the bird's-eye view panoramic image and the binarized mask;

[0009] 3D vehicle model rendering is performed based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

[0010] In some embodiments, acquiring the bird's-eye view panoramic image captured by the surround-view camera includes:

[0011] The original video stream is acquired based on the vehicle's fisheye camera, and the original image is extracted from the original video stream.

[0012] The original image is subjected to distortion correction to obtain the corrected image;

[0013] The corrected image is then transformed by a viewpoint and projected onto a unified ground plane coordinate system to obtain the transformed image.

[0014] The transformed images are stitched together to obtain a bird's-eye view panoramic image.

[0015] In some embodiments, the step of stitching images together based on the transformed images to obtain a bird's-eye view panoramic image includes:

[0016] The transformed image is stitched together to generate an initial panoramic image centered on the vehicle.

[0017] The initial panoramic image is processed using an image enhancement algorithm to obtain a bird's-eye view panoramic image.

[0018] In some embodiments, processing the bird's-eye view panoramic image to obtain a binary mask identifying the vehicle shadow region includes:

[0019] The bird's-eye view panoramic image is converted to a different color space to obtain the converted image.

[0020] The converted image is subjected to adaptive threshold segmentation on the brightness channel to obtain an initial binary image;

[0021] After performing geometric constraint post-processing on the initial binary image, a binary mask identifying the shadow region of the vehicle is obtained.

[0022] In some embodiments, the main light source parameter estimation based on the vehicle 3D model is performed according to the bird's-eye view panoramic image and the binarized mask to obtain a main light source parameter set, including:

[0023] Obtain a 3D model of the vehicle;

[0024] Based on the vehicle 3D model, bird's-eye view panoramic image and binarized mask, the light source direction is estimated to obtain the main light source direction vector;

[0025] Based on the vehicle 3D model, bird's-eye view panoramic image, binarized mask and main light source direction vector, the light source color and intensity are estimated to obtain the light source color and intensity parameters.

[0026] A main light source parameter set is generated based on the main light source direction vector and the light source color and intensity parameters.

[0027] In some embodiments, the step of estimating the light source direction based on the vehicle 3D model, the bird's-eye view panoramic image, and the binarized mask to obtain the main light source direction vector includes:

[0028] Select key points based on the vehicle 3D model;

[0029] Based on the bird's-eye view panoramic image and the binarized mask, the key points are matched by projection points to obtain paired points;

[0030] The initial three-dimensional direction vector of the light ray is obtained based on the geometric projection relationship and the pairing points;

[0031] The initial three-dimensional direction vector is optimized using a random sampling consensus algorithm to obtain the main light source direction vector.

[0032] In some embodiments, determining the ambient light parameters based on the bird's-eye view panoramic image and the binarized mask includes:

[0033] The shadow region pixels are obtained based on the binarized mask and the bird's-eye view panoramic image;

[0034] Determine the average RGB value of the pixels in the shaded area;

[0035] The ambient light color and ambient light intensity are obtained based on the average RGB values.

[0036] The ambient light color and ambient light intensity are used as ambient light parameters.

[0037] Secondly, embodiments of the present invention provide a vehicle 3D model lighting rendering system, comprising:

[0038] The image acquisition module is used to acquire bird's-eye view panoramic images captured by the surround-view camera;

[0039] The image processing module is used to process the bird's-eye view panoramic image to obtain a binary mask that identifies the shadow areas of the vehicles.

[0040] The main light source parameter estimation module is used to estimate the main light source parameters based on the vehicle 3D model according to the bird's-eye panoramic image and the binarized mask, and obtain the main light source parameter set.

[0041] An ambient light parameter determination module is used to determine ambient light parameters based on the bird's-eye view panoramic image and a binarized mask.

[0042] The rendering and compositing module is used to render 3D vehicle models based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

[0043] Thirdly, embodiments of the present invention provide an electronic device, including:

[0044] One or more processors;

[0045] Memory, used to store one or more programs;

[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described above.

[0047] Fourthly, embodiments of the present invention provide a computer-readable medium on which a computer program is stored, the computer program being executed by a processor to implement the steps of any of the methods described above.

[0048] The present invention provides a vehicle 3D model lighting rendering method, comprising: acquiring a bird's-eye view panoramic image captured by a surround-view camera; processing the bird's-eye view panoramic image to obtain a binary mask identifying the vehicle's shadow area; estimating the main light source parameters based on the vehicle 3D model according to the bird's-eye view panoramic image and the binary mask to obtain a main light source parameter set; determining the ambient light parameters according to the bird's-eye view panoramic image and the binary mask; and rendering the 3D vehicle model according to the main light source parameter set, the ambient light parameters, the vehicle 3D model, and the bird's-eye view panoramic image. This invention uses the shadows cast by the vehicle itself as the information source for lighting perception, achieving a fundamental shift from "macroscopic prediction" to "microscopic real-time perception," solving the fundamental defect of existing technologies that cannot adapt to local realistic lighting environments. Through precise calculation of light source direction and full parameter estimation, it provides a mathematical basis for generating correct shadow projection and specular reflection effects, significantly improving the realism of the rendering and producing more lifelike rendering effects. Attached Figure Description

[0049] Figure 1 A schematic flowchart of a vehicle 3D model lighting rendering method provided in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram illustrating the optional specific implementation process of the technical solutions involved in the embodiments of the present invention;

[0051] Figure 3 This is a schematic diagram of an optional specific implementation process involving the main light source parameter estimation step in an embodiment of the present invention;

[0052] Figure 4 A structural block diagram of a vehicle 3D model lighting rendering system provided in an embodiment of the present invention;

[0053] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0055] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0056] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0057] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0058] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0059] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0060] The key terms involved in this invention are defined as follows:

[0061] AVM (Around View Monitoring): A panoramic surround view monitoring system that provides a 360-degree view of the vehicle's surroundings through multiple cameras;

[0062] BEV (Bird's Eye View): A bird's-eye view, a perspective taken from above.

[0063] PBR (Physically Based Rendering): A more realistic 3D rendering technology;

[0064] RANSAC (Random Sample Consensus): A random sampling consensus algorithm used to estimate mathematical model parameters from data containing outliers;

[0065] HSV: Hue, Saturation, Value color space;

[0066] Phong lighting model: A classic local lighting model, consisting of three components: ambient light, diffuse reflection, and specular reflection;

[0067] Otsu algorithm: an image segmentation method that automatically selects a threshold.

[0068] In related technologies, lighting adjustment schemes based on image brightness features analyze a panoramic image, calculate the average brightness of the image as an ambient light parameter, identify the brightest area in the image as the "light source position," and adjust the lighting effect of the 3D car model based on this simple brightness information before rendering the adjusted car model onto the panoramic image. This scheme attempts to dynamically adjust lighting through image analysis, which is an improvement over completely static schemes. However, it severely lacks light source positioning and direction calculation, simply equating the brightest area with the main light source, which is prone to errors in complex scenes with specular reflections and strongly reflective objects. More importantly, it completely ignores how to calculate the three-dimensional direction vector of the light source, which is a fundamental prerequisite for realistic rendering (especially generating correct shadows and highlights). The lack of direction information leads to the inability to generate correct shadow casting and specular reflection effects. The lighting model of this scheme is too simplified, using only average brightness and a single highlight point to define the lighting of the entire scene, failing to distinguish the different physical characteristics of the main light source and ambient light, and neglecting key physical parameters such as diffuse reflection and specular reflection. This simplified model cannot support high-fidelity rendering, resulting in poor integration of the car model with the environment. The solution is poorly adaptable to complex lighting conditions (such as multiple light sources, strong reflections, and shadow occlusion), easily leading to incorrect lighting estimates, affecting the stability of rendering quality, and lacking robust design.

[0069] In another related technology, a lighting prediction scheme based on multi-sensor information fusion obtains the solar incidence angle information of the current location from the network through a TBOX (On-Board Observatory Box), obtains the vehicle's current heading angle using EPS (Electric Power Steering), and combines it with image information captured by cameras. The scheme calculates solar illumination parameters through multi-sensor data fusion and adjusts the lighting rendering of the 3D car model based on the calculation results. This scheme attempts to improve the accuracy of lighting estimation through multi-source information fusion. However, this scheme cannot perceive the local real lighting environment, which is a fundamental and systemic flaw of this type of scheme. It relies on macroscopic astronomical data obtained from the network by the TBOX to predict lighting, rather than directly perceiving the real microscopic environment in which the vehicle is located. When the vehicle enters a tunnel, underground parking garage, or the shaded area of ​​a building on a sunny day, the scheme completely fails, incorrectly continuing to use a full sunlight model for rendering, causing the virtual car model to appear to have its own glow in the shadows, severely deviating from the real environment. This scheme is essentially an open-loop prediction system; it predicts without a closed loop rather than perceives, and it cannot compare and correct the rendering results with the real scene. The lack of a feedback mechanism prevents the system from self-correcting and makes it difficult to adapt to complex and ever-changing real-world lighting environments. This solution cannot perceive complex light and shadow changes caused by local obstructions (such as buildings, trees, and bridges), and therefore performs poorly in complex scenarios such as urban environments and mountain roads. Furthermore, the core function of this solution relies on a network to acquire astronomical data; its dependence on an external network connection means that system functionality will be affected by poor network signals or network outages, reducing the system's reliability and usability.

[0070] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides a method for lighting rendering of a vehicle 3D model. Figure 1 This is a flowchart illustrating a method for lighting and rendering a 3D vehicle model, as provided in an embodiment of the present invention.

[0071] As an embodiment of the present invention, such as Figure 1 As shown, the vehicle 3D model lighting rendering method includes:

[0072] Step S1: Acquire bird's-eye view panoramic images captured by the surround-view camera;

[0073] Step S2: Process the bird's-eye view panoramic image to obtain a binary mask that identifies the shadow areas of the vehicles;

[0074] Step S3: Based on the vehicle 3D model, estimate the main light source parameters according to the bird's-eye view panoramic image and the binarized mask to obtain the main light source parameter set;

[0075] Step S4: Determine the ambient light parameters based on the bird's-eye view panoramic image and the binarized mask;

[0076] Step S5: Render the 3D car model based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

[0077] It should be noted that the execution subject in this embodiment can be an electronic device, which can be a computer device with data processing function, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, the execution subject is a computer device as an example for explanation.

[0078] Specifically, this embodiment involves technical fields such as computer vision, image processing, and 3D rendering. The method described in this embodiment analyzes 360-degree panoramic images collected in real time by an onboard surround-view camera, automatically detects the shadow information projected by the vehicle itself, uses a known 3D vehicle model as a "calibrator," calculates the precise parameters of the main light source in the real environment through geometric inverse kinematics, and estimates the ambient light parameters by combining image analysis, ultimately achieving dynamic, high-fidelity rendering of the 3D vehicle model. The following describes the specific steps.

[0079] In some embodiments, acquiring a bird's-eye panoramic image captured by a surround-view camera includes: acquiring an original video stream based on a vehicle fisheye camera; extracting an original image from the original video stream; performing distortion correction on the original image to obtain a corrected image; performing a perspective transformation on the corrected image to project it onto a unified ground plane coordinate system to obtain a transformed image; and performing image stitching based on the transformed image to obtain a bird's-eye panoramic image.

[0080] In some embodiments, image stitching is performed on the transformed image to obtain a bird's-eye view panoramic image, including: performing image stitching on the transformed image to generate an initial panoramic image centered on the vehicle; and performing image processing on the initial panoramic image based on an image enhancement algorithm to obtain the bird's-eye view panoramic image.

[0081] Specifically, such as Figure 2 As shown, image acquisition and preprocessing are performed. Real-time images from the vehicle's surround-view cameras are acquired and preprocessed to generate a bird's-eye panoramic image. The system can accept raw video streams from multiple fisheye cameras around the vehicle and output high-quality, real-time updated BEV images (bird's-eye panoramic images). The image acquisition and preprocessing process includes: i. distortion correction of the raw images to eliminate radial distortion from the fisheye lenses; ii. viewpoint transformation to project the images onto a unified ground-plane coordinate system; iii. image stitching to generate a 360-degree bird's-eye (BEV) panoramic image centered on the vehicle; and iv. application of image enhancement algorithms to improve image quality and contrast.

[0082] For example, the images captured by a vehicle-mounted surround-view camera (e.g., four cameras: front, rear, left, and right) may have curved edges (fisheye effect) and overlap in the middle, making direct stitching impossible. The original video stream may include four (or more) original fisheye images with distortion and uneven exposure. This embodiment uses geometric correction (distortion removal + IPM) to solve the distortion problem; topology reconstruction (stitching) to solve the spatial relationship problem; and image quality optimization (enhancement) to solve the visual perception problem.

[0083] For example, this embodiment employs distortion correction. Using calibrated parameters, the image is stretched in reverse to straighten curved roads, restore flattened objects to their normal proportions, and recreate the true physical geometry, eliminating radial distortion from the fisheye lens. This embodiment also uses perspective transformation (IPM), for example, applying an inverse perspective transformation algorithm. Assuming the ground is flat, mathematical matrix operations are used to smooth out the "nearer objects appear larger, farther objects smaller" perspective effect in the image. Now, with four straightened, top-down images whose edges overlap, this embodiment uses image stitching to find feature points in the overlapping areas, align, cut, and merge them to generate a 360-degree bird's-eye view (BEV) panoramic image centered on the vehicle. Furthermore, to further improve image quality and contrast, this embodiment employs image enhancement and retouching to adjust the brightness, contrast, and saturation of the entire image, making the final output image clear, bright, and color-uniform, allowing users to easily see ground markings and obstacles. Outputs high-quality BEV images (bird's-eye view panoramic images) that are updated in real time. For example, as the vehicle moves, the image can be refreshed more than 30 times per second, outputting seamless, clear bird's-eye view panoramic images without obvious misalignment or color difference.

[0084] In some embodiments, processing the bird's-eye view panoramic image to obtain a binary mask identifying the vehicle shadow region includes: performing color space conversion on the bird's-eye view panoramic image to obtain a converted image; performing adaptive threshold segmentation on the converted image in the luminance channel to obtain an initial binary image; and performing geometric constraint post-processing on the initial binary image to obtain a binary mask identifying the vehicle shadow region.

[0085] Specifically, such as Figure 2 As shown, vehicle shadow detection and segmentation detects and segments the shadow region projected by the vehicle itself in a bird's-eye view panoramic image. It can take a preprocessed BEV image (bird's-eye view panoramic image) as input and output a binary mask that accurately identifies the vehicle shadow region. The vehicle shadow detection and segmentation process includes:

[0086] i. Color Space Conversion: Converting an RGB image to the HSV color space, taking advantage of its robustness to changes in lighting conditions. The conversion formula is:

[0087]

[0088] ii. Adaptive thresholding: The Otsu algorithm is applied to dynamically determine the optimal segmentation threshold on the luminance channel.

[0089]

[0090] in, These represent the pixel ratios of the background and foreground, respectively. represents their average gray level; T represents the optimal segmentation threshold.

[0091] iii. Post-processing of geometric constraints: Using prior knowledge of the vehicle model, filter out isolated dark areas that are not connected to the bottom contour of the vehicle.

[0092] For example, in this embodiment, color space conversion is used to convert the RGB image (red, green, blue) to the HSV (hue, saturation, brightness) color space, where the V (value / brightness) channel is independent of color information (H and S). By converting to the HSV space, we can focus on the V channel (brightness) and ignore color changes. This makes it possible to stably identify dark areas whether in strong midday light or weak evening light.

[0093] For example, this embodiment employs adaptive thresholding to dynamically find the boundary between light and dark areas. Thresholding involves setting a line, classifying areas darker than this line as shadows and brighter areas as the road surface. However, fixed thresholds (e.g., defining areas with a brightness less than 50 as shadows) are fragile because ambient light changes constantly. This embodiment can use the Otsu algorithm, an automatic image binarization algorithm that automatically analyzes the image's brightness histogram to find an optimal value that maximizes the inter-class variance between the foreground (shadow) and background (road surface) at this threshold. This transforms the continuous changes in brightness into binary images that are either black or white (potentially shadow areas).

[0094] For example, in this embodiment, after color space conversion and adaptive threshold segmentation, the binary image may contain many black objects besides shadows, such as asphalt cracks on the road surface, shadows of distant bushes, or black spots caused by lens dirt. Here, prior knowledge can include the fact that vehicle shadows are always connected to the bottom of the vehicle. This embodiment uses geometric constraint post-processing to lock the bottom position of the vehicle in the image (usually a specific area at the bottom of the image or determined by a vehicle detection box), and then performs connected component analysis. For example, if a dark spot does not touch the bottom of the vehicle body, it is considered noise and directly removed. Finally, a binary mask accurately identifies the vehicle shadow area, thereby greatly improving the purity of shadow detection and avoiding false detections.

[0095] In some embodiments, the main light source parameter set is obtained by estimating the main light source parameters based on the vehicle 3D model using the bird's-eye view panoramic image and a binarized mask. This includes: acquiring the vehicle 3D model; estimating the light source direction based on the vehicle 3D model, the bird's-eye view panoramic image, and the binarized mask to obtain the main light source direction vector; estimating the light source color and intensity based on the vehicle 3D model, the bird's-eye view panoramic image, the binarized mask, and the main light source direction vector to obtain the light source color and intensity parameters; and generating the main light source parameter set based on the main light source direction vector and the light source color and intensity parameters.

[0096] In some embodiments, estimating the light source direction based on the vehicle 3D model, the bird's-eye view panoramic image, and the binarized mask to obtain the main light source direction vector includes: selecting key points based on the vehicle 3D model; performing projection point matching on the key points based on the bird's-eye view panoramic image and the binarized mask to obtain paired points; calculating the initial three-dimensional direction vector of the light rays based on the geometric projection relationship and the paired points; and optimizing the initial three-dimensional direction vector according to the random sampling consensus algorithm to obtain the main light source direction vector.

[0097] Specifically, such as Figure 2 As shown, the main light source parameters are estimated. Using the vehicle's known 3D model as a reference, the precise direction of the main light source is determined by inverse geometry through establishing the geometric relationship between key points of the vehicle body and their projection points on the ground shadow contour. Inputs can include BEV images, shadow masks, and vehicle 3D model data, and the output is a complete set of main light source parameters (direction vector, diffuse and specular color and intensity). In this embodiment, based on the geometric features of the shadow area (binarized mask) and combined with a preset vehicle 3D model, the 3D direction vector of the scene's main light source is calculated through geometric inverse geometry. Based on the Phong lighting model, the diffuse and specular components of the main light source, as well as the color and intensity parameters of the ambient light, are estimated by analyzing the image features of the illuminated and shadowed areas. Figure 3 As shown, the process for estimating the parameters of the main light source includes:

[0098] A. Estimation of light source direction:

[0099] i. Key point selection: On the vehicle 3D model, a set of key points with fixed positions and obvious features (such as roof corners, rearview mirror vertices, etc.) are predefined and their three-dimensional coordinates in the vehicle coordinate system are stored.

[0100] ii. Projection point matching: For each keypoint, based on its two-dimensional vertical projection position, a search is performed along multiple directions on the shadow mask contour to find the best projection point match.

[0101] iii. Geometric Calculation: Based on the pairing of each "3D keypoint-2D projection point", the three-dimensional direction vector of the light ray (including azimuth and elevation angles) can be calculated using geometric projection relationships. Its mathematical expression can be:

[0102]

[0103] in, Three-dimensional coordinates of key points of the vehicle; These are the coordinates of the corresponding shadow projection point.

[0104] iv. RANSAC Optimization: Due to matching errors, the results of single-point solutions may be inaccurate. This embodiment can use the RANSAC (Random Sample Consensus) algorithm to iteratively find the optimal solution from multiple pairs, eliminate interference from abnormal pairs, and finally calculate a stable and accurate main light source direction vector.

[0105] B. Estimation of light source color and intensity:

[0106] Comprehensive illumination parameter estimation based on the Phong illumination model:

[0107]

[0108] Among them, I phong This represents the final intensity of a pixel calculated using the Phong lighting model; I ambient Indicates ambient light intensity; I diffuse Indicates diffuse reflection intensity; I specular This indicates the intensity of light reflected from the specular surface.

[0109] i. Diffuse component estimation: Estimate the diffuse parameters of the main light source by comparing the color difference between the inner and outer areas of the shadow edge.

[0110]

[0111] Among them, I diffuse Indicates diffuse reflection intensity; C diffuse k represents the diffuse color coefficient. d L represents the diffuse reflection coefficient; L represents the light source direction vector; N represents the surface normal vector.

[0112] ii. Specular reflection component estimation: Specular reflection parameters are estimated by searching for highlight points on the vehicle body surface.

[0113]

[0114] Where R represents the reflection vector, V represents the observation vector, and the reflection vector... I specular Indicates the intensity of specular reflection; C speculark represents the specular reflection color coefficient. s α represents the specular reflection coefficient; α represents the specular index, which controls the sharpness of highlights.

[0115] For example, light source direction estimation: the vehicle 3D model provides key point coordinates, the shadow mask provides shadow contours, and the BEV image / projection matrix provides spatial mapping relationships. Point selection, matching, calculation, and RANSAC optimization are performed to output the main light source direction vector, including azimuth and elevation angles.

[0116] In one example, key points are selected for ease of calculation, such as the outermost point of the rearview mirror or the four highest corners of the roof, because the projection relationship of these points is clearest and less likely to be obscured by the vehicle body itself. Projection point matching: Since the shadow mask is a black and white block with jagged edges composed of pixels, the search starts from directly below the key point (vertical projection from the BEV perspective) and radiates outwards. Once the edge pixel of the shadow mask is touched, it is considered that the shadow of the key point has been found, forming a "point-shadow" pair (pairing point). Geometric calculation: In space, a straight line is formed by connecting the "light source", "3D key point", and "ground projection point". Knowing the coordinates of two points (the point on the vehicle and the shadow point on the ground), the vector direction of this line can be calculated. This embodiment uses RANSAC optimization for noise reduction and stability. The RANSAC algorithm is used to randomly sample a few pairs of points to calculate a direction, and then check whether most other points support this direction. Through multiple iterations, outliers (such as incorrectly matched points) are eliminated, and the most accurate light source vector is calculated using the points that conform to the majority of cases.

[0117] For example, light source color and intensity estimation: the original BEV image provides color values, the shadow mask provides region division, the vehicle 3D model provides the vehicle body range and the light source direction (main light source direction vector), and by comparing differences and searching for highlights, the output light source color and intensity parameters include diffuse color / intensity and specular color / intensity.

[0118] In one example, after knowing the direction, it's also necessary to know the quality of the light (color, brightness, glare). This embodiment can introduce the Phong lighting model from computer graphics. Diffuse component estimation: Compare the road surface color outside the shadow area (lit) and inside the shadow area (unlit). The difference between the two is actually the diffuse color and intensity brought by the main light source. For example, if the lit road surface is much yellowish and brighter than the shadowed road surface, it indicates that the main light source is warm and strong. Specular component estimation: Search for pixels with extremely high brightness (close to saturation) on the vehicle surface. The color of these highlight points is usually very close to the color of the light source itself (e.g., sunlight is white light, and highlights are white). By analyzing the brightness of these points, the specular reflection intensity parameters of the light source are calculated.

[0119] In this embodiment, through shadow detection and geometric inverse kinematics, closed-loop illumination perception based on the vehicle's own shadow is realized for the first time. It can accurately perceive the local real illumination environment such as tunnels, underground garages, and building shadows, and realize closed-loop illumination perception. This fundamentally solves the problem that existing technologies rely on external prediction and cannot adapt to complex scenarios.

[0120] In this embodiment, through geometric calculation and RANSAC optimization, the three-dimensional direction vector of the main light source can be accurately calculated, providing accurate light source direction calculation. This provides a mathematical basis for generating correct shadow casting and specular reflection effects, significantly improving the realism of the rendering.

[0121] In this embodiment, the robustness of the system is significantly improved: through multiple image processing techniques (color space conversion, adaptive thresholding, geometric constraints) for vehicle shadow detection and segmentation and the RANSAC optimization algorithm for main light source parameter estimation, the system can maintain stable and accurate parameter estimation under complex lighting conditions, which greatly improves the reliability of the system, i.e., significantly improves the robustness of the system.

[0122] It is understandable that shadow detection can also employ deep learning-based semantic segmentation methods instead of color space + adaptive thresholding, contour extraction based on edge detection, or region segmentation based on texture analysis. Light source direction calculation can also use least squares instead of the RANSAC algorithm for parameter optimization, triangulation based on multi-view geometry, or direction prediction using a regression model based on machine learning. Illumination parameter estimation can also employ illumination estimation methods based on HDR (High Dynamic Range) images, ambient light modeling based on spherical harmonic functions, or surface normal vector estimation based on photometric stereo vision. This embodiment does not impose any limitations on these methods.

[0123] In some embodiments, determining ambient light parameters based on the bird's-eye view panoramic image and a binarized mask includes: obtaining shadow region pixels based on the binarized mask and the bird's-eye view panoramic image; determining the average RGB value of the shadow region pixels; obtaining ambient light color and ambient light intensity based on the average RGB value; and using the ambient light color and ambient light intensity as ambient light parameters.

[0124] Specifically, such as Figure 2 As shown, ambient light parameters are determined. Inputs can be shadow masks or BEV images, and outputs the color and intensity parameters of the ambient light (ambient light parameters). The process for determining ambient light parameters includes:

[0125] i. Region pixel extraction: Obtain all pixels covered by the shadow mask.

[0126] ii. Parameter Calculation: Calculate the average RGB value of the pixels in the shadow area as the ambient light color, and estimate the ambient light intensity:

[0127]

[0128] Where S is the set of pixels in the shadow region; I(p) is the pixel brightness value.

[0129] iii. Robustness optimization: Remove extreme outliers and introduce the ground color of non-shaded areas as a reference for correction.

[0130] It's important to note that shadows lack direct sunlight, containing only diffused light from the sky—the so-called ambient light. Therefore, shadowed areas act as natural ambient light detectors. When the main light source (the sun) is blocked by a car, the shadowed area on the ground cuts off direct sunlight. Consequently, the colors and brightness seen in shadows most closely resemble the effect of pure ambient light; analyzing shadows helps determine ambient light.

[0131] For example, ambient light parameters are determined as follows: the input BEV image provides the actual color data (RGB pixels), and the shadow mask provides the sampling location of the data; the output is the color and intensity of the ambient light (ambient light parameters). Specifically, all pixels covered by the shadow mask are obtained to determine the calculation range, processing only the dark areas of the light source and ignoring the lit areas. After obtaining the extracted shadow pixels, the RGB values ​​of these pixels are calculated. Color: The average value is calculated; for example, if the road surface in the shadow has a bluish tint, it indicates that the sky is blue (Rayleigh scattering), which is the color of the ambient light. Intensity: The average brightness value is calculated. Through robust optimization, extreme outliers are removed, and the ground color in non-shadow areas is introduced as a reference for correction to prevent false detections and improve the accuracy and stability of the parameters.

[0132] In this embodiment, by estimating the color and intensity of the light source and determining the ambient light parameters, the diffuse reflection and specular reflection components of the main light source and the ambient light parameters can be fully obtained, thus constructing a complete and dynamic Phong lighting model. Compared with the simplified model of the prior art, it can produce a more realistic rendering effect.

[0133] In some embodiments, 3D vehicle model rendering is performed based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

[0134] Specifically, such as Figure 2 As shown, real-time rendering and compositing utilizes all calculated lighting parameters to render the vehicle's 3D model in real time and composite it onto the bird's-eye view. Inputs include main light source parameters, ambient light parameters, the vehicle's 3D model, and the original BEV image (bird's-eye view panoramic image). Output: dynamic, high-fidelity 3D surround-view image. For example, the real-time rendering and compositing process includes:

[0135] i. Rendering Engine Settings: Configure the dynamic lighting environment in the rendering engine. This includes: creating directional lights, setting their direction, diffuse reflection, and specular reflection parameters; and setting the scene's ambient light parameters.

[0136] ii. Vehicle Model Rendering: Assign PBR material properties to different parts of the vehicle and perform high-quality rendering.

[0137] iii. Image Compositing: Alpha Blending technology can be used to seamlessly blend the rendered result into the original image.

[0138]

[0139] Among them, C out Indicates the final mixed color; C src Indicates the color of the original image (or foreground color); C dst Indicates the color (or background color) of the target image; α src This represents the transparency coefficient of the original image.

[0140] For example, the rendering engine settings include: creating parallel lights to represent the sun, using previously calculated direction vectors to place the virtual sun in the correct position in the sky; setting colors, for example, white light at noon and warm light at dusk, to match its brightness to real sunlight. Ambient light is set to represent the diffuse reflection of the sky, adjusting the background color to ensure the shadow colors are also natural, for example, shadows are grayish on cloudy days and bluish on sunny days. For car model rendering: adding texture, PBR materials are physically based rendering; for example, car paint is metallic (highly reflective), tires are rubber (lowly reflective), and glass is transparent. The rendering engine calculates the refraction and reflection of light hitting the car paint and glass based on the set lighting, generating a single (or sequence of) vehicle image with realistic lighting effects. At this time, the background is usually transparent. Image compositing: Alpha Blending pixel-level blending technology can be used, with the original image serving as the background and the rendered result serving as the foreground. By utilizing the Alpha channel (transparency channel), edge transitions such as feathering can be processed, so that the edges of the virtual car and the ground are smooth without jagged edges, making it look like a real car parked there. This solves the problem of simple texture occlusion and achieves seamless blending.

[0141] In this embodiment, through high-quality rendering and Alpha Blending compositing technology using real-time rendering and compositing, the generated 3D car model can be seamlessly integrated with the real environment, achieving seamless integration of virtual and real, significantly improving the user's visual experience and immersion, and bringing a qualitative leap to the in-vehicle HMI system.

[0142] It is understood that the method described in this embodiment is based entirely on real-time data from the vehicle-mounted camera for light perception, without relying on network connections or external data sources, thus reducing dependence on external factors and improving the independence and reliability of the system.

[0143] It should be noted that the rendering technology can also use ray tracing-based global illumination rendering to replace the Phong model, image-based lighting (IBL) technology can be used, and deferred rendering pipeline can be used to improve rendering efficiency, etc. This embodiment does not limit these aspects.

[0144] The vehicle 3D model lighting rendering method provided in this embodiment includes the following core technologies: Closed-loop lighting perception based on the vehicle's own shadow. This method, for the first time, proposes using the shadow cast by the vehicle itself as the information source for lighting perception, achieving a fundamental shift from "macro-level prediction" to "micro-level real-time perception," and solving the fundamental deficiency of existing technologies in adapting to local realistic lighting environments. A geometrically constrained inverse kinematics algorithm for the main light source direction is employed. Using a known onboard 3D model as a calibration object, the geometric relationship between key points on the vehicle body and shadow contour points is established. Combined with the RANSAC optimization algorithm, this achieves accurate calculation of the three-dimensional direction vector of the main light source, providing a mathematical foundation for generating correct shadow projection and specular reflection effects. A full-parameter lighting estimation technique based on the Phong model is used, which not only calculates the light source direction but also comprehensively estimates the diffuse and specular reflection components of the main light source, as well as ambient light parameters. By analyzing the image features of the illuminated area, shadow area, and highlight points, a complete and dynamic lighting model is constructed, significantly improving the realism and accuracy of the rendering effect. This embodiment employs a robust design that integrates multiple technologies: combining various image processing techniques such as color space transformation, adaptive thresholding, and geometric constraints, and using the RANSAC algorithm to eliminate abnormal matches, ensuring the stability of parameter estimation and guaranteeing the reliability and accuracy of the system in complex scenarios.

[0145] The vehicle 3D model lighting rendering method provided in this embodiment includes: acquiring a bird's-eye view panoramic image captured by a surround-view camera; processing the bird's-eye view panoramic image to obtain a binary mask identifying the vehicle's shadow area; estimating the main light source parameters based on the vehicle 3D model using the bird's-eye view panoramic image and the binary mask to obtain a main light source parameter set; determining the ambient light parameters based on the bird's-eye view panoramic image and the binary mask; and rendering the 3D vehicle model based on the main light source parameter set, ambient light parameters, the vehicle 3D model, and the bird's-eye view panoramic image. This embodiment uses the shadows cast by the vehicle itself as the information source for lighting perception, achieving a fundamental shift from "macroscopic prediction" to "microscopic real-time perception," solving the fundamental defect of existing technologies that cannot adapt to local realistic lighting environments. Through accurate calculation of light source direction and full parameter estimation, a mathematical basis is provided for generating correct shadow projection and specular reflection effects, significantly improving the realism of the rendering and producing more lifelike rendering effects.

[0146] Reference Figure 4 , Figure 4 This is a structural block diagram of an embodiment of the vehicle 3D model lighting rendering system of the present invention. Figure 4 As shown, the vehicle 3D model lighting and rendering system includes:

[0147] Image acquisition module 10 is used to acquire bird's-eye view panoramic images captured by the surround-view camera;

[0148] Image processing module 20 is used to process the bird's-eye view panoramic image to obtain a binary mask that identifies the shadow area of ​​the vehicle;

[0149] The main light source parameter estimation module 30 is used to estimate the main light source parameters based on the bird's-eye view panoramic image and the binarized mask according to the vehicle 3D model, and obtain the main light source parameter set.

[0150] Ambient light parameter determination module 40 is used to determine ambient light parameters based on the bird's-eye view panoramic image and the binarized mask;

[0151] The rendering and compositing module 50 is used to render a 3D vehicle model based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

[0152] The vehicle 3D model lighting rendering system provided in this embodiment uses the shadows cast by the vehicle itself as the information source for lighting perception, achieving a fundamental shift from "macroscopic prediction" to "microscopic real-time perception," thus solving the fundamental defect of existing technologies that cannot adapt to local realistic lighting environments. Through precise calculation of light source direction and full parameter estimation, it provides a mathematical basis for generating correct shadow projection and specular reflection effects, significantly improving the realism of the rendering and producing more lifelike rendering results.

[0153] In addition, for technical details not described in detail in this embodiment of the vehicle 3D model lighting rendering system, please refer to the vehicle 3D model lighting rendering method provided in any embodiment of the present invention, which will not be repeated here.

[0154] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the vehicle 3D model lighting rendering methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0155] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0156] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0157] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0158] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps in any of the vehicle 3D model lighting rendering methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0159] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described vehicle 3D model lighting rendering method.

[0160] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0161] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0162] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0163] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0164] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0165] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0166] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0167] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0169] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for lighting and rendering a 3D vehicle model, characterized in that, include: Acquire bird's-eye view panoramic images captured by surround-view cameras; The bird's-eye view panoramic image is processed to obtain a binary mask that identifies the shadow areas of the vehicles; Based on the vehicle 3D model, the main light source parameters are estimated using the bird's-eye view panoramic image and the binarized mask to obtain the main light source parameter set; Ambient light parameters are determined based on the bird's-eye view panoramic image and the binarized mask; 3D vehicle model rendering is performed based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

2. The method according to claim 1, characterized in that, The acquisition of bird's-eye view panoramic images captured by the surround-view camera includes: The original video stream is acquired based on the vehicle's fisheye camera, and the original image is extracted from the original video stream. The original image is subjected to distortion correction to obtain the corrected image; The corrected image is then transformed by a viewpoint and projected onto a unified ground plane coordinate system to obtain the transformed image. The transformed images are stitched together to obtain a bird's-eye view panoramic image.

3. The method according to claim 2, characterized in that, The step of stitching together the transformed image to obtain a bird's-eye view panoramic image includes: The transformed image is stitched together to generate an initial panoramic image centered on the vehicle. The initial panoramic image is processed using an image enhancement algorithm to obtain a bird's-eye view panoramic image.

4. The method according to claim 1, characterized in that, The process of processing the bird's-eye view panoramic image to obtain a binary mask identifying the shadow areas of the vehicles includes: The bird's-eye view panoramic image is converted to a different color space to obtain the converted image. The converted image is subjected to adaptive threshold segmentation on the brightness channel to obtain an initial binary image; After performing geometric constraint post-processing on the initial binary image, a binary mask identifying the shadow region of the vehicle is obtained.

5. The method according to claim 1, characterized in that, The main light source parameter estimation based on the vehicle 3D model is performed using the bird's-eye view panoramic image and a binarized mask to obtain a main light source parameter set, including: Obtain a 3D model of the vehicle; Based on the vehicle 3D model, bird's-eye view panoramic image and binarized mask, the light source direction is estimated to obtain the main light source direction vector; Based on the vehicle 3D model, bird's-eye view panoramic image, binarized mask and main light source direction vector, the light source color and intensity are estimated to obtain the light source color and intensity parameters. A main light source parameter set is generated based on the main light source direction vector and the light source color and intensity parameters.

6. The method according to claim 5, characterized in that, The step of estimating the light source direction based on the vehicle 3D model, the bird's-eye view panoramic image, and the binarized mask to obtain the main light source direction vector includes: Select key points based on the vehicle 3D model; Based on the bird's-eye view panoramic image and the binarized mask, the key points are matched by projection points to obtain paired points; The initial three-dimensional direction vector of the light ray is obtained based on the geometric projection relationship and the pairing points; The initial three-dimensional direction vector is optimized using a random sampling consensus algorithm to obtain the main light source direction vector.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining ambient light parameters based on the bird's-eye view panoramic image and the binarized mask includes: The shadow region pixels are obtained based on the binarized mask and the bird's-eye view panoramic image; Determine the average RGB value of the pixels in the shaded area; The ambient light color and ambient light intensity are obtained based on the average RGB values. The ambient light color and ambient light intensity are used as ambient light parameters.

8. A vehicle 3D model lighting and rendering system, characterized in that, include: The image acquisition module is used to acquire bird's-eye view panoramic images captured by the surround-view camera; The image processing module is used to process the bird's-eye view panoramic image to obtain a binary mask that identifies the shadow areas of the vehicles. The main light source parameter estimation module is used to estimate the main light source parameters based on the vehicle 3D model according to the bird's-eye panoramic image and the binarized mask, and obtain the main light source parameter set. An ambient light parameter determination module is used to determine ambient light parameters based on the bird's-eye view panoramic image and a binarized mask. The rendering and compositing module is used to render 3D vehicle models based on the main light source parameter set, ambient light parameters, vehicle 3D model, and bird's-eye view panoramic image.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images