Identity recognition-based vehicle-mounted welcome projection control method and device

Abstract

This invention relates to the field of automotive electronics technology, specifically to a vehicle-mounted welcome projection control method and device based on identity recognition. The invention determines the confirmation weight and target projection area by matching user characteristics; accordingly, it determines a rendering strategy including the allocation of computing resources; it uses a ray tracing algorithm to calculate the collision cross-section between the light beam and physical obstacles, generating a dynamic shadow mask matrix; it uses a spatial coordinate mapping model to map this mask matrix to the global ground coordinate system, identifies blind-spot pixels, and locates their registration coordinates within the field of view of the auxiliary projection module; based on the rendering strategy, it shuts down the corresponding blind-spot output of the main projection module, drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, and finally outputs a seamlessly stitched relay projection frame. This invention solves the problem of light path interference under dynamic occlusion environments and achieves cross-hardware pixel-level collaborative stitching under limited computing power.

Description

Technical Field

[0001] This invention relates to the field of automotive electronics technology, specifically to a method and device for controlling in-vehicle welcome projection based on identity recognition. Background Technology

[0002] With the development of intelligent connected vehicle technology, in-vehicle external welcome projection technology has evolved from a single static sign projection to an image communication carrier with high resolution and dynamic interactive capabilities. In order to expand the projection coverage and enhance the visual effect, existing vehicles typically have multiple projection modules distributed and installed in different locations such as rearview mirrors, door handles, or side skirts.

[0003] However, the aforementioned distributed projection systems have certain technical shortcomings when facing complex dynamic scenarios with multiple concurrent targets. Firstly, regarding physical light path propagation, due to the low installation height and long projection path of the projection modules, when multiple users or obstacles approach a vehicle, the physical light path between the projection module and the ground projection area can be easily severed, leading to severe black spots, incompleteness, or even visual interference caused by direct light hitting the human body in the ground welcome pattern. Secondly, regarding distributed hardware collaboration, the multiple projection modules in existing systems typically operate in isolation. When a light path is blocked, the system lacks cross-hardware pixel-level spatial registration and compensation capabilities. Although the system incorporates heterogeneous sensors for perception, existing technology cannot convert the perceived obstacle spatial envelope into a real-time optical mask matrix, nor can it drive the remaining unblocked projection modules to perform precise pixel relay on the damaged image. Finally, at the image processing and driving level, due to the significant differences in projection angles between projection modules at different locations, simple multi-source superposition can lead to severe geometric ghosting and light intensity overload in the overlapping projection area.

[0004] In summary, how to utilize limited distributed projection resources to achieve physical anti-interference of optical output and pixel-level seamless splicing in a dynamic occlusion environment is a technical problem that urgently needs to be solved in this field.

[0005] To address this, a vehicle-mounted welcome projection control method and device based on identity recognition are proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a vehicle-mounted welcome projection control method and device based on identity recognition. This invention determines the confirmation weight and target projection area by matching user characteristics; accordingly, it determines a rendering strategy including the allocation ratio of computing resources; it uses a ray tracing algorithm to calculate the collision cross-section of the light beam and physical obstacles, generating a dynamic shadow mask matrix; it uses a spatial coordinate mapping model to map this mask matrix to the global ground coordinate system, identifies blind zone pixels and locates their registration coordinates within the field of view of the auxiliary projection module; according to the rendering strategy, it shuts down the corresponding blind zone output of the main projection module, drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, and finally outputs a seamlessly stitched relay projection frame. This invention solves the problem of light path interference under dynamic occlusion environments and realizes cross-hardware pixel-level collaborative stitching under limited computing power.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A vehicle-mounted welcome projection control method based on identity recognition is applied to a vehicle-mounted system that includes multiple distributed projection modules and an environmental perception module, including: The system acquires user feature information and matches it with a historical user feature database to determine the user's identity and corresponding confirmation weight; it determines the target projection area based on the user's real-time 3D spatial coordinates outside the vehicle; and it determines the rendering strategy for the target projection area based on the confirmation weight and the target projection area. Using the target projection area, the optical center coordinates of each distributed projection module, and the physical obstacle contours obtained by the environmental perception module as input parameters, the ray tracing algorithm is used to calculate the beam collision cross section in real time, generating a dynamic shadow mask matrix corresponding to the projection field of view of each distributed projection module. The dynamic shadow masking matrix is ​​mapped to the global ground coordinate system using a spatial coordinate mapping model. The blind pixels in the target projection area are identified, and the registration coordinates of the blind pixels in the projection range of the other unmasked distributed projection modules are located. The controller shuts down the output of the corresponding blind pixels of the main projection module according to the rendering strategy, and drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, and outputs seamlessly stitched relay projection frames.

[0008] Preferably, the steps for determining the user identity identifier and the corresponding confirmation weight include: acquiring spatial distance parameters using an ultra-wideband radar node and acquiring three-dimensional visual parameters using a multispectral camera; outputting a multimodal fusion feature matrix after temporal alignment of the spatial distance parameters and the three-dimensional visual parameters; comparing the multimodal fusion feature matrix with a pre-stored historical user feature database, calculating the Euclidean distance of each entry, and if the entry with the smallest Euclidean distance value is lower than a preset security identification threshold, then the entry is taken as the matching target, and the user identity identifier associated with the matching target is output; a preset non-zero smoothing bias constant is superimposed on the Euclidean distance value corresponding to the matching target, the reciprocal of the superimposed value is calculated and normalized, and the resulting value is assigned as the confirmation weight to the user identity identifier.

[0009] Preferably, the pixel grid size parameter of the target projection area is extracted; the difference between the confirmation weight and the system's preset computing power allocation benchmark threshold is calculated; the difference and the pixel grid size parameter are input into a preset multivariate mapping function to obtain a dynamic computing power allocation coefficient including a basic weight coefficient and a spatial scale compensation coefficient; a rendering strategy including the dynamic computing power allocation coefficient is generated, instructing the graphics processing unit to establish the calling ratio of the stream processor cores according to the dynamic computing power allocation coefficient, and activating a corresponding number of stream processor cores to perform ray tracing calculations of the target projection area according to the calling ratio.

[0010] Preferably, the step of generating a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module includes: establishing a three-dimensional Cartesian environmental coordinate system with the vehicle's physical center as the origin; in the three-dimensional Cartesian environmental coordinate system, using the optical center coordinates of each distributed projection module as the emission starting point, emitting virtual ray direction vectors to each grid division vertex of the target projection area; if the virtual ray direction vector has a spatial geometric intersection with the three-dimensional envelope grid surface of the physical obstacle contour, then writing an occlusion assignment at the pixel coordinate position corresponding to the landing point of the virtual ray direction vector in the target projection area; if there is no spatial geometric intersection, then writing a pass-through assignment at the pixel coordinate position corresponding to the landing point of the virtual ray direction vector in the target projection area; and matrix-encapsulating the pixel coordinate positions containing the corresponding assignments to construct an output two-dimensional binary matrix as the dynamic shadow masking matrix.

[0011] Preferably, when the target projection area is determined to be a standard horizontal plane, the spatial coordinate mapping model is a homography transformation parameter matrix; the steps of identifying blind pixels within the target projection area and locating registration coordinates include: calculating the homography transformation parameter matrix by combining the spatial pose parameters calculated in real time by the environmental perception module and the local perspective transformation parameters of the main projection module; performing matrix multiplication on the pixels carrying occlusion assignments in the dynamic shadow masking matrix and the homography transformation parameter matrix to obtain the absolute physical coordinates in the global ground coordinate system and marking them as blind pixels; if the absolute physical coordinates are within the two-dimensional field of view of the auxiliary projection module, using the inverse spatial transformation matrix to reverse transform the absolute physical coordinates to the digital micromirror array plane of the auxiliary projection module to generate registration coordinates.

[0012] Preferably, when it is determined that there is a three-dimensional topological abrupt change in the target projection area with non-standard elevation, the spatial coordinate mapping model is a spatial reprojection model constructed based on the calibration intrinsic and extrinsic parameter matrix and three-dimensional terrain point cloud data; the steps of identifying blind pixels in the target projection area and locating registration coordinates include: calling the three-dimensional terrain point cloud data containing relative elevation parameters collected by the environmental perception module to construct a global three-dimensional topological mesh for the target projection area; according to the calibration intrinsic and extrinsic parameter matrix of the main projection module, projecting the pixels carrying occlusion assignments in the dynamic shadow masking matrix forward onto the surface of the global three-dimensional topological mesh in the manner of line-of-sight ray tracing; calculating the intersection of the ray and the surface of the global three-dimensional topological mesh, obtaining the absolute three-dimensional physical coordinates of the pixel in the global ground coordinate system and marking it as a blind pixel; if the absolute three-dimensional physical coordinates are within the three-dimensional field of view envelope boundary of the auxiliary projection module, performing an inverse perspective projection transformation using the calibration intrinsic and extrinsic parameter matrix to transform the absolute three-dimensional physical coordinates to the digital micromirror array plane and generate registration coordinates.

[0013] Preferably, the steps of driving the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates include: calling the three-dimensional terrain point cloud data and extracting the ground tilt angle and relative elevation difference of the area covered by the corresponding registration coordinates; calculating the distortion correction matrix based on the ground tilt angle and relative elevation difference; using the distortion correction matrix to perform reverse spatial stretching and pixel coordinate offset processing on the source image to be rendered at the registration coordinates to complete the geometric pre-distortion; driving the auxiliary projection module to project the pre-distorted complementary pixels, and synchronously receiving the updated registration coordinates and distortion correction matrix at a set frame rate, and performing continuous frame refresh output; terminating the projection process when a vehicle door opening signal is received and / or the user coordinates deviate from the preset boundary.

[0014] Preferably, the step of outputting seamlessly stitched relay projection frames includes: extracting the spatially overlapping region of the output field-of-view boundary coordinates of the main projection module and the auxiliary projection module; setting a pixel transparency gradient function in the digital image layer of the spatially overlapping region; controlling the pixel transparency of the main projection module in the spatially overlapping region to decrease from full load to zero towards the blind pixel direction according to the pixel transparency gradient function, while simultaneously controlling the pixel transparency of the auxiliary projection module to increase from zero to full load, thereby generating seamlessly stitched relay projection frames through the superposition and complementarity of transparency within the region.

[0015] The vehicle-mounted welcome projection control device based on identity recognition includes: The strategy generation module is used to obtain user feature information through the environment perception module and match it with the historical user feature database to determine the user's identity and corresponding confirmation weight; determine the target projection area based on the user's real-time three-dimensional spatial coordinates outside the vehicle; and determine the rendering strategy for the target projection area based on the confirmation weight and the target projection area. The dynamic masking simulation module is used to calculate the beam collision cross section in real time using the target projection area, the optical center coordinates of each distributed projection module, and the physical obstacle contours obtained by the environment perception module as input parameters, and to generate a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module. The pixel relay module is used to map the dynamic shadow masking matrix to the global ground coordinate system using a spatial coordinate mapping model, identify blind pixels in the target projection area, and locate the registration coordinates of the blind pixels within the projection range of the other unmasked distributed projection modules. The controller shuts down the output of the corresponding blind pixels of the main projection module according to the rendering strategy, and drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, outputting seamlessly stitched relay projection frames.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention introduces a ray tracing algorithm based on multi-source collaboration and geometric correction logic, which can solve the spatial collision between the projected light cone and physical obstacles on the ground in real time, and perform inverse perspective transformation on non-planar terrain, so that the projected content has good geometric integrity and visual presentation accuracy on complex and irregular physical ground, and improves the display robustness of the welcoming image in different road surface environments.

[0017] 2. This invention utilizes identity verification weights and spatial registration mechanisms to achieve collaboration and pixel-level reuse of multiple distributed projection modules. When the system detects light path occlusion, it automatically schedules auxiliary projection modules to perform complementary pixel rendering, transforming multiple projection terminals into a logically unified optical array. This effectively supports the continuity and logical consistency of projected content in environments with multiple target occlusions or complex interactions.

[0018] 3. The present invention dynamically configures the system rendering strategy based on the user identity confirmation weight, which can adjust the allocation scheme of computing resources and light flux in real time according to the interaction priority of different users. This enables the in-vehicle welcome system to meet personalized interaction needs while optimizing the system load through a logical resource scheduling mechanism, thereby enhancing the response efficiency of the welcome function in multi-user concurrent scenarios. Attached Figure Description

[0019] Figure 1 A flowchart of an in-vehicle welcome projection control method based on identity recognition provided in an embodiment of the present invention; Figure 2 A schematic diagram of the vehicle-mounted welcome projection control device based on identity recognition provided in an embodiment of the present invention; Figure 3 This is a diagram of a dynamic shadow mask and pixel complementarity architecture provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1 to 3 This invention provides a vehicle-mounted welcome projection control method and device based on identity recognition, the technical solution of which is as follows: A vehicle-mounted welcome projection control method based on identity recognition is applied to a vehicle-mounted system that includes multiple distributed projection modules and an environmental perception module, including: The system acquires user feature information through the environmental perception module and matches it with the historical user feature database to determine the user's identity and corresponding confirmation weight; it also determines the target projection area based on the user's real-time three-dimensional spatial coordinates outside the vehicle. Based on the confirmed weight and the target projection area, a rendering strategy for the target projection area is determined, wherein the rendering strategy includes the proportion of computing resources allocated according to the confirmed weight. Using the target projection area, the optical center coordinates of each distributed projection module, and the physical obstacle contours obtained by the environmental perception module as input parameters, the ray tracing algorithm is used to calculate the beam collision cross section in real time, generating a dynamic shadow mask matrix corresponding to the projection field of view of each distributed projection module. The dynamic shadow masking matrix is ​​mapped to the global ground coordinate system using a spatial coordinate mapping model. The blind pixels in the target projection area are identified, and the registration coordinates of the blind pixels in the projection range of the other unmasked distributed projection modules are located. The controller shuts down the output of the corresponding blind pixels of the main projection module according to the rendering strategy, and drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, and outputs seamlessly stitched relay projection frames. Example 1:

[0022] This embodiment addresses the typical application scenario of "parking on non-standard elevation paved surfaces." For example, a vehicle is parked half-side on a curb (road step) with a sudden change in elevation or on a non-standard level surface with a significant slope, and an authorized user approaches the vehicle normally along the non-standard elevation surface. This embodiment aims to solve a series of background problems in such complex environmental scenarios, including uneven lighting leading to fluctuations in recognition confidence, physical steps causing light path obstruction and interference, and terrain undulations causing severe topological stretching and distortion of the projected image.

[0023] As one embodiment of the present invention, refer to Figure 1 The flowchart of the vehicle-mounted welcome projection control method based on identity recognition is shown below. Figure 3 Dynamic shadow masking and pixel complementary architecture diagram.

[0024] Further, the steps for determining the user identity identifier and the corresponding confirmation weight include: acquiring spatial distance parameters containing time-of-flight ranging values ​​using an ultra-wideband radar node, and acquiring three-dimensional visual parameters containing facial geometric topology nodes using a multispectral camera; outputting a multimodal fusion feature matrix after temporal alignment of the spatial distance parameters and the three-dimensional visual parameters; comparing the multimodal fusion feature matrix with a pre-stored historical user feature database, calculating the Euclidean distance between the multimodal fusion feature matrix and each entry in the historical user feature database, determining whether the entry with the smallest Euclidean distance value is lower than a preset security identification threshold, and if it is lower, then using the entry as a matching target and outputting the user identity identifier associated with the matching target; superimposing a preset non-zero smoothing bias constant onto the Euclidean distance value corresponding to the matching target, taking the reciprocal of the superimposed value and normalizing it, and assigning the resulting value as the confirmation weight to the user identity identifier.

[0025] The preset security identification threshold is a critical distance value in the feature space set based on the intersection of the false acceptance rate and the false rejection rate under massive sample testing before the system leaves the factory. If the minimum Euclidean distance is higher than this threshold, it is determined to be an unauthorized unfamiliar target.

[0026] In the process of outputting the multimodal fusion feature matrix, the system first maps the one-dimensional spatial distance parameter into a distance feature vector with the same dimension as the three-dimensional visual parameter through a preset scalar dimension expansion function. Then, it expands the coordinate matrix of the facial geometric topology nodes into a visual feature vector. By assigning preset confidence weights to the distance feature vector and the visual feature vector, the system performs weighted concatenation and finally outputs the normalized multimodal fusion feature matrix.

[0027] Specifically, in parking environments with non-standard elevations, complex ambient light and shadow interference (such as streetlight shadows and dark areas around steps) often occurs. The system first utilizes ultra-wideband radar nodes deployed around the vehicle to transmit pulse signals to user targets entering the perception range, acquiring spatial distance parameters including precise time-of-flight ranging values. Simultaneously, a multispectral camera, aided by infrared illumination, extracts 3D visual parameters of the user's facial geometric topology nodes. The system's built-in temporal alignment module unifies the spatial distance parameters and 3D visual parameters onto the same system timeline, outputting a high-dimensional multimodal fusion feature matrix. Subsequently, the system controller reads a pre-stored historical user feature database. This database is constructed using a key-value pair data structure, with the registered user's identity ID as the key and the standard multimodal fusion feature matrix collected and normalized during the user's historical authorization as the value. The system iterates through all entries in the database, extracting their feature matrices, and calculates the Euclidean distance between the currently generated multimodal fusion feature matrix and each entry's feature matrix. The system sets the entry with the smallest Euclidean distance as the matching target and retrieves its associated user identity identifier. To quantify the reliability of identification and avoid computational overflow, the system extracts the smallest Euclidean distance value and superimposes a preset, minimal non-zero smoothing bias constant onto this value. The value of this bias constant is determined based on the background noise variance of the ultra-wideband radar node and multispectral camera, as well as the minimum quantization error of the analog-to-digital converter (e.g., 0.0001). This aims to avoid division-by-zero overflow errors that occur when the distance is perfectly matched, while ensuring that the superposition of this constant does not obscure subtle physical differences between real features. The system takes the reciprocal of the superimposed sum and performs normalization, assigning the result mapped to the 0-1 interval as a confirmation weight to the user's identity identifier, thereby achieving high-confidence anchoring of identity information.

[0028] The confirmation weight is calculated as follows: Let the minimum Euclidean distance corresponding to the matching target be... The total number of entries in the historical user feature database used for comparison is K. First, the original weights are calculated. The smoothing bias constant (This value is determined based on the ultra-wideband radar ranging accuracy of ±10mm and the background noise variance after normalization of the multispectral camera feature dimensions, ensuring...) (The difference is less than the smallest distinguishable feature difference and can effectively prevent division by zero overflow). Then, the Softmax function is used for normalization: ,in This represents the original weight corresponding to the k-th entry in the historical user feature database. The exponential amplification effect of Softmax normalization concentrates the confirmation weight on the identity with the highest matching confidence—when only one candidate's distance is significantly lower than other candidates, its confirmation weight can exceed 0.9; when multiple candidates are close in distance, the weight is evenly distributed, reflecting the uncertainty of multiple candidates.

[0029] This invention effectively overcomes the limitations of single-vision recognition caused by uneven lighting and shadow interference in non-standard elevation road environments by combining spatial ranging of ultra-wideband radar with three-dimensional features of multispectral vision. By utilizing confirmation weights generated through Euclidean distance reciprocal normalization, it not only improves the accuracy of target identification under complex lighting conditions but also transforms recognition confidence into a quantitative numerical indicator, providing objective and reliable logical data support for subsequent system computing power allocation and rendering scheduling.

[0030] Further, the pixel grid size parameter of the target projection area is extracted; the difference between the confirmation weight and the system's preset computing power allocation benchmark threshold is calculated; the difference and the pixel grid size parameter are input into a preset multivariate mapping function to obtain a dynamic computing power allocation coefficient including a basic weight coefficient and a spatial scale compensation coefficient; a rendering strategy including the dynamic computing power allocation coefficient is generated, instructing the graphics processing unit to establish the calling ratio of the stream processor cores according to the dynamic computing power allocation coefficient, and activating a corresponding number of stream processor cores to perform ray tracing calculations of the target projection area according to the calling ratio.

[0031] The specific form of the multivariate mapping function is a piecewise linear function with spatial scale compensation. Let the difference between the confirmation weight and the computing power allocation benchmark threshold be Δw = w_confirm − T_compute (where T_compute is preset to 0.4, indicating that the system reserves 40% of the total computing power for basic instrument display and backend operation), and let the pixel grid size parameter be N_pixels (in megapixels, for example, 1024×1024 corresponds to N_pixels = 1.048). The formula for calculating the dynamic computing power allocation coefficient α_compute is: when hour:

[0032] when hour:

[0033] in, The minimum computing power guarantee factor (meaning at least 15% of the stream processor cores will be used). The reference pixel grid size (approximately 700×700 pixels) is used for reference. The weight gain slope (ensuring that when the confirmed weight increases from 0.4 to 1.0, the computing power allocation coefficient smoothly increases from the minimum value to close to the upper limit). This is a spatial scale compensation coefficient (the coefficient increases by 0.1 for every 1 million pixels added to the pixel grid). The upper limit for computing power allocation is set (15% of computing power is reserved for other system tasks). The parameter values ​​are determined based on benchmark test data of the in-vehicle graphics processing unit in a typical configuration (1024 stream processor cores, 800MHz).

[0034] Specifically, after confirming the user's identity and corresponding weight, the system needs to handle the high-intensity rendering computational load brought about by the stepped and sloping road surface. First, features are extracted from the determined target projection area to obtain the corresponding pixel grid size parameter (e.g., when the distance is close, the projection area needs to cover 1024×1024 grid vertices, while when the distance is far or in a confined space, only 512×512 grid vertices are needed). Then, the controller extracts the user's confirmed weight value and calls the system's preset computing power allocation benchmark threshold. The system's preset computing power allocation benchmark threshold is set based on the minimum idle ratio of the stream processors in the onboard graphics processing unit while maintaining the vehicle's basic instrument display and background system operation (e.g., set to 40% of the total computing power). The system performs difference calculation to obtain the corresponding difference value.

[0035] The difference, along with the pixel grid size parameter, is input into a pre-defined multivariate mapping function to calculate the dynamic computing power allocation coefficient corresponding to the current weight state. Based on this coefficient, the system generates a high-priority rendering strategy for the target projection area and sends low-level control commands to the onboard graphics processing unit. Upon receiving the commands, the graphics processing unit abandons the conventional equal-distribution computing power scheduling mode and strictly activates the corresponding number of stream processor cores according to the calling ratio of the dynamic computing power allocation coefficient. This cluster of selectively activated stream processor cores is dedicated to performing intensive spatial ray tracing and multi-source solution tasks on the current complex road surface. If other low-weight unauthorized targets are detected approaching, the system generates a low computing power calling strategy based on the negative difference, maintaining only basic background monitoring, thus achieving non-linear tilting of onboard computing power based on identity weight.

[0036] This invention employs a dynamic computing power allocation coefficient to control the calling ratio of the stream processor cores in real time, changing the traditional extensive resource allocation mode. When dealing with high-load calculations such as complex ray tracing on non-standard elevation roads, it can accurately allocate underlying computing power based on the target user's identity and weight. This ensures that core users maintain high frame rate characteristics in projection rendering under harsh terrain, while avoiding overheating and frequency throttling caused by indiscriminate computation in vehicle computing chips, significantly improving the system's energy efficiency.

[0037] Further, the step of generating a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module includes: establishing a three-dimensional Cartesian environmental coordinate system with the vehicle's physical center as the origin; in the three-dimensional Cartesian environmental coordinate system, using the optical center coordinates of each distributed projection module as the emission starting point, emitting virtual ray direction vectors to each grid division vertex of the target projection area; calculating whether the virtual ray direction vectors and the three-dimensional envelope grid surface of the physical obstacle contour have a spatial geometric intersection; if a spatial geometric intersection exists, writing an occlusion assignment at the pixel coordinate position corresponding to the landing point of the virtual ray direction vector in the target projection area; if no spatial geometric intersection exists, writing a transmission assignment at the pixel coordinate position corresponding to the landing point of the virtual ray direction vector in the target projection area; and, according to the two-dimensional pixel array arrangement order of the target projection area, matrix-encapsulating the pixel coordinate positions containing the occlusion assignment and the transmission assignment to construct an output two-dimensional binary matrix as the dynamic shadow masking matrix.

[0038] The specific implementation of the ray tracing algorithm is as follows: For each grid vertex (e.g., a 1024×1024 grid) in the target projection region, a 4x multisampling anti-aliasing strategy is adopted—virtual rays are emitted from the optical center coordinates of the distributed projection module to the four sub-sampling point positions generated according to the Halton low-difference sequence within the coverage area of ​​each grid vertex, totaling approximately 4.19 million rays. The ray-triangle intersection operation adopts the Möller-Trumbore algorithm. This algorithm solves the linear equation system of the ray parametric equation and the centroid coordinates of the triangle. A single intersection requires approximately 15 floating-point multiplications and 9 floating-point additions, and does not require pre-calculation of the triangle plane equation, making it suitable for efficient implementation on embedded GPUs. To accelerate the intersection process, the system simultaneously constructs a hierarchical bounding box tree when constructing the 3D envelope mesh of the physical obstacles. The construction of the BVH tree adopts a surface area heuristic partitioning strategy, with the tree depth limited to 16 levels, and leaf nodes containing a maximum of 8 triangle faces. When traversing a BVH tree using rays, the graphics processing unit uses its dedicated hardware-level ray tracing core to perform stack-based iterative traversal, with the recursion depth not exceeding the upper limit of the tree depth, thus avoiding stack space overflow of the embedded chip.

[0039] Specifically, when a vehicle is parked at the edge of a curb, the curb's facade constitutes a significant source of physical occlusion. The system first establishes a three-dimensional Cartesian environmental coordinate system with the vehicle's physical center as the origin. Within this coordinate system, the system emits virtual ray direction vectors one by one to the 1024 grid vertices of the ground target projection area, using the optical center coordinates of each distributed projection module (such as the left rearview mirror node) as the emission starting point. For each virtual ray, the algorithm calculates in real time whether it intersects with the three-dimensional envelope grid surface of the curb step extracted by the environmental perception module. The environmental perception module includes an ultra-wideband radar node, a multispectral camera, and a lidar. Due to significant elevation changes in the curb, some rays will inevitably collide with the curb facade before reaching the expected projection plane. In the two-dimensional pixel buffer corresponding to the target projection area, the system writes an occlusion value of 1 at the coordinates of the virtual ray landing point where the collision intersection occurs; for rays landing directly on an unobstructed, flat surface, a pass-through value of 0 is written at their pixel coordinates. After completing the calculation, the system encapsulates the occlusion assignments and pass-through assignments into a matrix according to the two-dimensional pixel array arrangement order of the target projection area, and outputs a two-dimensional binary matrix that accurately represents the physical blocking state of the step, which is then used as the dynamic shadow masking matrix for subsequent scheduling.

[0040] This invention introduces a virtual ray emission and envelope mesh collision calculation mechanism in a three-dimensional Cartesian coordinate system, enabling the system to possess micromirror-level physical occlusion prediction capabilities. When faced with irregular obstacles such as curbs and steps, this method can accurately extract the light path obstruction region caused by abrupt elevation changes and generate a masking matrix. It blocks visual distortion and glare caused by ineffective light beams hitting the side edges of steps from the bottom pixel level, providing a precise data index for subsequent hardware complementarity implementation.

[0041] Further, when the environmental perception module determines that the target projection area is a standard horizontal topological plane, the spatial coordinate mapping model is a homography transformation parameter matrix; the step of using the spatial coordinate mapping model to map the dynamic shadow masking matrix to the global ground coordinate system, identifying blind pixels in the target projection area and locating the registration coordinates includes: calculating the homography transformation parameter matrix by combining spatial pose parameters and the local perspective transformation parameters of the main projection module; performing matrix multiplication on the pixels carrying occlusion assignments in the dynamic shadow masking matrix and the homography transformation parameter matrix to obtain the absolute physical coordinates of the pixels in the global ground coordinate system and marking them as blind pixels; inputting the absolute physical coordinates into the two-dimensional field of view boundary range function of the auxiliary projection module for set inclusion relationship determination; when it is determined that the absolute physical coordinates exist within the two-dimensional field of view boundary range function of the auxiliary projection module, using the inverse spatial transformation matrix of the auxiliary projection module to inversely transform the absolute physical coordinates to the internal digital micromirror array plane of the auxiliary projection module to generate the registration coordinates.

[0042] The specific mathematical logic for solving the homography transformation parameter matrix is ​​as follows: First, the local perspective transformation parameters of the main projection module are extracted and constructed into a third-order camera intrinsic parameter matrix. Simultaneously, the vehicle's current spatial pose parameters are read, and the vehicle's 3D rotation matrix and 3D translation vector are constructed. Second, under the constraint that the target projection area is determined by the environment perception module to be a standard horizontal topological plane, the unit normal vector of the target ground in the vehicle coordinate system and the vertical distance from the optical center of the projection module to the target ground are obtained.

[0043] Finally, the system performs matrix combination operations based on physical parameters: first, it calculates the product of the 3D translation vector and the transpose of the unit normal vector, and divides the product by the vertical distance to obtain the translation compensation matrix; next, it subtracts the translation compensation matrix from the 3D rotation matrix to obtain the intermediate transformation matrix; finally, it multiplies the camera intrinsic parameter matrix by the intermediate transformation matrix, and then by the inverse of the camera intrinsic parameter matrix. The third-order transformation matrix output by the above continuous matrix operations is the homography transformation parameter matrix. By using this matrix to perform a linear mapping on the dynamic shadow masking matrix, the absolute physical coordinates of the occluded pixels in the global ground coordinate system can be directly calculated.

[0044] When the environmental perception module determines that there is a three-dimensional topological abrupt change in the target projection area with non-standard elevation, the spatial coordinate mapping model is a spatial reprojection model constructed based on the calibration intrinsic and extrinsic parameter matrix and three-dimensional terrain point cloud data. The step of using the spatial coordinate mapping model to map the dynamic shadow mask matrix to the global ground coordinate system, identify blind pixel in the target projection area and locate the registration coordinates includes: calling the three-dimensional terrain point cloud data containing relative elevation parameters collected by the environmental perception module to construct a global three-dimensional topological mesh of the target projection area; according to the calibration intrinsic and extrinsic parameter matrix of the main projection module, the pixels carrying occlusion assignment in the dynamic shadow mask matrix are mapped by the line of sight ray. The tracing method projects the ray forward onto the global 3D topological mesh surface; the intersection of the ray and the global 3D topological mesh surface is calculated, and the absolute physical coordinates of the pixel in the global ground coordinate system are obtained and marked as blind zone pixels; the absolute physical coordinates are input into the 3D field of view envelope boundary of the auxiliary projection module for set inclusion relationship determination; when it is determined that the absolute physical coordinates exist within the 3D field of view envelope boundary of the auxiliary projection module, the absolute physical coordinates are subjected to inverse perspective projection transformation using the calibration intrinsic and extrinsic parameter matrix of the auxiliary projection module, and mapped and transformed to the internal digital micromirror array plane of the auxiliary projection module to generate the registration coordinates.

[0045] The spatial reprojection model is essentially a geometric mapping pipeline composed of a forward ray tracing sub-model and a reverse perspective projection sub-model connected in series. Its detailed execution steps and calculation logic are as follows: The first step is to perform forward ray tracing: using the three-dimensional physical coordinates of the optical center of the main projection module as the fixed starting point of the ray, and using the spatial vector formed by the pixels to be registered in the camera coordinate system carrying the occlusion assignment in the mask matrix as the direction vector of the ray, and constructing the spatial line parametric equation of the line of sight ray. The second step is to perform spatial mesh intersection: the spatial line parameter equation of the line of sight ray is combined with the global three-dimensional topological mesh surface equation pre-generated using three-dimensional terrain point cloud data. The equation system is solved iteratively using computer graphics to determine the first spatial geometric intersection point between the ray and the mesh surface. The coordinates of this intersection point are then extracted as the absolute three-dimensional physical coordinates of the blind zone pixel on the current complex terrain. The third step is to perform inverse projection mapping: the calculated absolute 3D physical coordinates are used as input values ​​and substituted into the inverse perspective projection equation constructed by the calibration intrinsic and extrinsic parameter matrices of the auxiliary projection module. By performing perspective division and coordinate system projection transformation operations from 3D spatial coordinates to the 2D pixel plane inside the auxiliary projection module, the final output is the registration coordinates that perfectly eliminate terrain elevation distortion.

[0046] Specifically, after acquiring the dynamic shadow masking matrix, the system immediately triggers a cross-hardware spatial registration mechanism. To address the complex terrain that a vehicle may encounter when parked, the system controller first calls upon real-time 3D terrain point cloud data collected by environmental perception modules (such as chassis LiDAR and multi-dimensional vision sensors) to assess the flatness of the terrain topology of the target projection area, and dynamically switches the spatial coordinate mapping model accordingly. Scenario 1: Standard Plane Mapping Based on Two-Dimensional Homography Matrix When the system calculates that the elevation variance of each point cloud within the target projection area is lower than the system's preset flatness tolerance (e.g., parked in a standard underground garage or on a smooth asphalt road), the environmental perception module determines that the area is a standard horizontal topological plane. At this time, the controller configures the spatial coordinate mapping model as a homography transformation parameter matrix.

[0047] During pixel mapping, the controller reads the vehicle's global spatial pose parameters (including yaw, pitch, and roll angles) calculated in real time by the environmental perception module, and combines them with the local perspective transformation parameters written into the firmware of the main projection module before it leaves the factory to calculate a homography transformation parameter matrix suitable for the current flat ground. Subsequently, the system performs a matrix multiplication operation between the coordinates of pixels carrying occlusion assignments (i.e., physically occluded) in the dynamic shadow masking matrix and the homography transformation parameter matrix. Through this operation, the system can quickly obtain the absolute physical coordinates of the occluded pixels in the global ground coordinate system (at which point the Z-axis relative elevation is approximately 0) and mark them as blind zone pixels. Next, the system inputs the absolute physical coordinates into the two-dimensional field-of-view boundary range function of the auxiliary projection module for set inclusion relationship determination; when it is verified that the physical coordinates fall within the effective two-dimensional range of the auxiliary projection module, the system calls the inverse spatial transformation matrix of the auxiliary projection module to stretch and transform the absolute physical coordinates in reverse to the digital micromirror array plane inside the auxiliary projection module, accurately generating registration coordinates for subsequent rendering.

[0048] Scenario 2: Non-standard elevation reprojection mapping based on 3D ray tracing When the system calculates that there is an elevation difference exceeding the flatness tolerance within the target projection area (e.g., parking on a curb with steps, speed bumps, or terrain with a significant slope), the environmental perception module determines that there is a three-dimensional topological abrupt change in the area with non-standard elevation. At this time, the homography plane assumption fails, and the controller automatically switches the spatial coordinate mapping model to a spatial reprojection model constructed based on the calibrated intrinsic and extrinsic parameter matrices and three-dimensional terrain point cloud data.

[0049] During pixel mapping, the controller first uses high-frequency collected 3D terrain point cloud data containing relative elevation parameters from the environmental perception module to construct a global 3D topological mesh in the video memory that recreates the true undulation of the target projection area. Then, based on the calibration intrinsic and extrinsic parameter matrices of the main projection module, the system uses line-of-sight ray tracing to project pixels carrying occlusion assignments from the dynamic shadow masking matrix as emission sources onto the surface of the global 3D topological mesh along the light path. By calculating the physical intersection points of the rays and the 3D mesh surface using rigorous calculus or envelope intersection algorithms, the system accurately obtains the absolute physical coordinates of the pixel in the global ground coordinate system (including the true X, Y, and Z axis elevation components) and marks it as a blind zone pixel. Next, the system inputs the absolute physical coordinates containing depth information into the three-dimensional field of view envelope boundary (i.e., the spatial range of the three-dimensional view frustum shape) of the auxiliary projection module to determine the set inclusion relationship; when it is determined that the three-dimensional coordinates exist within the three-dimensional view frustum of the auxiliary projection module, the system uses the calibration intrinsic and extrinsic parameter matrix of the auxiliary projection module to perform an inverse perspective projection transformation (i.e., 3D to 2D dimensionality reduction inverse mapping) on ​​the absolute physical coordinates, and maps and transforms its coordinate system to the internal digital micromirror array plane of the auxiliary projection module to generate registration coordinates that can perfectly compensate for elevation parallax.

[0050] When the system calculates that the elevation variance of each point cloud within the target projection area is lower than the system's preset flatness tolerance, the environmental perception module determines that the area is a standard horizontal topological plane. The preset value of the flatness tolerance is 0.02 (i.e., the elevation variance is less than 2cm, corresponding to the allowable deviation of flatness for standard paved roads). When the elevation variance is not less than the preset value of the flatness tolerance, the environmental perception module determines that there is a three-dimensional topological abrupt change in the area with non-standard elevation and automatically switches to the spatial reprojection model.

[0051] This invention constructs a spatial coordinate mapping mechanism based on environmental perception and adaptive switching. For standard level terrain, the system invokes a computationally efficient homography transformation to save underlying computing power. However, when facing non-standard physical terrain such as curbs and steep slopes with significant Z-axis elevation changes, a "2D-3D-2D" spatial reprojection mapping architecture is constructed by introducing 3D terrain point cloud data and calibrating intrinsic and extrinsic parameter matrices. This overcomes the theoretical limitations of traditional homography matrices, which strictly rely on the assumption of an absolute two-dimensional plane. This adaptive mechanism can accurately obtain the absolute 3D physical coordinates of pixels in the blind zone through line-of-sight ray tracing and mesh intersection, eliminating cross-device registration coordinate misalignment caused by depth parallax. Thus, while considering the system's computational efficiency, it improves the spatial mapping accuracy and geometric integrity of the welcome projection in complex 3D road environments.

[0052] Furthermore, the steps of driving the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates include: when the blind zone pixels are identified as being dispersed into multiple pixel sets exceeding the system's rendering computing power load limit, the controller locks the concurrent rendering computing power quota of the auxiliary projection module and starts the time-division multiplexing relay mode; extracts the user interaction priority weights corresponding to the multiple pixel sets, and allocates corresponding time slice lengths according to the weights; controls the digital micromirror array of the auxiliary projection module to alternately reconstruct and output the registration coordinate regions corresponding to each pixel set within the allocated time slice at a refresh rate exceeding the human eye's flicker fusion critical frequency.

[0053] In non-standard elevation curb environments, the ground may simultaneously contain debris, gravel, or multi-step edges, resulting in an extremely large and dispersed set of blind pixel areas within the target projection area. The specific steps for determining the concurrent computing power load limit and initiating the time-division multiplexing relay mode include: First, obtaining the total number of stream processor cores in the graphics rendering unit of the auxiliary projection module and multiplying it by the system's single-frame time budget to calculate the theoretical maximum computing power resource; then, multiplying the theoretical maximum computing power resource by a preset computing power reduction factor (e.g., 0.7) to reserve computing power margin for handling pixel transparency gradient blending and system scheduling, thus obtaining the actual available concurrent computing power limit; simultaneously, pre-calibrating and extracting the average rendering computing power requirement for each blind pixel set through offline benchmark testing; when the product of the total number of detected blind pixel sets and the average rendering computing power requirement exceeds the actual available concurrent computing power limit, the controller determines that the current rendering demand is overloaded, forcibly locks the current computing power quota, and activates the time-division multiplexing relay mode.

[0054] In time-division multiplexing mode, the system configures the total refresh rate to a fixed high frequency (e.g., 120 Hz), and the corresponding total refresh cycle is the total time budget of a single frame. To ensure that the actual refresh rate of each set is not lower than the conservative safety lower limit of the human eye flicker fusion critical frequency (e.g., 30 Hz) during multi-blind zone polling, the system sets the upper limit of the number of blind zone pixel sets participating in the polling to a fixed value of four. When the total number of blind zone pixel sets detected by the system does not exceed four, all sets participate in time-division multiplexing polling. When the total number of blind zone pixel sets exceeds four, the system starts a fault-tolerant degradation and load reduction strategy, extracts the user interaction priority weights corresponding to all blind zone pixel sets and sorts them in descending order. Only the top four high-priority blind zone pixel sets are selected to participate in the time-division multiplexing relay rendering of the current frame, and the remaining low-priority blind zone pixel sets remain occluded in the current frame and are removed from the system operation log.

[0055] For each blind pixel set selected for polling, the specific rule for calculating the allocated microsecond-level time slice length is as follows: First, calculate the user interaction priority weight of the current blind pixel set as a proportion of the total weight of all selected high-priority blind pixel sets; then, multiply this proportion by the total refresh cycle, and the product is used as the exclusive time slice length for that blind pixel set; through this proportional weighted allocation, it is ensured that the sum of all allocated time slice lengths is strictly equal to the total refresh cycle. Subsequently, the controller drives the digital micromirror array of the auxiliary projection module to rapidly and alternately reconstruct and output the registration coordinate region corresponding to each pixel set within its allocated time slice.

[0056] This invention effectively overcomes the physical hardware computing power bottleneck of in-vehicle distributed projection systems under extreme dynamic occlusion environments by introducing a time-division multiplexing relay mode when the system's concurrent rendering computing power reaches its load limit. Utilizing the visual fusion characteristics of the human eye, the system transforms time-division polling slices into spatial concurrent rendering based on visual perception. Without increasing the size of the graphics processor's stream processor cores, it ensures timely and complete complementary rendering of multiple dispersed blind spots. This fundamentally guarantees system availability and the robust operation of the anti-interference relay mechanism under extremely limited resources.

[0057] Furthermore, the steps of driving the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates also include: calling the three-dimensional elevation spatial point cloud data collected by the vehicle chassis LiDAR; extracting the ground tilt angle and relative elevation difference of the area covered by the corresponding registration coordinates in the three-dimensional elevation spatial point cloud data; calculating the distortion correction matrix based on the ground tilt angle and relative elevation difference; using the distortion correction matrix to perform reverse spatial stretching and pixel coordinate offset processing on the source image to be rendered at the registration coordinates to complete the geometric pre-distortion action; the driving auxiliary projection module projects the image data after geometric pre-distortion as complementary pixels, and synchronously receives the updated registration coordinates and distortion correction matrix at a set frame rate (not less than 30 Hz) for continuous frame refresh output; the projection process is terminated when a vehicle door opening signal is received and / or the user coordinates deviate from the preset boundary.

[0058] The specific steps for calculating the distortion correction matrix based on the ground tilt angle and relative elevation difference include: calculating the geometric stretching coefficient of the projection surface along the corresponding axis using the tangent of the ground tilt angle, and using the ratio of the relative elevation difference to the system reference projection distance as the projection scaling coefficient; using the geometric stretching coefficient as the diagonal element of the distortion correction matrix for local scale scaling compensation, and using the projection scaling coefficient as the translation parameter of the matrix, thereby constructing a complete three-dimensional distortion correction matrix.

[0059] The complete calculation formula for the distortion correction matrix is ​​as follows: Let the components of the ground tilt angle of the registered coordinate coverage area in the X and Y axes of the vehicle coordinate system be respectively... and The relative elevation difference is A positive value indicates that the ground is higher than the reference plane; the system reference projection distance is... Factory calibration value, e.g., 2.0m). Distortion correction matrix. (Homogeneous coordinate form) is defined as:

[0060] in, and These are the geometric stretching pre-compensation coefficients in the X and Y axes, respectively (due to the ground tilt causing the projected pattern to stretch outward along the corresponding axes, the pre-distortion requires performing a reverse compression product); the top two items in the fourth column are the parallax compensation amounts for the view space offset caused by the elevation difference. In the rendering pipeline of the auxiliary projection module, this matrix is ​​inserted into the model-view-projection transformation chain of the vertex shader, executed before perspective division, to achieve geometric pre-distortion processing of the source image.

[0061] Specifically, on curb surfaces with drainage slopes or elevation differences, simple point registration cannot eliminate the stretching and destruction of image topology caused by non-planar terrain. The system controller calls the chassis LiDAR to collect 3D elevation spatial point cloud data in the global ground coordinate system at high frequency, accurately extracting the local ground tilt angle (e.g., 12-degree lateral tilt) and elevation difference value relative to the reference plane (e.g., a sudden drop of 150 mm due to steps) within the corresponding registration coordinate coverage area. Based on these parameters, the calculation module calculates and generates a geometric correction matrix in real time specifically for offsetting this terrain deformation. In the video memory of the auxiliary projection module, the processor uses the geometric correction matrix to force reverse spatial stretching and pixel coordinate offset processing on the source image to be rendered at the target registration coordinates, completing the geometric pre-distortion action of the graphics rendering layer. Subsequently, the auxiliary projection module is driven to project the pre-distorted image data onto the undulating ground, achieving proportional visual restoration through the stretching and offset of the physical ground. In this state, the system synchronously receives the updated coordinates and correction matrix at a frame rate of 60 Hz, maintaining dynamic refresh output. When the sensor detects a gate opening electrical signal or detects that the user's coordinates have deviated from the preset boundary, the system issues a power cut-off command to terminate the projection process. The preset boundary is defined as the effective optical projection safety radius (e.g., 2.5 meters) radiating outward from the vehicle's physical center.

[0062] This invention innovatively introduces a geometric correction matrix calculated from chassis lidar elevation point cloud data, specifically addressing the problem of nonlinear stretching distortion caused by vehicle-mounted projections on complex micro-topological surfaces such as curbs and steep slopes. By utilizing pre-distortion actions of reverse stretching and coordinate offset, the projected pattern maintains the correct visual proportions on irregular media. Simultaneously, a forced termination mechanism based on gating and boundary decoupling constructs a complete control closed loop, avoiding unnecessary power consumption and visual pollution caused by redundant operation.

[0063] The steps of driving the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates also include: extracting the angle difference between the optical axis of the main projection module and the optical axis of the auxiliary projection module when they are incident on the ground of the target projection area, and calling the global ground reflectance map obtained by the environment perception module; based on the angle difference and the global ground reflectance map, performing inverse dispersion pre-compensation in RGB color space on the complementary pixels output by the auxiliary projection module to eliminate chromatic parallax generated by multi-angle projection across hardware at the physical seam.

[0064] Specifically, after completing geometric pre-distortion, considering that when a vehicle is parked at the edge of a curb, there is a significant difference in the incident angle of the optical axes projected onto the sloping ground by the main projection module (e.g., the rearview mirror end) and the auxiliary projection module (e.g., the door sill end), the system extracts the angle difference between these two incident optical axes on the sloping ground in real time. It then uses multispectral cameras to capture ground reflectance images in eight narrowband bands (center wavelengths of 400nm, 440nm, 480nm, 520nm, 560nm, 600nm, 640nm, and 680nm, with a bandwidth of approximately 40nm). These images are combined with the spatial coordinates of the lidar point cloud for texture mapping, generating a global ground spectral reflectance map that corresponds one-to-one with the pixel grid of the target projection area. The spectral data is stored as an 8-channel floating-point texture per pixel, with each channel value representing the directional-hemispherical reflectance of the pixel in the corresponding wavelength band (range [0, 1]). The spatial resolution is consistent with the target projection area. During RGB dispersion pre-compensation, the reflectance values ​​at the three wavelengths R (620nm), G (532nm), and B (470nm) are reconstructed from the 8-channel spectral data using cubic spline interpolation.

[0065] Based on this optical environment data, the controller performs inverse dispersion pre-compensation in the RGB color space for the complementary pixels to be output by the auxiliary projection module in the video memory rendering pipeline. Due to the inherent optical characteristics of physical lenses, light beams at different incident angles exhibit a wavelength-dependent pupil-apod effect after passing through the projection lens; simultaneously, the micro-facets of the rough road surface have angle-selective reflection / scattering characteristics for broadband light at different incident angles (based on the Oren-Nayar rough surface diffuse reflection model, the standard deviation σ of the micro-facet normal distribution determines the spectral reflectance attenuation mode at different incident angles). The superposition of these two effects results in a color deviation in the stitching area when the main projection module and the auxiliary projection module project onto the same road surface area at different incident angles along their optical axes, which is related to the difference in incident angles. The system extracts the difference in the angle between these two incident angles on the inclined ground in real time and calls the global ground spectral reflectance map generated by the multispectral camera. Based on the aforementioned optical environment data and the CIE standard colorimetric observer color matching function, the controller performs a 3×3 color correction matrix transformation in the XYZ color space on the complementary pixels that the auxiliary projection module is about to output in the video memory rendering pipeline to compensate for angle-dependent color deviations.

[0066] The specific algorithm for the RGB color space inverse dispersion pre-compensation is as follows: Let the incident angles on the ground of the optical axis of the main projection module and the optical axis of the auxiliary projection module be respectively... and The difference in their included angles is Based on the Torrance-Sparrow model of light scattering on rough surfaces, light of different wavelengths λ at the angle of incidence... Single scattering reflectance satisfy ,in This is the geometric attenuation factor related to the normal distribution of the road surface micro-element. Based on this physical relationship, a 3×3 dispersion compensation matrix is ​​constructed for the system. Its diagonal elements , , These correspond to the compensation coefficients for the R (620nm), G (532nm), and B (470nm) channels, respectively. The formulas for calculating the compensation coefficients for each channel are as follows: in The angle of incidence for the main projection module (as a color reference). The incident angle (on the compensated side) of the auxiliary projection module. The center wavelength corresponds to the channel. The global ground reflectance map is acquired by a multispectral camera in eight narrow bands (intervals of approximately 40 nm, covering the visible light range from 400 nm to 700 nm), and after spatial registration, it is stored as a reflectance grid with 8 channels per pixel. The spatial resolution is consistent with the pixel grid of the target projection area. The off-diagonal elements of the compensation matrix are initialized to zero (assuming that the spectral crosstalk of the rough road surface is negligible). In the fragment shader of the auxiliary projection module, the RGB values ​​of the pixel to be output are multiplied with the compensation matrix to output the pre-compensated complementary pixel.

[0067] This invention solves the spectral dispersion and color shift defects that inevitably occur on complex physical surfaces due to differences in incident angles by extracting the angle difference across the hardware optical axis and performing reverse dispersion pre-compensation in the RGB color space. It elevates the consistency to the level of the underlying color, eliminates red and blue artifacts and color difference breaks that are easy to occur at the splicing seams, and enhances the optical purity and visual fusion quality of the spliced ​​image.

[0068] Further, the step of outputting seamlessly stitched relay projection frames includes: extracting the spatial overlap region of the output field-of-view boundary coordinates of the main projection module and the auxiliary projection module, defining the spatial overlap region as an overlap space set, and defining its maximum width along the gradient direction as the maximum width of the transition zone. The maximum width of the transition zone depends on the field-of-view overlap range of the main projection module and the auxiliary projection module, and is dynamically and automatically measured by the system during the calibration phase by projecting a full-white checkerboard pattern and acquiring images of the stitching area. Its typical value range is limited to 5% to 15% of the diagonal length of the main projection area. A spatial step mapping relationship is established within the overlap space set, and a normalized distance variable is defined. , which represents the relative distance of a pixel within the overlapping area along the direction perpendicular to the seam to the boundary of the exclusive area of ​​the main projection module.

[0069] The specific form of the pixel transparency gradient function is constructed based on the logistic nonlinear variation curve. Specifically, the current pixel transparency of the main projection module within the overlap space set is calculated in the video memory rendering pipeline. Its specific mathematical logic is as follows: Where k=10 is the kurtosis parameter of the Sigmoid function, ensuring that the transparency completes the main transition in the central region of d∈[0.3,0.7], and at d=0... (Fully loaded), at d=1 (Completely transparent).

[0070] Simultaneously, in order to satisfy the energy conservation constraint of seamless optical superposition at any pixel location in space, the pixel transparency of the auxiliary projection module at the same pixel point is strictly controlled to be equal to... The physical basis for using a logistic nonlinear curve instead of a traditional linear transition is that the human eye's perception of changes in the brightness of an optical image strictly follows the Weber-Fechner logarithmic law. The progressively smooth characteristics of the logistic function at both ends of the transition band can eliminate visually perceptible overexposure white spots, hard edge boundaries, and hard splicing cracks from a physiological visual perspective. Through the dynamic superposition and complementarity of transparency within the region, a seamless splicing relay projection frame with uniform illumination and smooth visual transition is generated.

[0071] Specifically, in the final execution stage where the main and auxiliary projection modules jointly output the light field, their fields of view inevitably overlap on the ground. The system extracts the spatial overlap region of the field of view boundaries of the main and auxiliary projection modules in the global ground coordinate system and sets a specific pixel transparency gradient function in the corresponding digital image layer of this region. To avoid overexposure white spots caused by brightness superposition in the overlapping area, the system controller controls the pixel channel transparency of the main projection module in this overlapping area according to this gradient function, smoothly decreasing the gradient rate from full load to zero along the direction pointing to the blind pixel. At the same time, the system synchronously controls the auxiliary projection module through the internal clock bus, so that the pixel channel transparency in the same area smoothly increases from zero to full load value. Through this precise "one decrease, one increase" synchronous complementary control at the underlying driving level, the light flux projected by different hardware undergoes optical superposition and attenuation neutralization on the rugged ground.

[0072] While performing the aforementioned digital layer transparency blending, to address the indirect glare interference caused by ground diffuse reflection on pedestrians, the system controller invokes the environmental perception module to obtain the current local background ambient illuminance. Combining the nonlinear response characteristics of human visual perception (i.e., the logarithmic response where the human eye is sensitive to changes in brightness in dark areas but insensitive to changes in brightness in bright areas), the controller dynamically reduces the global pulse width modulation duty cycle of the light source at the back end of the digital micromirror device in the auxiliary projection module at the underlying electrical drive end. The system sets a visual contrast recognition threshold based on the local background ambient illuminance. This threshold is calculated based on the proportion of the minimum perceptible difference for the human eye under the current ambient light, according to the Weber-Fechner law, to ensure that the visual contrast between the projected pixels and the ground background is not lower than this recognition threshold while performing the duty cycle voltage drop. According to the Talbot-Platau law, a decrease in duty cycle will cause a decrease in physical average light intensity; however, in low-light or complex parking environments, thanks to the nonlinear perception compensation of human vision, the decrease in visual brightness perceived by the user is limited, and the pattern remains clearly discernible. By combining the superposition and complementation of transparency gradients with the duty cycle voltage drop of the underlying luminous flux, a seamless splicing relay projection frame with uniform illumination, smooth transition, and low diffuse reflection characteristics is finally output.

[0073] The specific calculation logic for the dynamic adjustment of the pulse width modulation duty cycle is as follows: The system extracts the ground diffuse reflection coefficient and background ambient illuminance of the target projection area, multiplies them, and divides the product by pi to convert them into background radiant brightness from the human eye's perspective. Simultaneously, it calls the factory-set full-load center brightness of the auxiliary projection module and the system-set minimum visual contrast threshold. To prevent the projection from becoming invisible due to dual attenuation in the digital and physical domains, the system extracts the minimum transparency minimum value of complementary pixels within the current frame as a physical constraint to calculate the global pulse width modulation duty cycle of the auxiliary projection module in the current frame. The specific calculation rules are as follows: First, the digital value is added to the minimum visual contrast threshold, and the sum is multiplied by the background radiant brightness to calculate the target display brightness requirement value; second, the minimum transparency minimum value is multiplied by the factory-set full-load center brightness to calculate the physical output brightness reference value; next, the target display brightness requirement value is divided by the physical output brightness reference value to calculate the theoretical duty cycle; finally, the theoretical duty cycle is compared with the system-preset lower limit of the duty cycle, and the larger of the two is selected as the final determined global pulse width modulation duty cycle. The preset duty cycle lower limit (e.g., a value of 0.1) is designed to prevent hardware color temperature drift caused by extremely low duty cycles. Furthermore, if the global pulse width modulation duty cycle calculated by the system is greater than the number one, the controller will forcibly truncate it and set the value to the number one, that is, abandon the duty cycle voltage drop operation in the physical domain, thereby ensuring the basic visibility of the image.

[0074] This invention solves the problem of brightness spikes and hard-edge splicing cracks that easily occur at overlapping edges in multi-source projection by setting a pixel transparency gradient function in the overlapping area of ​​the distributed projection module output, and implementing a synchronous control strategy of the transparency of the main and auxiliary light sources increasing and decreasing in opposite directions. By utilizing the superposition and complementary effect of physical light flux, the invention ensures the uniformity of the overall brightness of the relay projection image and the smoothness of the visual transition.

[0075] Furthermore, this invention combines the nonlinear response characteristics of local background ambient illumination and human visual perception. By dynamically reducing the pulse width modulation duty cycle of the underlying light source while ensuring a minimum visual recognition contrast threshold, the system utilizes the nonlinear difference between the attenuation amount perceived by human vision and the attenuation amount of objective physical radiation. This achieves a significant reduction in the total physical photon radiation with a small compromise in visual brightness. This mechanism effectively reduces indirect glare interference caused by diffuse reflection of high-intensity projection beams on complex physical ground without increasing additional optical hardware costs, and optimizes the thermal power consumption of the vehicle-mounted projection module, thereby improving the optical safety of the welcome projection system in public transportation environments.

[0076] Example 2: This embodiment provides an identity recognition-based vehicle-mounted welcome projection control device. Physically, the device relies on an onboard central computing platform or a separate domain controller for execution. It interacts with an external environmental perception module and multiple distributed projection modules (2 to 8 modules) via a high-speed onboard communication bus (e.g., Ethernet or CAN-FD bus) for low-latency data exchange. Logically, the device is divided into a strategy generation module, a dynamic masking inference module, and a pixel relay module. These modules work collaboratively according to strict timing logic to complete the entire physical control process, from environmental perception, computing power scheduling, anti-occlusion calculation to multi-source optical complementary stitching.

[0077] As one embodiment of the present invention, refer to Figure 2 A schematic diagram of the structure of an in-vehicle welcome projection control device based on identity recognition.

[0078] The strategy generation module, acting as the system's front-end data aggregation and scheduling hub, is responsible for initializing the projection interaction task. During system operation, the strategy generation module continuously monitors the data stream from the environment perception module. When an external target entity is detected approaching, the strategy generation module extracts the user's feature information and sends it to a pre-stored database in the secure operating environment for traversal matching. Upon successful matching, the module not only determines the user's identity but also simultaneously outputs a confirmation weight associated with the current matching confidence level. Simultaneously, the strategy generation module parses the user's real-time 3D spatial coordinates returned by the environment perception module, thereby defining the expected landing area of ​​the welcoming image in the global ground physical coordinate system and establishing the target projection area. After obtaining the identity confirmation weight and the target projection area, the strategy generation module initiates the underlying resource scheduling logic to determine the rendering strategy for the target projection area. Since the total number of stream processor cores in the system's underlying graphics processing unit has a physical limit, the strategy generation module dynamically determines the allocation ratio of computing resources based on the value of the confirmation weight. Targets with higher weights will receive a larger proportion of computing resources for high-precision image rendering pipelines, while targets with lower weights will have their computing resources allocated more efficiently. This enables logical slicing and non-linear skew scheduling of underlying hardware resources in scenarios where multiple targets are in close proximity.

[0079] Furthermore, the dynamic masking simulation module receives the computing power quota instructions and target projection area coordinates output by the strategy generation module, and is responsible for making advanced predictions of physical optical path collisions in the digital space. This module first calls the factory-set optical center coordinates of each distributed projection module recorded in the vehicle's underlying non-volatile memory, and simultaneously reads the real-time 3D contour data of external physical obstacles constructed by the environmental perception module. Using the grid division vertices of the target projection area, the optical center coordinates of the projection modules, and the obstacle contours as fixed input parameters, the dynamic masking simulation module calls the allocated stream processor cores in the graphics processing unit to execute high-concurrency ray tracing calculations.

[0080] To ensure real-time performance in the automotive embedded environment, the system employs the following optimization strategies: The initial calculation of the dynamic shadow masking matrix is ​​performed at a reduced resolution (512×512 pixels), followed by bilinear interpolation upsampling to the target projection area's resolution (1024×1024 pixels), reducing the total number of rays from approximately 1 million to approximately 260,000. The 3D envelope mesh of obstacles uses an adaptive simplification algorithm—merging obstacle faces more than 3 meters from the projection module into larger triangular faces (maximum area of ​​0.1m² per face), reducing the number of leaf nodes in the BVH tree. When the number or complexity of obstacles exceeds a preset limit, the system degrades to a motion compensation prediction mode based on the previous frame's masking matrix, skipping the ray tracing calculation for the current frame until the next system cycle resumes.

[0081] During the calculation process, the module simulates the physical path of the light beam emitted from the optical center to the target projection area, and calculates in real time whether spatial geometric cross-section collisions occur between these virtual light beam direction vectors and the 3D envelope mesh surface of the physical obstacles. For mesh coordinates where cross-section collisions occur, the module writes occlusion assignments to the video memory buffer; for transparent mesh coordinates where cross-section collisions do not occur, it writes transparent assignments. After all mesh traversal calculations are completed, the dynamic masking deduction module encapsulates the above assignments into a matrix according to the spatial arrangement order of the 2D pixel array, and outputs a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module. This matrix provides the system with the underlying optical path interference index under the current physical topology environment.

[0082] Subsequently, the pixel relay module intervenes in the pipeline, responsible for cross-hardware spatiotemporal registration and the final physical light field driving output. After acquiring the dynamic shadow masking matrix, the pixel relay module first calls the 3D terrain point cloud data collected in real time by the environment perception module to evaluate the flatness of the target projection area, so as to adaptively configure the spatial coordinate mapping model.

[0083] When the region is determined to be a standard horizontal topological plane, the pixel relay module configures the spatial coordinate mapping model as a homography transformation parameter matrix, combines the spatial pose parameters and local perspective transformation parameters for calculation, maps the pixels carrying occlusion assignments in the masking matrix to obtain absolute physical coordinates, and inputs them into the two-dimensional field of view boundary range function of the auxiliary projection module for verification. Then, the inverse spatial transformation matrix is ​​used to generate registration coordinates for cross-device driving.

[0084] When a non-standard elevation 3D topological abrupt change is detected in the region, the pixel relay module configures the spatial coordinate mapping model as a spatial reprojection model. This module reads the calibration intrinsic and extrinsic parameter matrix of the main projection module and constructs a global 3D topological mesh for the target projection region in video memory. The pixel relay module projects the pixels carrying occlusion assignments from the masking matrix onto the surface of the global 3D topological mesh using line-of-sight ray tracing, calculates the intersection points to obtain their absolute 3D physical coordinates, and marks them as blind zone pixels. After locking the absolute 3D coordinates of the blind zone pixels, the pixel relay module uses a set boundary judgment algorithm to verify whether the absolute 3D physical coordinates are within the 3D field of view envelope boundaries of the other unoccluded auxiliary projection modules. Upon successful verification, the module calls the calibration intrinsic and extrinsic parameter matrix of the auxiliary projection modules to perform a strict inverse perspective projection transformation on the absolute 3D physical coordinates, mapping their coordinates inversely to the digital micromirror array plane at the bottom layer of the auxiliary projection modules, generating registration coordinates for cross-device driving.

[0085] During the final optical output execution phase, the controller within the pixel relay module, based on the aforementioned generated rendering strategy, issues a low-level truncation command to the main projection module. This forcibly shuts down the output channels of the corresponding blind pixel positions of the main projection module, preventing interference caused by invalid light beams illuminating obstacles from the physical light flux source. Simultaneously, the pixel relay module drives the auxiliary projection module to call the source image data to be compensated at the generated registration coordinates and performs geometric pre-distortion processing and complementary pixel rendering in conjunction with ground elevation parameters. By precisely superimposing the physical light flux of the main and auxiliary projection modules within the overlapping field of view of multiple devices, the pixel relay module ultimately outputs a seamlessly stitched relay projection frame to the external environment in terms of spatial geometric connectivity and brightness performance. All the above modules continuously execute actions at the main loop update frequency set by the system, ensuring robust output of the vehicle-mounted welcome projection in complex 3D topological environments.

[0086] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A vehicle-mounted welcome projection control method based on identity recognition, applied to a vehicle-mounted system comprising multiple distributed projection modules and an environmental perception module, characterized in that, include: Obtain user feature information and match it with the historical user feature database to determine user identity and corresponding confirmation weight; The target projection area is determined based on the user's real-time 3D spatial coordinates outside the vehicle; the rendering strategy for the target projection area is determined based on the confirmed weights and the target projection area. Using the target projection area, the optical center coordinates of each distributed projection module, and the physical obstacle contours obtained by the environment perception module as input parameters, the beam collision cross section is calculated in real time to generate a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module. The dynamic shadow masking matrix is ​​mapped to the global ground coordinate system using a spatial coordinate mapping model. The blind pixels in the target projection area are identified, and the registration coordinates of the blind pixels in the projection range of the other unmasked distributed projection modules are located. The controller shuts down the output of the corresponding blind pixels of the main projection module according to the rendering strategy, and drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, and outputs seamlessly stitched relay projection frames.

2. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, The steps for determining the user identity identifier and corresponding confirmation weight include: acquiring spatial distance parameters using an ultra-wideband radar node and acquiring three-dimensional visual parameters using a multispectral camera; outputting a multimodal fusion feature matrix after temporal alignment of the spatial distance parameters and the three-dimensional visual parameters; comparing the multimodal fusion feature matrix with a pre-stored historical user feature database, calculating the Euclidean distance of each entry, and if the entry with the smallest Euclidean distance value is lower than a preset security identification threshold, then the entry is taken as the matching target, and the user identity identifier associated with the matching target is output; a preset non-zero smoothing bias constant is superimposed on the Euclidean distance value corresponding to the matching target, the reciprocal of the superimposed value is calculated and normalized, and the resulting value is assigned as the confirmation weight to the user identity identifier.

3. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, The steps for determining a rendering strategy for a target projection region based on the confirmed weight and the target projection region include: extracting the pixel grid size parameter of the target projection region; calculating the difference between the confirmed weight and a preset computing power allocation benchmark threshold; inputting the difference and the pixel grid size parameter into a preset multivariate mapping function to obtain a dynamic computing power allocation coefficient that includes a basic weight coefficient and a spatial scale compensation coefficient; generating a rendering strategy that includes the dynamic computing power allocation coefficient, and sending it to the graphics processing unit so that the graphics processing unit establishes the call ratio of the stream processor cores according to the dynamic computing power allocation coefficient.

4. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, The steps for generating a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module include: establishing a three-dimensional Cartesian environmental coordinate system with the vehicle's physical center as the origin; in the three-dimensional Cartesian environmental coordinate system, using the optical center coordinates of each distributed projection module as the emission starting point, emitting virtual ray direction vectors to each grid division vertex of the target projection area; if the virtual ray direction vector has a spatial geometric intersection with the three-dimensional envelope grid surface of the physical obstacle contour, then writing an occlusion assignment at the pixel coordinate position corresponding to the landing point of the virtual ray direction vector in the target projection area; if there is no spatial geometric intersection, then writing a pass-through assignment at the pixel coordinate position corresponding to the landing point of the virtual ray direction vector in the target projection area; and matrix-encapsulating the pixel coordinate positions containing the corresponding assignments to construct an output two-dimensional binary matrix as the dynamic shadow masking matrix.

5. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, When the target projection area is determined to be a standard horizontal plane, the spatial coordinate mapping model is a homography transformation parameter matrix; The steps for identifying blind pixels within the target projection area and locating registration coordinates include: combining the spatial pose parameters calculated in real time by the environmental perception module and the local perspective transformation parameters of the main projection module to calculate the homography transformation parameter matrix; The pixels carrying occlusion assignments in the dynamic shadow masking matrix are multiplied with the homography transformation parameter matrix to obtain the absolute physical coordinates in the global ground coordinate system and are marked as blind zone pixels. If the absolute physical coordinates are within the two-dimensional field of view of the auxiliary projection module, the absolute physical coordinates are reverse-transformed to the digital micromirror array plane of the auxiliary projection module using the inverse spatial transformation matrix to generate registration coordinates.

6. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, When it is determined that there is a three-dimensional topological abrupt change with non-standard elevation in the target projection area, the spatial coordinate mapping model is a spatial reprojection model. The steps for identifying blind pixels within the target projection area and locating registration coordinates include: calling the 3D terrain point cloud data containing relative elevation parameters collected by the environmental perception module to construct a global 3D topological mesh for the target projection area, and constructing a spatial reprojection model in conjunction with the calibration intrinsic and extrinsic parameter matrix; importing the dynamic shadow masking matrix as an input parameter into the homography transformation parameter matrix, and projecting the pixels carrying occlusion assignments in the dynamic shadow masking matrix onto the surface of the global 3D topological mesh using line-of-sight ray tracing according to the calibration intrinsic and extrinsic parameter matrix of the main projection module; calculating the intersection of the ray and the surface of the global 3D topological mesh, obtaining the absolute 3D physical coordinates of the pixel in the global ground coordinate system, and marking it as a blind pixel; if the absolute 3D physical coordinates are within the 3D field of view envelope boundary of the auxiliary projection module, performing an inverse perspective projection transformation using the calibration intrinsic and extrinsic parameter matrix to transform the absolute 3D physical coordinates to the digital micromirror array plane, generating registration coordinates.

7. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, The steps of driving the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates include: calling the three-dimensional terrain point cloud data and extracting the ground tilt angle and relative elevation difference of the area covered by the corresponding registration coordinates; calculating the distortion correction matrix based on the ground tilt angle and relative elevation difference; using the distortion correction matrix to perform reverse spatial stretching and pixel coordinate offset processing on the source image to be rendered at the registration coordinates to complete the geometric pre-distortion; driving the auxiliary projection module to project the pre-distorted complementary pixels, and synchronously receiving the updated registration coordinates and distortion correction matrix at a set frame rate, and continuously refreshing the output frame by frame; terminating the projection process when a vehicle gate opening signal is received and / or the user coordinates deviate from the preset boundary.

8. The vehicle-mounted welcome projection control method based on identity recognition according to claim 1, characterized in that, The steps for outputting seamlessly stitched relay projection frames include: extracting the spatial overlap region of the output field-of-view boundary coordinates of the main projection module and the auxiliary projection module; setting a pixel transparency gradient function in the digital image layer of the spatial overlap region; controlling the pixel transparency of the main projection module in the spatial overlap region to decrease from full load to zero towards the blind pixel direction according to the pixel transparency gradient function, while simultaneously controlling the pixel transparency of the auxiliary projection module to increase from zero to full load, and generating seamlessly stitched relay projection frames through the superposition and complementarity of transparency within the region.

9. A vehicle-mounted welcome projection control device based on identity recognition, characterized in that, include: The strategy generation module is used to obtain user feature information through the environment perception module and match it with the historical user feature database to determine the user's identity and corresponding confirmation weight. The target projection area is determined based on the user's real-time 3D spatial coordinates outside the vehicle; the rendering strategy for the target projection area is determined based on the confirmed weights and the target projection area. The dynamic masking simulation module is used to take the target projection area, the optical center coordinates of each distributed projection module, and the physical obstacle contours obtained by the environment perception module as input parameters, and use the ray tracing algorithm to calculate the beam collision section in real time to generate a dynamic shadow masking matrix corresponding to the projection field of view of each distributed projection module. The pixel relay module is used to map the dynamic shadow masking matrix to the global ground coordinate system using a spatial coordinate mapping model, identify blind pixels in the target projection area, and locate the registration coordinates of the blind pixels within the projection range of the other unmasked distributed projection modules. The controller shuts down the output of the corresponding blind pixels of the main projection module according to the rendering strategy, and drives the auxiliary projection module to perform geometric pre-distortion and complementary pixel rendering at the registration coordinates, outputting seamlessly stitched relay projection frames.

Images