ISD matrix lamp character user-defined editing method based on vehicle multi-terminal linkage

Abstract

The invention relates to the technical field of vehicle intelligent lighting and man-machine interaction, and discloses an ISD matrix lamp character self-defined editing method based on vehicle multi-terminal linkage, which comprises the following steps: firstly, establishing heterogeneous spatial association between a mobile terminal and a vehicle terminal, resolving a real-time pose through feature matching and a perspective point positioning algorithm, and finally, establishing a real-time pose; the method comprises the following steps: acquiring an initial visual intention of a user on an augmented reality interface, dynamically generating a target observation viewpoint in combination with a real-time semantic state of a vehicle and perception data, constructing a homography matrix by utilizing a spatial difference between the target viewpoint and an initial viewpoint, and performing perspective correction on an image to obtain an image; and establishing a geometric mapping relationship between the corrected image and a vehicle lamp curved surface grid through a reverse ray casting technology, and calculating a target brightness value of the physical light-emitting unit by using a discrete topology energy jitter algorithm to drive display. According to the method, the WYSIWYG cross-terminal interaction is realized, the problems of curved surface display distortion and multi-view angle inconsistency are solved, and the display quality and safety in a dynamic traffic scene are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle intelligent lighting and human-computer interaction technology, specifically to a method for custom editing ISD matrix light characters based on multi-terminal linkage of vehicle systems. Background Technology

[0002] With the rapid development of automotive intelligent technology, vehicle exterior lighting systems have evolved from simple nighttime illumination tools into intelligent signal display (ISD) systems that carry information transmission and emotional interaction functions. Matrix LED headlights and taillight assemblies, with their high brightness and programmability, enable drivers to interact instantly with surrounding road users through customized characters or patterns, which has broad application prospects in autonomous driving assistance and personalized expression.

[0003] Existing custom editing solutions for ISD matrix lights typically employ a two-dimensional planar mapping method based on mobile terminal applications. This involves the user drawing pixel patterns on a rectangular canvas on their phone screen, and the system directly mapping these patterns one-to-one onto the LED dot matrix of the vehicle's lights. However, this traditional planar-to-planar mapping mechanism has significant technical limitations in practical applications. Because modern vehicle exterior designs generally employ streamlined, high-curvature, complex surfaces, forcibly mapping two-dimensional image data onto the three-dimensional curved surface of the lights inevitably results in severe geometric distortion. This causes the displayed characters to be stretched and deformed, compromising the visual aesthetics and readability of the user's original design. Furthermore, existing editing methods sever the spatial pose connection between the mobile terminal and the actual vehicle. Users cannot perceive the actual spatial projection effect while editing on the screen and often need to repeatedly try and adjust to achieve the desired display form.

[0004] Furthermore, traditional static mapping methods do not consider the differences in perspective and semantic requirements in dynamic traffic scenarios. Observers of vehicle light signals (such as pedestrians or vehicles approaching from behind) are often located in a specific position relative to the vehicle, rather than always directly behind or in front. When the observer is at a large angle away from the light's normal, or when the vehicle is in a dynamic driving state such as turning, the fixed projection pattern will undergo visual distortion due to perspective effects, making it difficult to accurately identify warning information and reducing the safety and effectiveness of information interaction. At the same time, due to limitations in physical hardware costs and heat dissipation requirements, the pixel density of vehicle matrix lights is much lower than that of conventional display screens. When directly displaying complex curves or diagonal lines, obvious jagged edges and uneven energy distribution will appear, seriously affecting the smoothness and detail of the display. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicles, which solves the problem of visual information transmission distortion caused by the single viewing angle, curved display distortion, and physical LED resolution limitations in existing in-vehicle matrix light interaction systems.

[0006] To achieve the above objectives, the present invention provides a method for custom editing ISD matrix light characters based on multi-terminal linkage of vehicle and vehicle systems. This method achieves full-link closed-loop control from user intent input, dynamic semantic viewpoint migration, surface geometry inverse correction to discrete light energy driving by constructing a heterogeneous spatial association between the mobile terminal and the vehicle.

[0007] Specifically, the method of this invention first establishes a heterogeneous spatial association between the mobile terminal and the vehicle. The vehicle-side computing platform loads a curved 3D mesh model of the vehicle's ISD matrix light components and extracts key feature point data, which is then transmitted to the mobile terminal. The mobile terminal acquires external video streams from the vehicle, extracts 2D image feature points using image processing algorithms, and matches them with the received 3D key feature point data. Based on the matched set of corresponding points, the system utilizes perspective... The point localization algorithm constructs a reprojection error function and obtains the rotation matrix and translation vector of the mobile terminal's camera coordinate system relative to the vehicle's world coordinate system through nonlinear optimization. This process establishes the pose relationship between the mobile terminal and the physical vehicle in a unified physical space, laying the geometric foundation for subsequent augmented reality interaction.

[0008] Building upon this, the method acquires the initial visual intent based on the observation viewpoint. The system utilizes the augmented reality interface of the mobile terminal, allowing users to draw or input two-dimensional source image data onto an intuitive vehicle image. The system records the real-time spatial position of the mobile terminal at the moment of input, defining it as the initial observation viewpoint. This step logically maps the user's two-dimensional operations on the flat screen to a specific perspective intent in three-dimensional space, achieving a WYSIWYG editing experience.

[0009] To adapt to complex traffic interaction scenarios, this method further analyzes the vehicle's semantic state and generates a target viewpoint. The vehicle-side aggregates steering angle, wheel speed, and gear data from the vehicle bus in real time, along with the relative azimuth coordinates of the target object from blind spot monitoring radar and environmental perception cameras, to construct a real-time semantic state vector. The system determines whether this vector meets preset safety warning trigger conditions. If so, the system constructs a gaze-tracking vector based on the target object's (e.g., a pedestrian or a vehicle approaching from behind) azimuth data, and extends this vector along a preset distance from the vehicle's reference center point to calculate the target observation viewpoint. This mechanism ensures that the information delivery direction is no longer limited to directly behind the vehicle, but can dynamically point to traffic participants who need to receive warning information based on the actual traffic situation.

[0010] To address perspective distortion caused by changes in viewpoint, this method performs dynamic perspective correction based on visual intent. The system constructs the original projection plane and the target projection plane according to the positional differences between the initial and target viewpoints in 3D space, and calculates the homography matrix between them. Using this homography matrix, the system performs inverse perspective mapping calculations on the pixel coordinates of the 2D source image data. This process ensures that even with significant changes in viewing angle, the geometric features of the image data, when projected onto the target viewpoint, can still be restored to the shape initially designed by the user.

[0011] To overcome the imaging distortion caused by the curved surface of the headlights, this method performs surface-backward ray projection and texture coordinate picking. The system uses the target viewing point as the projection origin and emits virtual rays in the spatial direction corresponding to each pixel unit of the corrected image data, performing intersection tests with the curved 3D mesh model of the vehicle's ISD matrix headlight assembly. When the virtual ray intersects with a triangular facet, the system obtains the 3D coordinates of the intersection point in the vehicle's world coordinate system and establishes a mapping index between it and the image pixels. To accurately obtain texture information, the system determines the vertices of the triangular facet containing the intersection point and their texture coordinates, calculates the centroid coordinate weight of the intersection point relative to the vertices, and then obtains the mapped texture coordinates of the intersection point on the headlight's unfolded plane through linear interpolation. Furthermore, based on the angle between the surface normal vector at the intersection point and the incident ray, the system queries a preset optical attenuation compensation curve to generate a brightness gain coefficient, performing brightness compensation for large-angle incident areas to correct optical attenuation caused by the Fresnel effect.

[0012] Finally, to address the sawtooth effect caused by the discrete distribution of the LED array, this method implements discrete topological energy jitter and brightness driving. The system determines the center coordinates of each physical light-emitting unit in the texture coordinate system and sets the topological neighborhood range. By retrieving all projection points falling within this neighborhood, the system uses radial basis functions to calculate the Euclidean distance weights of each projection point relative to the center of the light-emitting unit, and then weights and accumulates the corresponding image brightness values ​​to obtain the target brightness value of the physical light-emitting unit. In the display driving stage, the system performs Gamma correction on the target brightness value to generate a digital grayscale sequence, and converts it into duty cycle data of a pulse width modulation signal, thereby controlling the conduction time of the LED beads and achieving a sub-pixel-level optically smooth display effect.

[0013] This invention provides a method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle systems. It offers the following advantages: This invention establishes a heterogeneous spatial association between a mobile terminal and a vehicle, and utilizes perspective... The point positioning algorithm accurately calculates the relative pose, enabling intuitive WYSIWYG interaction based on augmented reality. Unlike traditional planar mapping methods, this solution uses the PnP algorithm to register the two-dimensional image data input by the user on the mobile terminal to the vehicle's three-dimensional coordinate system in real time. This eliminates the spatial parallax between the editing perspective and the actual physical display plane, ensuring the accurate reproduction of the user's design intent on the physical vehicle body and significantly improving the accuracy and convenience of cross-terminal interaction.

[0014] This invention analyzes the semantic state of a vehicle and generates a target viewpoint. It can dynamically calculate the optimal warning direction based on blind spot monitoring radar and vehicle driving status data, and use homography transformation to perform viewpoint-adaptive perspective correction on the source image data. This mechanism breaks the limitation of traditional vehicle displays that can only face a fixed viewpoint directly behind. It can project pre-distorted and corrected images to traffic participants in specific positions (such as pedestrians in the side and rear blind spots), ensuring that even if the observer is at a large angle away from the normal of the lamp, they can still receive key warning information whose geometric features have not been distorted, thus improving active safety in dynamic traffic scenarios.

[0015] This invention effectively overcomes the imaging distortion caused by the irregular curved surface shape of the ISD matrix lamp component and the display jagged effect caused by the low resolution dot matrix distribution by executing the curved surface reverse ray projection and discrete topology energy dithering algorithm. It establishes a precise geometric mapping relationship by using the intersection operation of virtual rays and curved surface mesh model, and combines radial basis functions to perform sub-pixel level brightness weighted allocation of discrete physical light-emitting units. Under the hardware conditions of limited physical lamp density, it achieves a high-quality display effect with smooth edges, uniform brightness and perfect fit to the curved surface of the car body. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the viewpoint migration logic flow based on vehicle semantics of the present invention. Detailed Implementation

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This invention includes a customizable character editing system for ISD matrix lights based on multi-terminal linkage within a vehicle. The system comprises a mobile terminal, a vehicle-side computing platform, a vehicle status perception component, and an ISD matrix light component. The mobile terminal and the vehicle-side computing platform establish a bidirectional data transmission link via a wireless communication protocol.

[0019] The mobile terminal serves as the system's spatial perception probe and human-computer interaction interface. The mobile terminal includes an image acquisition module, a display and interaction module, a first processing module, and a first communication module.

[0020] The image acquisition module is used to acquire real-time video streams or still image frames of the vehicle's external environment. The real-time video streams or still image frames contain visual features of the vehicle's ISD lighting area. The image acquisition module includes, but is not limited to, CMOS or CCD image sensors.

[0021] The display interaction module is used to overlay a virtual augmented reality layer on the screen of the mobile terminal and to receive two-dimensional source image data input by external objects onto this augmented reality layer. The two-dimensional source image data defines the visual graphic content that the external object expects to see at the viewing point.

[0022] The first processing module is connected to the image acquisition module and the display interaction module. The first processing module is used to execute the spatial feature extraction algorithm, match the feature points in the image acquired by the image acquisition module with the pre-stored feature points of the vehicle 3D model, and calculate the real-time spatial pose of the image acquisition module relative to the vehicle.

[0023] The first communication module is used to send two-dimensional source image data and real-time spatial pose data to the vehicle-side computing platform.

[0024] The vehicle status perception component is used to monitor the vehicle's driving status and the surrounding environment in real time, and outputs vehicle status signals. The vehicle status perception component includes a steering angle sensor, wheel speed sensors, blind spot monitoring radar, and an environmental perception camera.

[0025] Steering angle sensor is used to collect steering wheel angle data. Wheel speed sensor is used to collect vehicle speed data. Blind spot monitoring radar and environmental perception camera are used to detect the presence of target objects within a specific area around the vehicle and output the relative position coordinates of the target objects.

[0026] The vehicle-side computing platform serves as the core computing unit of the system, handling semantic logic and graphics rendering mapping. The vehicle-side computing platform includes a second communication module, a semantic state parsing module, a viewpoint migration calculation module, and a topology mapping and rendering module.

[0027] The second communication module is used to receive data from the mobile terminal. The semantic state parsing module is connected to the vehicle state perception component and is used to receive vehicle state signals and generate a semantic state vector containing the vehicle's dynamic intention and target orientation according to preset logical rules.

[0028] The viewpoint migration calculation module is connected to the semantic state parsing module. The viewpoint migration calculation module calculates the target observation viewpoint position relative to the vehicle coordinate system based on the semantic state vector, and performs homography transformation on the two-dimensional source image data based on the difference between the target observation viewpoint position and the initial position of the mobile terminal, generating corrected image data.

[0029] The topology mapping rendering module stores the curved 3D mesh model data of the vehicle's ISD lighting fixtures and the LED chip distribution data. This module performs inverse ray casting calculations to establish a geometric mapping relationship between the corrected image data and the curved 3D mesh model, and calculates the corresponding LED chip brightness driving data based on the mapping results.

[0030] The ISD matrix lamp assembly includes an LED driver controller and an LED array arranged in a curved surface. The LED driver controller is connected to a topology mapping and rendering module to receive brightness driving data and convert it into a pulse width modulation signal. The LED array consists of multiple discretely distributed LED light-emitting units, which emit light under the drive of the pulse width modulation signal.

[0031] In this embodiment, the system defines multiple reference coordinate systems to unify spatial description. Vehicle World Coordinate System The origin is defined as the projection of the center of the vehicle's rear axle onto the ground. The X-axis is along the vehicle's transverse direction, the Y-axis is along the vehicle's longitudinal direction, and the Z-axis is perpendicular to the ground and pointing upwards.

[0032] Camera coordinate system ( The origin is defined at the optical center of the image acquisition module of the mobile terminal, and the Z-axis is along the optical axis.

[0033] Texture coordinate system Defined on the unfolded plane of the ISD matrix lamp assembly, it is used to describe the normalized position of the LED beads on the logic plane.

[0034] The real-time spatial pose calculated by the first processing module is specifically the rotation matrix and translation vector of the camera coordinate system relative to the vehicle world coordinate system. The curved 3D mesh model in the topology mapping rendering module is defined in the vehicle world coordinate system.

[0035] See attached document Figure 1This invention provides a method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicles. This method achieves adaptive mapping from two-dimensional visual intent to three-dimensional curved surface display through deep collaboration between mobile terminals and the vehicle. The method mainly includes the following steps: Step S100: Establish heterogeneous spatial association between the mobile terminal and the vehicle terminal: During system initialization, a rigid transformation relationship is first established between the mobile terminal camera coordinate system and the vehicle's world coordinate system. The user operates the mobile terminal to visually scan the vehicle's ISD lighting area. The mobile terminal's image acquisition module acquires continuous frame images containing vehicle features and extracts robust features from the images (such as body seam corners and lighting outline edges) using a feature point detection algorithm. Simultaneously, the vehicle terminal transmits pre-stored 3D CAD mesh models and key feature point data to the mobile terminal. The first processing module uses a perspective n-point localization algorithm (PnP) to calculate the real-time six-degree-of-freedom pose (rotation matrix and translation vector) of the mobile terminal relative to the vehicle, thereby accurately aligning the virtual 3D mesh model with the physical vehicle image on the mobile terminal's screen.

[0036] Step S200: Obtain the initial visual intent based on the observation viewpoint: After spatial alignment, the system generates a virtual 2D drawing plane in the augmented reality interface of the mobile terminal. This virtual plane is constructed perpendicular to the optical axis of the current mobile terminal camera. The user draws, inputs, or edits graphic content (such as text, icons, or animation frames) on this virtual plane. At this time, the system records the spatial position of the mobile terminal when the user performs the editing operation, marking it as the initial viewing point, and defines the image content input by the user as the source image data. This source image data represents the distortion-free visual effect that the user expects to observe from a specific viewpoint.

[0037] Step S300: Parse the vehicle semantic state and generate the target viewpoint: The vehicle-side computing platform monitors vehicle operation data and environmental perception data in real time. The semantic state parsing module reads the vehicle's physical state, including steering angle, speed, and gear position, from the vehicle bus and combines it with target detection data from blind spot monitoring radar and environmental cameras to construct a semantic state vector containing driving intention and the location of surrounding traffic participants. The viewpoint shift calculation module determines whether the current display task belongs to an entertainment scenario or a safety warning scenario based on preset semantic mapping logic. If it is determined to be a safety warning scenario (e.g., turning right and detecting a pedestrian), the system automatically calculates a target viewing point that deviates from the initial viewing point (e.g., a position pointing to the right rear blind spot), which represents the optimal viewing angle for the information receiver.

[0038] Step S400: Perform dynamic perspective correction based on visual intent: Based on the calculated spatial difference between the target viewing point and the initial viewing point, the system performs a homography matrix transformation on the source image data acquired in step S200. This transformation process simulates the perspective projection changes as the virtual camera moves from its initial position to the target position. By reprojecting and resampling the source image data, corrected intermediate image data is generated. This step ensures that even if the observer (such as a passerby) is not in the position edited by the user, or if the surface of the light fixture is significantly curved, the graphic seen by the observer at the target viewing point still retains the geometric features designed by the user, eliminating perspective compression distortion caused by changes in viewing angle.

[0039] Step S500: Surface backlighting and texture coordinate picking: The system utilizes backlighting to map the corrected intermediate image data onto the three-dimensional geometric surface of the vehicle's lighting fixture. Specifically, the system emits a virtual ray from a defined target viewpoint to each pixel unit of the intermediate image data. The spatial intersection points of these virtual rays with the vehicle's ISD lighting fixture surface mesh model are calculated. For each valid spatial intersection point, the system queries the texture coordinates of that point in the lighting fixture's UV unfolded map, thereby establishing a one-to-one mapping index between the two-dimensional image pixels and the positions on the three-dimensional physical lighting fixture surface.

[0040] Step S600: Discrete Topology Energy Jitter and Brightness Drive: Since the ISD matrix lamp is a physical dot matrix composed of discretely arranged LED beads, its resolution is much lower than that of a virtual image. Therefore, the system performs sub-pixel-level energy allocation processing. The system traverses each LED bead in the physical LED array, establishing a local topological neighborhood with the physical center of that bead as the origin. The system calculates the light intensity contribution value of all projection space intersections falling within this neighborhood and introduces a Gaussian kernel function or other radial basis functions as weights to calculate the luminance component that the physical LED bead should bear. Finally, the calculated luminance component is converted into a corresponding PWM (Pulse Width Modulation) duty cycle signal to drive the ISD matrix lamp components to light up, thereby presenting a visually smooth and geometrically regular image on the low-resolution curved surface hardware.

[0041] This embodiment details how to establish a precise rigid body transformation relationship between the mobile terminal camera coordinate system and the vehicle world coordinate system, which is the foundation for subsequent augmented reality interaction and viewpoint correction.

[0042] During system initialization, the mobile terminal establishes a communication connection with the vehicle-side computing platform. The vehicle-side computing platform reads the 3D geometric data of the vehicle's exterior from its local storage unit. This data specifically includes the CAD mesh model of the ISD matrix light assembly and its surrounding body panels (such as bumpers and trunk lids). The vehicle-side computing platform further extracts a set of key feature points with high geometric saliency from this mesh model. These key feature points include, but are not limited to, the corner points of the light fixture outlines, the intersection points of body seams, and the geometric center point of the vehicle logo. The vehicle-side computing platform then sets the 3D coordinate data of these key feature points in the vehicle's world coordinate system. Send to mobile device.

[0043] The mobile terminal's image acquisition module starts working, capturing a video stream of the vehicle's external environment in real time at a preset frame rate. The mobile terminal's first processing module preprocesses each frame of acquired 2D image data, including denoising and grayscale conversion. Subsequently, the first processing module runs a feature extraction algorithm to detect and locate image feature points on the 2D image plane. To ensure matching accuracy, the system encodes the image feature points using feature descriptors (such as ORB or SIFT descriptors) and performs a feature vector space matching search with the received vehicle 3D key feature points. During this process, a Random Sample Consensus (RANSAC) strategy is used to iteratively eliminate mismatched point pairs, ultimately establishing a set containing... The 2D-3D corresponding point set of valid data.

[0044] Based on the established set of corresponding points, the first processing module performs the pose calculation step. This step aims to solve for the rotation matrix of the mobile terminal camera relative to the vehicle's world coordinate system. Translation vector The solution process follows the pinhole camera projection model, and its mathematical expression is described as follows: ; in, Represents the homogeneous coordinates of the feature points in the image pixel coordinate system; This represents the homogeneous coordinates of the corresponding feature point in the vehicle's world coordinate system. Scale factor; This is the camera's intrinsic parameter matrix, including focal length. and principal point coordinates Intrinsic parameter matrix These are inherent parameters of the mobile terminal, acquired and stored through a pre-calibration process. The first processing module employs a nonlinear optimization algorithm (such as the Levenberg-Marquardt algorithm) to minimize the reprojection error function, that is, to minimize the sum of squared Euclidean distances between the observed pixel coordinates and the calculated projected pixel coordinates in all matching point pairs, thereby obtaining the optimal pose transformation matrix. .

[0045] ; in, This represents a 1×3 zero vector. Represents scalar .

[0046] After obtaining the transformation matrix, the system unifies the two heterogeneous coordinate systems. The mobile terminal uses this matrix to virtually project the 3D mesh model of the vehicle's ISD lights onto the current camera view. The system renders this virtual mesh in real time and overlays it onto the video stream on the screen. By monitoring the degree of alignment between the virtual mesh edges and the actual physical edges of the lights captured in the video, the accuracy of the spatial calibration is verified. As the user holds the mobile terminal and moves around the vehicle, the system continuously performs the aforementioned feature tracking and pose update process, ensuring that the virtual editing plane remains anchored to the physical entity of the vehicle.

[0047] See attached document Figure 2 This embodiment details how the vehicle-side computing platform dynamically adjusts the optimal viewing point of the ISD matrix lights based on real-time vehicle driving status and environmental perception data, and accordingly performs geometric reconstruction of the source image. This process aims to solve the problem that in dynamic traffic scenarios, images projected from a fixed perspective onto curved lighting fixtures cannot clearly identify information for observers in specific locations.

[0048] The semantic state parsing module of the vehicle-side computing platform first establishes a multi-dimensional semantic state vector. This vector serves as the input benchmark for system decisions, and its data originates from the vehicle's internal CAN bus network and ADAS (Advanced Driver Assistance Systems) sensor array. Semantic State Vector The data includes, but is not limited to, the following components: vehicle lateral motion parameters (such as steering wheel angle and yaw rate), vehicle longitudinal motion parameters (such as vehicle speed, acceleration, and gear position signal), and external target perception parameters (such as relative azimuth angle, distance, and target type of targets in the blind spot). The system refreshes this vector at a preset sampling frequency (e.g., every 10 milliseconds) to ensure real-time data accuracy.

[0049] The viewpoint shift computation module is based on semantic state vectors. The system executes viewpoint decision logic. It has two preset logic branches: a static display mode and a dynamic warning mode. When the vehicle is stationary (e.g., in P gear or at zero speed), the system maintains the initial observation viewpoint recorded by the mobile terminal. The viewpoint can remain unchanged, or be reset to the standard viewpoint directly behind the vehicle. The system activates the viewpoint shift function when the vehicle enters a dynamic driving state or when specific warning conditions are triggered.

[0050] Taking the right-turn blind spot warning scenario as an example, when the semantic state vector shows that the steering wheel turns to the right by more than a preset threshold, and the blind spot radar detects a pedestrian target behind the vehicle on the right, the system calculates the distance from the initial observation viewpoint. To the target observation point The spatial offset. This offset is calculated according to the line-of-sight tracking principle, that is, setting the target's viewpoint... The spatial coordinates of the detected pedestrian head coincide with or lie on the line of sight connecting the detected head to the pedestrian's head. The mathematical expression for viewpoint migration can be described as: ; in, This serves as the vehicle's own reference center point (such as the geometric center of the taillights). The target relative orientation vector output by the sensing system. This is the distance scaling factor, used to normalize the viewpoint to the optimal projection distance plane. To prevent drastic viewpoint jumps due to sensor data jitter, the system calculates... Apply temporal smoothing filters (such as Kalman filters or moving average filters) to generate smooth viewpoint motion trajectories.

[0051] After determining the real-time target observation point Then, the system performs a homography transformation on the image plane. The system constructs two virtual planes: one based on the initial viewpoint. The original plane The other is based on the current target observation viewpoint. target plane Since both planes are defined in the same three-dimensional space, a unique homography matrix exists between them. The system utilizes this homography matrix The system performs inverse perspective mapping on the user-input 2D source image data. Specifically, for each pixel in the source image, the system calculates its perspective mapping path. The transformed coordinates on the target plane. This process essentially involves pre-distorting the image: if the target viewpoint is on the right side of the vehicle, the system will perspective-stretch the image content to the left. This stretching is to counteract the perspective compression effect that inevitably occurs when viewing curved lights from the right. The corrected image data, although appearing distorted in the front view, will appear distorted from the target viewpoint. When observed from a distance, its geometric proportions will be restored to a normal visual form, thereby achieving precise information delivery to specific dynamic targets.

[0052] This embodiment details how to use reverse ray projection technology to accurately map viewpoint-corrected 2D image data onto the 3D non-Euclidean geometric surface of a vehicle's ISD matrix lights. This process not only solves the topological mapping problem of adapting planar images to curved surface carriers, but also introduces an optical compensation mechanism based on surface normal vectors.

[0053] The topology mapping rendering module first loads a high-precision 3D mesh model of the vehicle's ISD lighting fixtures into a virtual 3D space. This mesh model consists of tens of thousands of triangular faces, accurately replicating the curvature variations of the physical lampshade and the geometric undulations of the Fresnel lens. Simultaneously, the system, at a determined target viewing point... A virtual imaging plane is constructed between the grid model and the modified image data generated in the aforementioned embodiments.

[0054] To establish the geometric relationship between image pixels and the lamp surface, the system executes a ray casting algorithm. For each valid pixel in the corrected image data... The system constructs a virtual projection ray. The starting point of this ray is fixed at the target observation point. , ray direction vector Pointing to this pixel from the viewpoint coordinates Absolute coordinates in three-dimensional space. The parametric equation of a ray is defined as: ; in, For the propagation distance parameter of the ray, for The magnitude of the vector. The system performs a ray-meshIntersectionTest operation in 3D space. This operation traverses the spatial partitioning tree structure of the lighting fixture model (such as BVH or Octree) to quickly locate the specific triangular facets where the ray intersects the lighting fixture surface. And calculate the precise coordinates of the spatial intersection point. .

[0055] Obtaining spatial intersections Then, the system must determine which LED control unit corresponds to that intersection point. Because... Typically located inside the triangular facet rather than at its vertices, the system uses barycentric interpolation to resolve the texture properties of this point. Let the hit triangular facet be... Composed of three vertices The composition, and their corresponding texture coordinates are respectively The system first calculates the intersection point. The area weighting coefficients (centroid coordinates) relative to these three vertices. And satisfy Subsequently, the precise texture coordinates of the intersection point. Obtained through linear interpolation: ; Through the above calculations, the system establishes a reverse index table from visual image pixels to LED texture space coordinates. This means that the color value of each pixel in the corrected image is precisely assigned to a specific location in the ISD luminaire UV unfolded map.

[0056] Furthermore, during the calculation of the intersection point, the system further extracts the surface normal vector at the intersection point. The system calculates the direction vector of the incident ray. With surface normal vector The angle between This included angle reflects the degree of inclination of the viewing line of sight relative to the curved surface of the luminaire. Given that the light-transmitting material on the surface of ISD luminaires typically exhibits the Fresnel effect, meaning that light transmittance decreases at large viewing angles, the system adjusts the angle accordingly. Generate a brightness compensation gain coefficient. For projection points located at edges of high curvature or in areas with large tilt angles, the system automatically increases the pixel brightness value at that location to compensate for optical attenuation caused by physical materials.

[0057] Ultimately, the process generates a distorted texture map that is geometrically fully corrected and includes pre-compensated brightness information. When viewed on a two-dimensional plane, this texture map presents a distorted shape consistent with the unfolded shape of the lamp's curved surface (e.g., straight lines become arcs adapted to the lamp's curvature), but when it is fitted onto a physically curved lamp and viewed from the target viewpoint, the visual effect will strictly reproduce the planar geometric features designed by the user.

[0058] This embodiment details how to convert continuous, high-resolution virtual correction image data into discrete control signals to drive the physical ISD matrix lamp components. Since the physical LED array consists of discrete light-emitting units with fixed physical spacing, directly sampling or binarizing the virtual image (i.e., simply determining whether the LEDs are on or off) will cause broken geometric shapes, jagged edges, and flickering during dynamic movement. This embodiment solves these problems by introducing a sub-pixel energy dithering algorithm based on physical topology.

[0059] The topology mapping rendering module first reads the pre-stored physical distribution data of the LED array. This data defines the precise center coordinates of each physical LED emitting unit in the texture coordinate system (UV space). The system does not treat the physical LED as a geometrically singular point, but rather as an energy integration region with a specific optical influence range. The system provides each physical LED light-emitting unit with... Define a topological neighborhood The coverage radius of this topological neighborhood. Depending on the characteristics of the light-diffusing material on the surface of the physical luminaire and the physical pitch between adjacent LEDs, it is usually set to 1.5 to 2 times the LED pitch to ensure that the sampling areas of adjacent LEDs partially overlap, thereby ensuring visual continuity.

[0060] The system iterates through each physical LED light-emitting unit in the LED array. The system performs energy convolution operations. It then searches for LEDs falling within the topological neighborhood of that LED. The set of all virtual image pixels within the range For each virtual pixel in this set, the system calculates its texture coordinates and the coordinates of the LED center. Euclidean distance between Based on this distance, the system calculates the contribution weight of the virtual pixel to the physical LED housing using a radial basis function. This embodiment preferably uses a Gaussian kernel function as the weight distribution model to simulate the physical diffusion and attenuation process of light within the lampshade. Physical LED Light Emitting Unit Target brightness value The calculation is expressed as: ; in, In the LED topological neighborhood Within, for all adjacent points Perform a traversal. For virtual image pixels The original brightness value after geometric correction; A smoothing factor to control the degree of energy diffusion; this parameter is positively correlated with the haze value of the luminaire lens. Exponential function, point and adjacent points The Euclidean distance between them.

[0061] Through the above calculations, the energy of the virtual lines originally located in the gap between the two physical LEDs is distributed to the physical LEDs on both sides according to the distance ratio. When observed by the human eye, the difference in brightness between the two adjacent LEDs will visually reconstruct a virtual brightness center between them, thereby achieving a sub-pixel level fine display effect and eliminating the macroscopic jaggedness.

[0062] After achieving the target brightness value for all physical LED light-emitting units After calculation, the topology mapping rendering module performs gamma correction on the brightness data to adapt to the nonlinear perception characteristics of the human eye to changes in light intensity and to prevent the loss of low grayscale information. The corrected brightness data is quantized into a digital control sequence with a specific width (e.g., 8-bit or 12-bit). This digital control sequence is sent to the LED driver controller of the ISD matrix lamp assembly via a high-speed vehicle communication interface (such as LVDS or EthernetAVB).

[0063] The LED driver controller integrates a multi-channel PWM generator. Upon receiving a digital control sequence, the LED driver controller interprets it into a pulse width modulation (PWM) signal corresponding to each physical LED channel. The controller dynamically adjusts the duty cycle of the PWM waveform based on the interpreted values. For LEDs with high calculated brightness, the controller outputs a high duty cycle signal to extend the current conduction time; for auxiliary LEDs that only receive edge energy, the controller outputs a low duty cycle signal to produce a faint glow. Ultimately, driven by these PWM signals, the LED array presents graphic information on the physical surface that is visually highly consistent with the source image designed by the user in the mobile terminal's AR interface, with smooth edge transitions.

[0064] This embodiment further illustrates, through two specific typical application scenarios, how the system, based on different triggering conditions (user-initiated editing or vehicle-initiated passive triggering), utilizes the aforementioned spatial calibration, semantic parsing, and reverse projection technologies to achieve visual consistency across terminals and perspectives.

[0065] Scenario 1: Static Custom Graphic Editing Based on Augmented Reality This scenario describes the process of an external operator creating personalized content for ISD matrix lights using a mobile terminal while the vehicle is stationary.

[0066] In this scenario, an external operator stands with a mobile terminal at the left rear of the vehicle (defined as position A in physical space). The mobile terminal's image acquisition module captures a video stream of the vehicle's rear. The system uses the aforementioned PnP pose calculation algorithm to lock the mobile terminal's six-degree-of-freedom pose relative to the vehicle in real time and constructs a virtual mesh on the mobile terminal screen that perfectly overlaps with the physical vehicle lights.

[0067] The operator draws a standard circular geometric pattern on the AR layer of the mobile terminal screen. In traditional texture mapping techniques, if this circular pattern is directly projected onto the side of a taillight with a large curvature, it will typically appear as a flattened ellipse when viewed from position A due to perspective compression. However, in the system of this invention, the first processing module records position A as the initial viewing point.

[0068] The system performs reverse ray projection, emitting rays from position A as the projection origin onto the virtual mesh. The resulting mapping is essentially a pre-deformed texture map: on the UV unfolding plane of the ISD luminaire, the circular texture data is horizontally stretched. When the LED driver controller drives the physical LED array to emit light based on this stretched texture data, the light observed from position A, after physical occlusion and perspective scaling of the luminaire's curved surface, results in an image that is precisely restored to the standard circle drawn by the operator. This process achieves a WYSIWYG editing effect, eliminating visual distortion caused by the inconsistency between the viewing angle and the projection surface normal.

[0069] Scenario 2: Dynamic blind spot warning based on vehicle semantics This scenario describes the process by which the system automatically adjusts the direction of information delivery based on real-time traffic conditions while the vehicle is in motion.

[0070] Assuming the vehicle is making a right turn, the vehicle's semantic state parsing module receives signals from the CAN bus indicating that the right turn signal is on and the steering wheel angle to the right is greater than 30 degrees. Simultaneously, the blind spot monitoring radar located in the right rear bumper detects a cyclist at a position approximately 5 meters to the right rear of the vehicle, at an azimuth angle of 135 degrees (relative to the direction of the vehicle's front).

[0071] If the system only renders the warning icon (such as a right-pointing arrow) from directly behind the vehicle (i.e., from the perspective of the vehicle behind), due to the physical curvature of the ISD light surface and the directional design of the internal Fresnel lens, riders at a large angle to the right rear often only see a blurry spot of light or a discontinuous image with extremely low brightness, and cannot clearly identify the warning content.

[0072] After the viewpoint migration calculation module of this invention intervenes, the system dynamically migrates the target observation viewpoint from directly behind to the cyclist's location based on the cyclist's relative coordinates. The system calls the homography transformation algorithm to reconstruct the source image data of the right-pointing arrow in real time, using the cyclist's line of sight as a reference. The reconstructed image data is displayed on the physical LED array as a distorted image that is extremely stretched to the left of the vehicle (i.e., the side away from the cyclist).

[0073] When the light emitted from the distorted image is projected towards the cyclist at a 135-degree angle, due to the principle of perspective projection, the stretched portion is compressed back to its normal geometric proportions on the cyclist's retina. Simultaneously, the topology mapping rendering module calculates the angle of incidence between the light and the lamp surface, discovering that this angle falls within the high attenuation region of Fresnel reflection. The system then automatically increases the drive current (PWM duty cycle) of the corresponding LED beads to compensate for brightness.

[0074] The end result is that even when the vehicle is turning and the light fixture surface is curved, riders in the blind spot can still see a geometrically standard, sufficiently bright, and clearly defined warning arrow, thus effectively receiving information about the vehicle's turning intentions and improving traffic interaction safety. Through the above process, the system transforms the ISD matrix light from a passive display screen into an intelligent optical terminal that can actively seek out viewers and optimize the display effect.

Claims

1. A method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems, characterized in that, Includes the following steps: Step S100: Establish heterogeneous spatial association between mobile terminal and vehicle terminal: The mobile terminal receives key feature point data of vehicle 3D model sent by vehicle terminal, collects external images of vehicle, matches the external images of vehicle with the key feature point data, and calculates the real-time spatial pose of the mobile terminal relative to the vehicle based on the matching result. Step S200: Obtain the initial visual intent based on the observation viewpoint: Based on the real-time spatial pose, receive the two-dimensional source image data input by the user in the augmented reality interface of the mobile terminal, and record the spatial position of the mobile terminal at the time of input as the initial observation viewpoint. Step S300: Parse vehicle semantic state and generate target viewpoint: The vehicle terminal obtains the real-time semantic state vector of the vehicle and calculates the target observation viewpoint relative to the vehicle coordinate system through the real-time semantic state vector. Step S400: Perform dynamic perspective correction based on visual intent: Construct a transformation relationship based on the spatial position difference between the target viewing point and the initial viewing point, perform homography transformation on the two-dimensional source image data, and generate corrected image data; Step S500: Perform surface inverse ray casting and texture coordinate picking: Using the target viewing point as the projection origin, perform inverse ray casting on the corrected image data to establish the geometric mapping relationship between the corrected image data and the surface three-dimensional mesh model of the vehicle ISD matrix light component. Step S600: Perform discrete topology energy jitter and brightness drive: Calculate the target brightness value of each physical light-emitting unit in the ISD matrix lamp assembly through the geometric mapping relationship, and drive the ISD matrix lamp assembly to display according to the target brightness value.

2. The method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems according to claim 1, characterized in that, The steps in S100 for establishing the heterogeneous spatial association between the mobile terminal and the vehicle terminal specifically include: Extract two-dimensional image feature points from the vehicle exterior image; A point set corresponding to the two-dimensional image feature points and the key feature point data is established using a feature descriptor matching algorithm; Based on the corresponding point set, through perspective Point localization algorithm constructs reprojection error function; The rotation matrix and translation vector of the mobile terminal's camera coordinate system relative to the vehicle's world coordinate system are calculated by minimizing the reprojection error function through nonlinear optimization.

3. The method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems according to claim 1, characterized in that, The steps in S300, which involve parsing the vehicle's semantic state and generating the target viewpoint, specifically include: Vehicle steering angle data, wheel speed data, and gear data are obtained through the vehicle bus. The relative orientation coordinates of the target object are obtained by blind spot monitoring radar and environmental perception cameras. The real-time semantic state vector is constructed by combining the steering angle data, wheel speed data, gear data, and relative orientation coordinate data.

4. The method for custom editing ISD matrix light characters based on multi-terminal linkage of vehicle-mounted systems according to claim 3, characterized in that, The specific steps for calculating the target observation point relative to the vehicle coordinate system using the real-time semantic state vector include: Determine whether the real-time semantic state vector meets the preset security warning triggering conditions; If the safety warning triggering conditions are met, a line-of-sight tracking vector is constructed by extracting the relative orientation coordinates of the target object; Using the vehicle reference center point as the origin, spatial coordinates are calculated by extending a preset distance along the direction of the line-of-sight tracking vector, and these spatial coordinates are determined as the target observation viewpoint.

5. The method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems according to claim 1, characterized in that, The steps in step S400 that perform dynamic perspective correction of visual intent specifically include: Construct an original projection plane based on the initial viewing point and a target projection plane based on the target viewing point in three-dimensional space; The homography matrix between the original projection plane and the target projection plane is calculated. The pixel coordinates of the two-dimensional source image data are inversely mapped using the homography matrix, so that the projection geometry of the corrected image data at the target viewing point is consistent with that of the two-dimensional source image data.

6. The method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems according to claim 1, characterized in that, The step S500, which establishes the geometric mapping relationship between the corrected image data and the curved surface 3D mesh model, specifically includes: Using the target observation point as the ray starting point, a virtual ray is emitted into the spatial direction corresponding to each pixel unit of the corrected image data. The intersection test calculation is performed between the virtual ray and the triangular facets of the curved three-dimensional mesh model; When the virtual ray intersects the triangular facet, the three-dimensional coordinates of the intersection point in the vehicle world coordinate system are obtained, and a mapping index between the three-dimensional coordinates and the pixel units in the corrected image data is established.

7. The method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems according to claim 6, characterized in that, Step S500 further includes a step of obtaining the mapped texture coordinates based on barycentric coordinate interpolation, wherein the step of obtaining the mapped texture coordinates based on barycentric coordinate interpolation includes: Determine the three vertices of the triangular facet containing the intersection point and the texture coordinates corresponding to the three vertices; The centroid coordinate weights are calculated based on the positional relationship of the intersection points relative to the three vertices. By performing linear interpolation calculations on the texture coordinates of the three vertices using the centroid coordinate weights, the mapped texture coordinates of the intersection point on the unfolded plane of the ISD matrix lamp component are obtained.

8. The method for custom editing ISD matrix light characters based on multi-terminal linkage of vehicle-mounted systems according to claim 6, characterized in that, Step S500 further includes a brightness compensation step based on surface normal vectors, wherein the brightness compensation step based on surface normal vectors includes: Obtain the surface normal vector at the intersection point; The angle between the direction vector of the virtual ray and the normal vector of the surface is calculated. The brightness gain coefficient is calculated by querying the preset optical attenuation compensation curve based on the size of the included angle, and then superimposed on the brightness of the corresponding pixel in the corrected image data.

9. The method for custom editing ISD matrix light characters based on multi-terminal linkage of vehicle-mounted systems according to claim 6, characterized in that, Step S600, which involves calculating the target brightness value of each physical light-emitting unit in the ISD matrix lamp assembly using the geometric mapping relationship, specifically includes: Determine the center coordinates of each physical luminescent unit in the texture coordinate system and set the topological neighborhood range; Retrieve all projection points determined by the mapping index according to claim 6 that fall within the topological neighborhood; The Euclidean distance is calculated using the texture coordinates of each projection point and the center coordinates of the physical light-emitting unit. The target brightness value of the physical light-emitting unit is obtained by calculating the weights based on the Euclidean distance using the radial basis function and weighting and accumulating the image brightness values ​​corresponding to each projection point.

10. The method for custom editing ISD matrix light characters based on multi-terminal linkage in vehicle-mounted systems according to claim 9, characterized in that, The step of driving the ISD matrix lamp component to display according to the target brightness value specifically includes: A digital grayscale sequence is generated by performing Gamma correction calculation on the target brightness value; The digital grayscale sequence is converted into duty cycle data of a pulse width modulation signal using a conversion algorithm. The duty cycle data is used to control the conduction time of the corresponding LED beads in the ISD matrix lamp assembly.

Images