A-column perspective system based on vehicle-mounted AR glasses and two-dimensional code positioning method

Abstract

The invention discloses an A-column perspective system based on vehicle-mounted AR glasses and a two-dimensional code positioning method, and relates to the technical field of automobile safety and augmented reality. The system comprises a vehicle-mounted AR glasses module, an A column external camera module, a two-dimensional code positioning module, a space calculation module, a system integrated control unit and other vehicle systems. The A column external camera module is used for capturing the road condition on the outer side of the A column in real time; the two-dimensional code positioning module is used for accurately positioning the position of the A column in the visual field of a driver; the space calculation module is used for ensuring that a projected picture is matched with an actual visual angle; the vehicle-mounted AR glasses module is used for detecting whether the sight of a driver moves to the position of a column A or not; and the system integrated control module is responsible for coordinating the work of each module. According to the method, the sight blind area caused by the A column is effectively eliminated through the AR technology, the driving safety is remarkably improved, and meanwhile the structural integrity of the A column is kept.

Description

Technical Field

[0001] The present invention relates to the field of automobile safety and augmented reality technology, and belongs to the technical field, and specifically relates to an A-pillar perspective system based on vehicle-mounted AR glasses and a QR code positioning method. Background Art

[0002] The A-pillar is crucial to the safety of the car, and its thickness is directly related to the safety factor of the car. However, as the requirements for safety collision tests become more and more stringent, in order to improve the safety performance of the whole vehicle, car companies are designing new cars with wider and wider A-pillars, resulting in a larger and larger visual blind spot for the driver. This blind spot is especially obvious when the car turns left, and pedestrians and motorcycles 15 meters away from the left front of the car, and even cars, may be blocked by it, seriously affecting driving safety.

[0003] In order to solve the A-pillar blind spot problem, the industry has made many attempts:

[0004] 1. Change the A-pillar design: For example, Volvo proposed the transparent A-pillar concept in 2001, which made the A-pillar transparent by opening a small triangular window on the A-pillar. However, this method sacrificed the firmness of the A-pillar and affected the overall stability of the car, so it was only briefly unveiled on the concept car and no further development was made.

[0005] 2. Using the physical principle of light refraction: Toyota once applied for a patent for "an instrument that makes objects transparent" in the United States. However, this technology requires the use of expensive materials and equipment, so it has not been put into practical use.

[0006] 3. Virtualization of the A-pillar through camera and display technology: For example, the 360° virtual city windshield technology proposed by Jaguar Land Rover in 2015 not only makes the A-pillar transparent, but also the B-pillar and C-pillar. Continental also released virtual A-pillar technology in 2018, which mainly uses the vehicle's surround view camera and the OLED display embedded in the A-pillar to allow the driver to observe the vehicle's external environment in real time through the OLED screen. However, these technologies have not yet been widely used due to high costs or immature technologies.

[0007] 4. Nezha Auto's "transparent A-pillar" technology: This technology uses four OLED flexible screens, two external cameras and one internal head detection camera, as well as an AR-View system equipped with intelligent software algorithms. Its core is the "eyebrow center algorithm", which tracks the driver's eyes and calculates the position of the eyebrow center, thereby confirming and adjusting the display perspective angle of the A-pillar display in real time.

[0008] Although these approaches offer improvements in some areas, some issues remain:

[0009] 1. Affect the structural strength of the vehicle or increase manufacturing complexity.

[0010] 2. The cost is too high and difficult to apply in large-scale production.

[0011] 3. Some solutions may not work well at high speeds or in complex lighting conditions.

[0012] 4. Risk of causing visual distraction or increasing cognitive burden on the driver.

[0013] In addition, most existing solutions need to be considered during the vehicle design phase and are difficult to apply to the modification and upgrading of existing models. Summary of the invention

[0014] The purpose of the present invention is to provide an A-pillar perspective system and a QR code positioning method based on vehicle-mounted AR glasses to solve the problems raised in the above-mentioned background technology.

[0015] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0016] A method for A-pillar perspective and QR code positioning based on vehicle-mounted AR glasses, the steps comprising:

[0017] S100: The driver wears AR glasses that are wired to the vehicle system;

[0018] S200: Uses gaze tracking technology to detect whether the driver's gaze moves to the A-pillar position;

[0019] S300: Use the QR code positioning module to accurately locate the position of the A-pillar in the field of view;

[0020] S400: Real-time acquisition of images captured by the A-pillar external camera;

[0021] S500: Process the image through spatial calculation to ensure that it matches the actual viewing angle;

[0022] S600: Projecting the processed image to a corresponding A-pillar position in the driver's field of vision.

[0023] According to the above solution, the two-dimensional code positioning in step S300 includes:

[0024] S310: Paste a specific QR code mark on the surface of the A-pillar;

[0025] S320: Recognize the QR code through the camera of the AR glasses;

[0026] S330: Calculate the precise position and size of the A-pillar in the field of view.

[0027] According to the above solution, the space calculation in step S500 includes:

[0028] S510: Adjusting the perspective of the picture according to the position and direction of the driver's head;

[0029] S520: Corrects image brightness and contrast in real time to match current ambient light conditions;

[0030] S53O: Perform real-time performance optimization.

[0031] According to the above solution, in step S510:

[0032] Get the position and orientation of the driver's head, and use the Kalman filter algorithm to estimate the 6-DOF posture of the head. The posture matrix R is expressed as:

[0033]

[0034] Where R is expressed by quaternion q=[q 0 ,q 1 ,q 2 ,q 3 ] is converted to q 0 is the real part of the quaternion, q 1 ,q 2 ,q 3 is the imaginary part of the quaternion, which defines the direction of rotation in space. The formula is as follows:

[0035] R 11 =1-2(q 2 2 +q 3 2 ), R 12 =2(q 1 q 2 -q 0 q 3 ), R 13 =2(q 1 q 3 -q 0 q 2 );

[0036] R 2 1=2(q 1 q 2 +q 00 q 3 ), R 12 =1-2(q 1 2 +q 3 2 ), R 13 =2(q 2 q 3 -q 0 q 1 );

[0037] R 31 =2(q 1 q 3 -q 0 q 2 ), R 32 =2(q 2 q 3 +q 0 q 1 ), R 33 =1-2(q 1 2 +q 2 2 );

[0038] Using the 3D coordinate information provided by the QR code, combined with the head position and orientation, the A-pillar coordinates in the 3D space are converted to the 2D image plane, the projection position of the A-pillar in the driver's field of view is determined, and the projection matrix P is obtained. The formula is as follows:

[0039] P=K[R|t]

[0040] Among them, K represents the camera intrinsic parameter matrix; R represents the rotation matrix; t represents the translation vector;

[0041] Use the homography matrix H to perform perspective transformation, map the external camera image to the driver's perspective, and realize the perspective transformation from the external camera image to the AR glasses perspective. The formula of the homography matrix H is as follows:

[0042] H=K 2 ×R×(R 1 -1 )×K 1 -1

[0043] Among them, K 1 Represented as the intrinsic parameter matrix of the external camera; K 2 Represented as the internal parameter matrix of AR glasses; R 1 It is represented as the rotation matrix of the external camera relative to the world coordinate system; R is represented as the rotation matrix of the AR glasses relative to the world coordinate system;

[0044] The Brown-Conrady model is used for distortion correction to correct the effect of lens distortion on the image. distorted ,y distorted ) is converted to the ideal coordinates (x, y), and the distortion correction formula is as follows:

[0045]

[0046] Among them, k 1 , k2 , k 3 Expressed as radial distortion coefficient; p 1 , p 2 Expressed as the tangential distortion coefficient; r 2 =x 2 +y 2 Expressed as the square of the distance from the center of the image to the pixel;

[0047] Use the alpha blending algorithm to fuse the images and combine the AR content A and the actual background B into a single image C. The formula is as follows:

[0048] C=αA+(1-α)B

[0049] Among them, A represents the AR content; B represents the actual background; α represents the mixing coefficient.

[0050] According to the above scheme, the correction of picture brightness and contrast includes:

[0051] The single image C is passed through a Kalman filter algorithm to reduce noise and mutations. The formula is as follows:

[0052]

[0053] P k =(IK k H)P k-1

[0054] in, Represented as the current state estimate; Represents the previous state estimate; z k Expressed as a measured value; K k It is represented as Kalman gain, which is used to balance the prediction error and measurement error; H is represented as the measurement matrix, which maps the state vector to the measurement space; P k It is represented as the estimated error covariance; I is represented as the identity matrix, whose elements on the main diagonal are all 1, and other off-diagonal elements are all 0;

[0055] After reducing noise and mutations in the image, use the gamma correction formula to adjust the image brightness. The formula is as follows:

[0056]

[0057] Among them, V in Represented as input pixel value; V out It is represented as the output pixel value; A is represented as the scaling factor; β is represented as the gamma value;

[0058] The gamma value β is dynamically adjusted according to the ambient light intensity, and the formula is as follows:

[0059]

[0060] Among them, β base Indicates the basic gamma value; L indicates the ambient light intensity; L ref It is represented as reference light intensity; k is represented as adjustment coefficient;

[0061] Use the contrast stretching algorithm to enhance the contrast of the image. The formula is as follows:

[0062]

[0063] Among them, V in Represents the input pixel value; min and max represent the current minimum and maximum pixel values ​​of the image; newMin and newMax represent the minimum and maximum values ​​of the target contrast range;

[0064] The target ranges newMin and newMax are dynamically adjusted according to the ambient light intensity L. The adjustment formula is as follows:

[0065] newMin=a×log(L)+b

[0066] newMax=c×L 0.5 +d

[0067] Wherein, L represents the current ambient light intensity; a, b, c, d represent empirical parameters obtained by fitting a large amount of experimental data;

[0068] Apply the local adaptive histogram equalization algorithm to perform histogram equalization in a small area of ​​the image and limit contrast amplification to avoid excessive noise amplification;

[0069] The local adaptive histogram equalization algorithm divides the image into 8×8 blocks. The index of the divided blocks is (i, j). Each block is represented by B ij , calculate each small block B ij The histogram H ij , using contrast limitation, redistribute the part that exceeds the limit to keep the overall pixel value unchanged. The contrast limitation formula is as follows:

[0070]

[0071] Among them, μ ij Represented as a small block B ij The histogram mean of ij Expressed as standard deviation; H ij (x) is represented as the histogram of the number of pixels with pixel intensity x in the (i, j)th small square of the image; H′ ij Represented as a restricted histogram;

[0072] Calculate the cumulative distribution function F of each restricted histogram ij , the formula is as follows:

[0073]

[0074] Among them, l represents the pixel intensity level in the histogram;

[0075] Pixel mapping is performed according to the cumulative distribution function to convert the original pixel value into a new pixel value to enhance the contrast. The formula is as follows:

[0076] V out (x,y)=F -1 (V in (x,y)

[0077] Among them, V in (x, y) represents the pixel value of the original image at position (x, y); V out (x, y) represents the pixel value after mapping; F -1 Expressed as the inverse of the cumulative distribution function;

[0078] In scenes with great lighting differences, such as tunnel entrances and exits, high dynamic range synthesis technology is used for image processing;

[0079] The A-pillar exterior camera captures multiple images in rapid succession with different exposure settings, denoted as I i , where i represents the image index at different exposure levels; the final image I is synthesized using the exposure fusion algorithm final , the formula is as follows:

[0080]

[0081] Among them, I i represents an image captured under a specific exposure setting; w i Represented as I i The weight of .

[0082] According to the above scheme, the real-time performance optimization includes:

[0083] Use GPU to optimize performance and accelerate algorithms; use pipeline processing to process multiple frames of images simultaneously; use lookup tables to pre-calculate some complex functions to accelerate algorithm processing;

[0084] Through the combined application of these technologies and algorithms, the system is able to provide clear and comfortable A-pillar perspective effects under various lighting conditions (from bright sunlight to dark night), greatly improving driving safety and user experience.

[0085] An A-pillar perspective system based on vehicle-mounted AR glasses, the system includes a vehicle-mounted AR glasses module, an A-pillar external camera module, a QR code positioning module, a spatial calculation module, a system integration control unit and other vehicle systems;

[0086] The vehicle-mounted AR glasses module includes a display unit, a data transmission interface, a head tracking unit and an image acquisition unit; the A-pillar external camera module includes an image sensor, an image processing chip, an anti-shake unit and a data transmission interface; the spatial calculation module includes a perspective transformation unit, a distortion correction unit, an image enhancement unit and a real-time rendering unit; the system integration control unit includes a data processing unit, a real-time control unit and a vehicle interface unit; the two-dimensional code positioning module includes an image preprocessing unit, a two-dimensional code recognition algorithm unit, a position information decoding unit and an error correction unit;

[0087] The A-pillar external camera module is used to capture the road conditions outside the A-pillar in real time and transmit the collected video data to the system integration control unit; the QR code positioning module is used to accurately locate the position of the A-pillar in the driver's field of view and transmit the position information to the system integration control unit; the spatial calculation module is used to ensure that the projected image matches the actual viewing angle, send a data processing request to the system integration control unit, and receive feedback on the processing results of the system integration control unit; the on-board AR glasses module is used to detect whether the driver's line of sight moves to the A-pillar position and complete data interaction with the system integration control unit; the system integration control module is responsible for coordinating the work of each module and exchanging data with other vehicle systems.

[0088] According to the above solution, the display unit is used to display the processed image or video to the driver; the data transmission interface realizes data exchange between the AR glasses and the vehicle-mounted system; the head tracking unit detects the movement of the driver's line of sight by implementing tracking technology, especially when the line of sight moves to the A-pillar position; the image acquisition unit is used to capture the scene of the driver's surrounding environment;

[0089] The image sensor is used to capture the road conditions outside the A-pillar; the image processing chip is used to perform spatial calculations on the captured image; the anti-shake unit is used to ensure that the image captured by the camera is stable and clear; the data transmission interface transmits the image data to the vehicle-mounted system for further processing;

[0090] The image preprocessing unit is used to correct the distortion of the image captured by the AR glasses camera; the two-dimensional code recognition algorithm recognizes the two-dimensional code mark on the A-pillar, so as to determine the position of the A-pillar in the field of view; the position information decoding unit is used to extract the position information from the recognized two-dimensional code; the error correction unit corrects the errors that may be generated in the two-dimensional code recognition process through the Reed-Solomon error correction algorithm;

[0091] The perspective transformation unit uses a homography matrix to perform perspective transformation of the image to ensure that the image matches the driver's perspective; the distortion correction unit applies the Brown-Conrady model to perform distortion correction on the image; the image enhancement unit performs gamma correction, contrast stretching algorithm and local adaptive histogram equalization algorithm to optimize image quality; the real-time rendering unit renders the processed image in real time on the display unit of the AR glasses;

[0092] The data processing unit receives data from each module for centralized processing and analysis; the real-time control unit adjusts the system status in real time according to the processing results to ensure that the AR display is synchronized with the actual driving situation; the vehicle interface unit is connected to the real-time control unit and is used to obtain vehicle status information.

[0093] According to the above scheme, the vehicle interface unit monitors the current vehicle speed in real time; when the vehicle speed is lower than the preset threshold, the system integration control unit automatically activates the system; when the vehicle speed is higher than the preset threshold, the system integration control unit automatically shuts down the system.

[0094] According to the above scheme, the real-time control unit monitors the ambient light conditions in real time. When it is detected that the vehicle is in a low-light environment, the image enhancement unit automatically adjusts the brightness and contrast of the image and transmits the adjusted image to the vehicle-mounted AR glasses module.

[0095] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0096] 1. AR glasses connected via wires significantly reduce system latency and improve driving safety;

[0097] 2. QR code positioning technology provides sub-centimeter-level A-pillar position information without changing the vehicle structure;

[0098] 3. Real-time spatial computing ensures accurate matching of AR content with actual viewing angles, with visual fusion reaching over 95%;

[0099] 4. The system can automatically adjust its working state according to vehicle speed and ambient light to adapt to various driving environments;

[0100] 5. The entire system is modularly designed and can be implemented on more than 90% of existing vehicle models, with good compatibility and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0102] Figure 1 is a schematic diagram of an A-pillar perspective system based on vehicle-mounted AR glasses of the present invention;

[0103] Figure 2 It is a schematic diagram of the steps of an A-pillar perspective system and a QR code positioning method based on vehicle-mounted AR glasses of the present invention;

[0104] Figure 3 It is a comprehensive schematic diagram of AR perspective effect and spatial calculation principle of an A-pillar perspective system and a QR code positioning method based on vehicle-mounted AR glasses in the present invention. DETAILED DESCRIPTION

[0105] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0106] See also Figure 1-Figure 3 , the present invention provides a technical solution:

[0107] According to one embodiment of the present invention, an A-pillar perspective system based on vehicle-mounted AR glasses includes a vehicle-mounted AR glasses module, an A-pillar external camera module, a QR code positioning module, a spatial calculation module, a system integration control unit and other vehicle systems;

[0108] The vehicle-mounted AR glasses module includes a display unit, a data transmission interface, a head tracking unit and an image acquisition unit; the A-pillar external camera module includes an image sensor, an image processing chip, an anti-shake unit and a data transmission interface; the spatial calculation module includes a perspective transformation unit, a distortion correction unit, an image enhancement unit and a real-time rendering unit; the system integration control unit includes a data processing unit, a real-time control unit and a vehicle interface unit; the two-dimensional code positioning module includes an image preprocessing unit, a two-dimensional code recognition algorithm unit, a position information decoding unit and an error correction unit;

[0109] The vehicle-mounted AR glasses module is used to detect whether the driver's line of sight moves to the A-pillar position; the vehicle-mounted AR glasses adopt a lightweight design and weigh no more than 100g; an OLED micro display screen with a 2K resolution, a brightness of no less than 1000nits, and a contrast ratio of more than 10000:1 is used. A built-in 6-degree-of-freedom (6DOF) inertial measurement unit (IMU) with a sampling rate of 1000Hz is used to accurately track head movements. The vehicle-mounted AR glasses module completes data exchange with the system integration control unit.

[0110] AR glasses are connected to the vehicle system through a high-speed data transmission interface, such as USB, to ensure low latency in data transmission, with the delay controlled within 10ms.

[0111] The A-pillar external camera module is used to capture the road conditions outside the A-pillar in real time; the module uses a 1 / 1.8-inch large-size sensor with a pixel size of no less than 2μm, which can maintain high image quality even in low-light environments. The camera also integrates a real-time image processing chip (ISP), supports hardware-level HDR and defogging algorithms, and high dynamic range (HDR) functions, with a resolution of no less than 1080p and a field of view of no less than 120 degrees. It is combined with anti-shake technology to ensure the stability of the picture during driving. The collected video data is transmitted to the system integration control unit.

[0112] The QR code positioning module is used to accurately locate the position of the A-pillar in the driver's field of view; by pasting a specially designed high-contrast, micro QR code on the surface of the A-pillar, this QR code contains the precise position information and calibration data of the A-pillar, and the coding density reaches version 40 (177x177 module), while using the Reed-Solomon error correction algorithm, it can correctly identify even when 40% of the code is damaged; the high-speed camera built into the AR glasses can quickly capture and decode these QR codes at a frame rate of not less than 120fps, and the optimized QR code recognition algorithm ensures that the recognition rate under different lighting and angles reaches more than 99.9%. The QR code positioning module transmits the identified location information to the system integration control unit;

[0113] The spatial computing module is used to ensure that the projected image matches the actual viewing angle; send data processing requests to the system integration control unit, and receive processing result feedback from the system integration control unit. This module uses the on-board high-performance GPU for real-time computing, and can process no less than 60 frames per second. At the same time, it uses a deep learning accelerator to support TensorRT-optimized neural network reasoning, and can process 200+ AI reasonings per second. According to the location information provided by the QR code and the driver's head posture data, it performs real-time perspective transformation and distortion correction; achieves sub-pixel image registration accuracy to ensure seamless integration of AR content with the real world; when the system detects that the environment is in a low-light environment, it can automatically adjust the brightness and contrast of the AR glasses display content in a low-light environment to adapt to changes in the external environment.

[0114] The system integration control module is responsible for coordinating the work of each module and exchanging data with other vehicle systems. The vehicle speed is monitored. When the vehicle speed is higher than the preset threshold, the system is automatically shut down. When the vehicle speed is lower than the preset threshold, the system is automatically activated. This module adopts automotive-grade SoC and integrates ASIL-B safety level. It supports CAN-FD and automotive Ethernet to achieve seamless integration with other vehicle systems.

[0115] According to one embodiment of the present invention, an A-pillar perspective and QR code positioning method based on vehicle-mounted AR glasses.

[0116] The AR glasses worn by the driver are connected to the vehicle system through a high-speed data transmission interface, such as USB 3.1 or Thunderbolt. This connection not only ensures low latency of data transmission, but also provides the driver with a comfortable wearing experience and clear image quality through lightweight design, weighing no more than 100g, and high-resolution micro-display, such as 2K resolution. After detecting the connection of AR glasses, the system automatically initializes and calibrates, including checking the data transmission rate, adjusting the display parameters, and preliminary matching with the driver's perspective.

[0117] The system uses gaze tracking technology and infrared eye tracking to achieve fast response at an extremely high sampling rate to accurately determine whether the driver's gaze has moved to the A-pillar position. The infrared eye tracking sampling rate is not less than 250Hz, achieving a 1-2ms gaze tracking response.

[0118] The sight line determination threshold of the A-pillar area is set. When the sight line stays in this area for more than 100ms, the system triggers the QR code positioning module, and the high-speed camera built into the AR glasses captures and decodes the QR code pasted on the A-pillar. The QR code is a high-contrast, miniature (side length does not exceed 1cm) QR code; the QR code contains the precise location information and calibration data of the A-pillar, and the coding density reaches version 40 (177x177 modules). The Reed-Solomon error correction algorithm is applied, which can correctly identify even when part of the code is damaged, ensuring the accuracy and robustness of positioning.

[0119] At the same time, the camera outside the A-pillar captures the picture in real time, and uses anti-shake technology to ensure the stability of the picture, providing a high-quality video stream for the spatial computing module. The camera outside the A-pillar uses a 1 / 1.8-inch large-size sensor with a pixel size of no less than 2μm to ensure the image quality in low-light environments and can capture the picture outside the A-pillar in real time. The use of an on-board high-performance GPU for real-time computing, combined with a deep learning accelerator and TensorRT-optimized neural network reasoning, achieves sub-pixel image registration accuracy, ensuring the seamless integration of AR content with the real world.

[0120] The spatial calculation module performs real-time perspective transformation and distortion correction based on the location information provided by the QR code and the driver's head posture data. In this process, the system uses advanced image processing algorithms to estimate the 6-DOF posture of the head based on the position and direction of the driver's head through the Kalman filter algorithm, and then calculates the homography matrix, corrects the distortion of the Brown-Conrady model, and performs local adaptive histogram equalization (CLAHE) based on the cumulative distribution function. The comprehensive application of these algorithms not only improves the contrast and brightness of the image, but also enhances the image quality under different lighting conditions.

[0121] In scenes with extreme differences in image lighting, such as tunnel entrances and exits, high dynamic range synthesis technology is used to quickly and continuously capture multiple images with different exposure settings through the A-pillar external camera, and an exposure fusion algorithm is used to synthesize the final image, enhancing the visual effect of the image in scenes with extreme differences in lighting.

[0122] The perspective-transformed and distortion-corrected images are projected to the A-pillar position in the driver's field of view through wired AR glasses with low latency. The high brightness and contrast of the OLED microdisplay, combined with real-time ray tracing technology, ensure the clear visibility and realism of the AR overlay content in various lighting environments. The system's strict control of the total latency, from capturing the picture to displaying the process does not exceed 30ms, provides a smooth and delay-free visual experience.

[0123] Through the combined application of these technologies and algorithms, the system can provide clear and comfortable A-pillar perspective effects under various lighting conditions (from bright sunlight to dark night), greatly improving driving safety and user experience.

[0124] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0125] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for A-pillar perspective and QR code positioning based on vehicle-mounted AR glasses, characterized in that: Steps include: S100: The driver wears AR glasses that are wired to the vehicle system; S200: Uses gaze tracking technology to detect whether the driver's gaze moves to the A-pillar position; S300: Use the QR code positioning module to accurately locate the position of the A-pillar in the field of view; S400: Real-time acquisition of images captured by the A-pillar external camera; S500: Process the image through spatial calculation to ensure that it matches the actual viewing angle; S600: Projecting the processed image to a corresponding A-pillar position in the driver's field of vision.

2. According to claim 1, the A-pillar perspective and QR code positioning method based on vehicle-mounted AR glasses is characterized by: The two-dimensional code positioning in step S300 includes: S310: Paste a specific QR code mark on the surface of the A-pillar; S320: Recognize the QR code through the camera of the AR glasses; S330: Calculate the precise position and size of the A-pillar in the field of view.

3. The A-pillar perspective and QR code positioning method based on vehicle-mounted AR glasses according to claim 1, characterized in that: The spatial calculation in step S500 includes: S510: Adjusting the perspective of the picture according to the position and direction of the driver's head; S520: Corrects image brightness and contrast in real time to match current ambient light conditions; S530: Perform real-time performance optimization.

4. The A-pillar perspective and QR code positioning method based on vehicle-mounted AR glasses according to claim 3 is characterized in that: In step S510: Get the position and orientation of the driver's head, and use the Kalman filter algorithm to estimate the 6-DOF posture of the head. The posture matrix R is expressed as: Among them, R is obtained by converting the quaternion q = [q0, q1, q2, q3], q0 is the real part of the quaternion, q1, q2, q3 are the imaginary parts of the quaternion, which define the direction of rotation in space. The formula is as follows: <h2 style=";text-align:left;direction:ltr">R<h2 style=";text-align:left;direction:ltr"> 11 <h2 style=";text-align:left;direction:ltr"> =1-2(q2<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +q3<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> ),R<h2 style=";text-align:left;direction:ltr"> 12 <h2 style=";text-align:left;direction:ltr"> =2(q1q2-q0q3),R<h2 style=";text-align:left;direction:ltr"> 13 <h2 style=";text-align:left;direction:ltr"> (2(q1q3-q0q2)) R 21 =2(q1q2+q0q3),R 12 =1-2(q1 2 +q3 2 ),R 13 =2(q2q3-q0q1); R 31 =2(q1q3-q0q2),R 32 =2(q2q3+q0q1),R 33 =1-2(q1 2 +q2 2 ); Using the 3D coordinate information provided by the QR code, combined with the head position and orientation, the A-pillar coordinates in the 3D space are converted to the 2D image plane, the projection position of the A-pillar in the driver's field of view is determined, and the projection matrix P is obtained. The formula is as follows: P=K[R|t] Among them, K represents the camera intrinsic parameter matrix; R represents the rotation matrix; t represents the translation vector; Use the homography matrix H to perform perspective transformation, map the external camera image to the driver's perspective, and realize the perspective transformation from the external camera image to the AR glasses perspective. The formula of the homography matrix H is as follows: H=K2×R×(R1 -1 )×K1 -1 Among them, K1 represents the internal parameter matrix of the external camera; K2 represents the internal parameter matrix of the AR glasses; R1 represents the rotation matrix of the external camera relative to the world coordinate system; R represents the rotation matrix of the AR glasses relative to the world coordinate system; The Brown-Conrady model is used for distortion correction to correct the effect of lens distortion on the image. distorted ,y distorted ) is converted to the ideal coordinates (x, y), and the distortion correction formula is as follows: <h2 style=";text-align:left;direction:ltr"> x distorted <h2 style=";text-align:left;direction:ltr"> =x(1+k1r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +k2r<h2 style=";text-align:left;direction:ltr"> 4 <h2 style=";text-align:left;direction:ltr"> +k3r<h2 style=";text-align:left;direction:ltr"> 6 <h2 style=";text-align:left;direction:ltr"> )+2p1xy+p2(r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +2x<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> ) <h2 style=";text-align:left;direction:ltr"> y distorted <h2 style=";text-align:left;direction:ltr"> =y(1+k1r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +k2r<h2 style=";text-align:left;direction:ltr"> 4 <h2 style=";text-align:left;direction:ltr"> +k3r<h2 style=";text-align:left;direction:ltr"> 6 <h2 style=";text-align:left;direction:ltr"> )+p1(r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +2y<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> )+2p2xy Among them, k1, k2, k3 represent the radial distortion coefficients; p1, p2 represent the tangential distortion coefficients; r 2 =x 2 +y 2 Expressed as the square of the distance from the center of the image to the pixel; Use the alpha blending algorithm to fuse the images and combine the AR content A and the actual background B into a single image C. The formula is as follows: C=αA+(1-α)B Among them, A represents the AR content; B represents the actual background; α represents the mixing coefficient.

5. The A-pillar perspective and QR code positioning method based on vehicle-mounted AR glasses according to claim 4, characterized in that: The correction of picture brightness and contrast comprises: The single image C is passed through a Kalman filter algorithm to reduce noise and mutations. The formula is as follows: P k =(I-K k H)P k-1 in, Represented as the current state estimate; represents the previous state estimate; zk represents the measured value; K k It is represented as Kalman gain, which is used to balance the prediction error and measurement error; H is represented as the measurement matrix, which maps the state vector to the measurement space; P k It is represented as the estimated error covariance; I is represented as the identity matrix, whose elements on the main diagonal are all 1, and other off-diagonal elements are all 0; After reducing noise and mutations in the image, use the gamma correction formula to adjust the image brightness. The formula is as follows: Among them, V in Represented as input pixel value; V out It is represented as the output pixel value; A is represented as the scaling factor; β is represented as the gamma value; The gamma value β is dynamically adjusted according to the ambient light intensity, and the formula is as follows: Among them, β base Indicates the basic gamma value; L indicates the ambient light intensity; L ref It is represented as reference light intensity; k is represented as adjustment coefficient; Use the contrast stretching algorithm to enhance the contrast of the image. The formula is as follows: Among them, V in Represents the input pixel value; min and max represent the current minimum and maximum pixel values ​​of the image; newMin and newMax represent the minimum and maximum values ​​of the target contrast range; The target ranges newMin and newMax are dynamically adjusted according to the ambient light intensity L. The adjustment formula is as follows: newMin=a×log(L)+b newMax=c×L 0.5 +d Wherein, L represents the current ambient light intensity; a, b, c, d represent empirical parameters obtained by fitting a large amount of experimental data; Apply the local adaptive histogram equalization algorithm to perform histogram equalization in a small area of ​​the image and limit contrast amplification to avoid excessive noise amplification; The local adaptive histogram equalization algorithm divides the image into 8×8 blocks. The index of the divided blocks is (i, j). Each block is represented by B ij , calculate each small block B ij The histogram H ij , using contrast limitation, redistribute the part that exceeds the limit to keep the overall pixel value unchanged. The contrast limitation formula is as follows: Among them, μ ij Represented as a small block B ij The histogram mean of ij Expressed as standard deviation; H ij (x) is represented as the histogram of the number of pixels with pixel intensity x in the (i, j)th small square of the image; H′ ij Represented as a restricted histogram; Calculate the cumulative distribution function F of each restricted histogram ij , the formula is as follows: Among them, l represents the pixel intensity level in the histogram; Pixel mapping is performed according to the cumulative distribution function to convert the original pixel value into a new pixel value to enhance the contrast. The formula is as follows: V out (x,y)=F -1 (V in (x,y)) Among them, V in (x, y) represents the pixel value of the original image at position (x, y); V out (x, y) represents the pixel value after mapping; F -1 Expressed as the inverse of the cumulative distribution function; In scenes with extremely different lighting conditions, high dynamic range synthesis technology is used for image processing; The A-pillar exterior camera captures multiple images in rapid succession with different exposure settings, denoted as I i , where i represents the image index at different exposure levels; the final image I is synthesized using the exposure fusion algorithm final , the formula is as follows: Among them, I i represents an image captured under a specific exposure setting; w i Represented as I i The weight of .

6. The A-pillar perspective and QR code positioning method based on vehicle-mounted AR glasses according to claim 3, characterized in that: The real-time performance optimization includes: Use GPU to optimize performance and accelerate algorithms; use pipeline processing to process multiple frames of images simultaneously; use lookup tables to precalculate some complex functions to accelerate algorithm processing.

7. An A-pillar perspective system based on vehicle-mounted AR glasses, characterized in that: The system includes an on-board AR glasses module, an A-pillar external camera module, a QR code positioning module, a spatial computing module, a system integration control unit and other vehicle systems; The vehicle-mounted AR glasses module includes a display unit, a data transmission interface, a head tracking unit and an image acquisition unit; the A-pillar external camera module includes an image sensor, an image processing chip, an anti-shake unit and a data transmission interface; the spatial calculation module includes a perspective transformation unit, a distortion correction unit, an image enhancement unit and a real-time rendering unit; the system integration control unit includes a data processing unit, a real-time control unit and a vehicle interface unit; the two-dimensional code positioning module includes an image preprocessing unit, a two-dimensional code recognition algorithm unit, a position information decoding unit and an error correction unit; The A-pillar external camera module is used to capture the road conditions outside the A-pillar in real time and transmit the collected video data to the system integration control unit; the QR code positioning module is used to accurately locate the position of the A-pillar in the driver's field of view and transmit the position information to the system integration control unit; the spatial calculation module is used to ensure that the projected image matches the actual viewing angle, send a data processing request to the system integration control unit, and receive feedback on the processing results of the system integration control unit; the on-board AR glasses module is used to detect whether the driver's line of sight moves to the A-pillar position and complete data interaction with the system integration control unit; the system integration control module is responsible for coordinating the work of each module and exchanging data with other vehicle systems.

8. The A-pillar perspective system based on vehicle-mounted AR glasses according to claim 7, characterized in that: The display unit is used to display the processed image or video to the driver; the data transmission interface realizes data exchange between the AR glasses and the vehicle-mounted system; the head tracking unit detects the movement of the driver's line of sight by implementing tracking technology, especially when the line of sight moves to the A-pillar position; the image acquisition unit is used to capture the scene of the driver's surrounding environment; The image sensor is used to capture the road conditions outside the A-pillar; the image processing chip is used to perform spatial calculations on the captured image; the anti-shake unit is used to ensure that the image captured by the camera is stable and clear; the data transmission interface transmits the image data to the vehicle-mounted system for further processing; The image preprocessing unit is used to correct the distortion of the image captured by the AR glasses camera; the two-dimensional code recognition algorithm recognizes the two-dimensional code mark on the A-pillar, so as to determine the position of the A-pillar in the field of view; the position information decoding unit is used to extract the position information from the recognized two-dimensional code; the error correction unit corrects the errors that may be generated in the two-dimensional code recognition process through the Reed-Solomon error correction algorithm; The perspective transformation unit uses a homography matrix to perform perspective transformation of the image to ensure that the image matches the driver's perspective; the distortion correction unit applies the Brown-Conrady model to perform distortion correction on the image; the image enhancement unit performs gamma correction, contrast stretching algorithm and local adaptive histogram equalization algorithm to optimize image quality; the real-time rendering unit renders the processed image in real time on the display unit of the AR glasses; The data processing unit receives data from each module for centralized processing and analysis; The real-time control unit adjusts the system status in real time according to the processing results to ensure that the AR display is synchronized with the actual driving situation; The vehicle interface unit is connected to the real-time control unit and is used to obtain vehicle status information.

9. The A-pillar perspective system based on vehicle-mounted AR glasses according to claim 8, characterized in that: The vehicle interface unit monitors the current vehicle speed in real time; when the vehicle speed is lower than a preset threshold, the system integration control unit automatically activates the system; when the vehicle speed is higher than the preset threshold, the system integration control unit automatically shuts down the system.

10. The A-pillar perspective system based on vehicle-mounted AR glasses according to claim 8, characterized in that: The real-time control unit monitors the ambient light conditions in real time. When it detects that the vehicle is in a low-light environment, the image enhancement unit automatically adjusts the brightness and contrast of the image and transmits the adjusted image to the vehicle-mounted AR glasses module.

Images