AR-HUD-based motion compensation method, device, equipment and storage medium

Abstract

The present application provides a motion compensation method, device, equipment and storage medium based on AR‑HUD. The method includes: determining the relative motion trajectory of the projection object at the reference position origin of the vehicle according to the historical motion trajectory of the projection object in the AR‑HUD and the historical driving trajectory of the vehicle; predicting the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory; using the position of the vehicle at the current projection moment as the new reference position origin, compensating the relative position information, and obtaining the current projection position of the projection object. The present application realizes the projection compensation of the projection object under the motion of the vehicle, prevents the problem of offset of the projection position due to the motion of the vehicle, improves the actual fit of the projection object in the AR‑HUD after projection, and does not require high computing power, so that on the basis of ensuring the projection accuracy of the projection object in the AR‑HUD, the projection calculation overhead in the AR‑HUD is greatly reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a motion compensation method, device, equipment and storage medium based on AR-HUD. Background Art

[0002] With the rapid development of augmented reality (AR) and head-up display (HUD), existing AR-HUD products will be used on vehicles to combine vehicle information and surrounding environment information to form corresponding virtual images of various projection object information, and calculate the corresponding projection position to project it to the corresponding position in front of the driver, so that the driver can focus on driving.

[0003] However, since the vehicle may continue to be in an irregular driving state, and it takes a certain amount of time to generate a virtual image, when the virtual image is actually projected, the projection position will be offset due to the movement of the vehicle, resulting in a mismatch between the virtual image of the projected object and the actual position. At present, a large amount of driving-related information of the projection object is usually collected, and the corresponding computing power is improved to accurately analyze the real projection position of the projection object through a large amount of computing and resources. However, this method will generate a lot of computing overhead, greatly reducing the projection computing efficiency of AR-HUD. Summary of the invention

[0004] The present application provides a motion compensation method, device, equipment and storage medium based on AR-HUD. By analyzing the relative motion between the projection object and the vehicle, the position of the vehicle is used to perform motion compensation on the projection position of the projection object, so as to ensure the projection accuracy of the projection object in the AR-HUD, prevent the problem of offset of the projection position due to the movement of the vehicle, and improve the actual fit of the projection object in the AR-HUD after projection.

[0005] In a first aspect, an embodiment of the present application provides a motion compensation method based on AR-HUD, the method comprising:

[0006] According to the historical motion trajectory of the projection object in the augmented reality head-up display AR-HUD and the historical driving trajectory of the vehicle, determining the relative motion trajectory of the projection object under the reference position origin of the vehicle;

[0007] According to the relative motion trajectory, predict the relative position information of the projection object at the reference position origin at the current projection moment;

[0008] The vehicle position at the current projection moment is used as a new reference position origin, the relative position information is compensated, and the current projection position of the projection object is obtained.

[0009] In a second aspect, an embodiment of the present application provides a motion compensation device based on AR-HUD, the device comprising:

[0010] A relative trajectory determination module is used to determine the relative motion trajectory of the projection object at the reference position origin of the vehicle according to the historical motion trajectory of the projection object in the augmented reality head-up display AR-HUD and the historical driving trajectory of the vehicle;

[0011] A relative position prediction module, used to predict the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory;

[0012] The motion compensation module is used to use the vehicle position at the current projection moment as a new reference position origin, compensate the relative position information, and obtain the current projection position of the projection object.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0014] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the AR-HUD-based motion compensation method provided in the first aspect of the present application.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the AR-HUD-based motion compensation method provided in the first aspect of the present application.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the AR-HUD-based motion compensation method provided in the first aspect of the present application is implemented.

[0017] The embodiments of the present application provide a motion compensation method, device, equipment and storage medium based on AR-HUD, which sets the reference position origin of the vehicle in advance, determines the relative motion trajectory of the projection object at the reference position origin according to the historical motion trajectory of the projection object in the AR-HUD and the historical driving trajectory of the vehicle, and predicts the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory, so as to analyze the relative motion between the projection object and the vehicle at the current projection moment, and then uses the position of the vehicle at the current projection moment as the new reference position origin to compensate for the relative position information and obtain the current projection position of the projection object, thereby realizing the projection compensation of the projection object under the movement of the vehicle, preventing the problem of offset of the projection position due to the movement of the vehicle, improving the actual fit of the projection object in the AR-HUD after projection, and eliminating the need for high computing power requirements, so that on the basis of ensuring the projection accuracy of the projection object in the AR-HUD, the projection calculation overhead in the AR-HUD is greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a flow chart of a motion compensation method based on AR-HUD shown in an embodiment of the present application;

[0020] Figure 2 This is a flowchart of another AR-HUD-based motion compensation method shown in an embodiment of the present application;

[0021] Figure 3 This is a schematic diagram of the principle of the relative position transformation process between the vehicle and the projection object shown in the embodiment of the present application;

[0022] Figure 4 This is a principle block diagram of a motion compensation device based on AR-HUD shown in an embodiment of the present application;

[0023] Figure 5 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0026] In order to solve the problem of projection offset caused by irregular movement of the vehicle when AR-HUD projects various relevant driving references in the vehicle's surrounding environment during vehicle driving, the embodiment of the present application designs a solution for calculating and compensating the projection position of the projection object using the real-time motion state of the vehicle, by transforming the motion trajectory predicted by the projection object in the world coordinate system into the relative coordinate system of the real-time driving state of the vehicle to separate the driving trajectory of the vehicle and the relative motion trajectory of the projection object, thereby realizing real-time projection compensation of the projection object under the movement of the vehicle, preventing the problem of offset of the projection position caused by the movement of the vehicle, improving the actual fit of the projection object in the AR-HUD after projection, and without the requirement of high computing power, so that on the basis of ensuring the projection accuracy of the projection object in the AR-HUD, the projection calculation overhead in the AR-HUD is greatly reduced.

[0027] Figure 1 This is a flow chart of a motion compensation method based on AR-HUD shown in an embodiment of the present application. Figure 1 , the method may specifically include the following steps:

[0028] S110, determining the relative motion trajectory of the projection object at the reference position origin of the vehicle according to the historical motion trajectory of the projection object in the AR-HUD and the historical driving trajectory of the vehicle.

[0029] Specifically, AR-HUD, as a normal display implemented with AR technology, can generate a virtual image containing various driving-related information in combination with the vehicle driving information and the surrounding driving environment during the driving process of the vehicle, and project it at the corresponding position in front of the driver to prevent the driver from looking around during the driving process of the vehicle and ensure driving safety. Among them, the projection objects in this application are mainly various driving reference objects in the driving environment around the vehicle specified in AR-HUD that have relative position display requirements with the vehicle, such as but not limited to the front vehicle, pedestrians, obstacles, lane lines, navigation destinations, intersection guide points and road guide lines during the driving process of the vehicle.

[0030] Considering that when AR-HUD displays the virtual images of each projection object in front of the vehicle, the virtual images of each projection object are viewed from the perspective of the real-time driving state of the vehicle. Therefore, in order to accurately analyze the projection position of each projection object, this application needs to analyze the relative motion state of each projection object relative to the vehicle.

[0031] At this time, by using various data interaction detections between the vehicle and various projection objects during driving, such as radar, camera or navigation analysis and other interactive devices, the various position information of each projection object during the historical driving process of the vehicle can be collected, so as to determine the historical motion trajectory of the projection object. Among them, the historical motion trajectory is a trajectory determined in the world coordinate system. At this time, the projection object can be always in a fixed position in the world coordinate system during the driving process of the vehicle, such as obstacles around the vehicle and fixed facilities such as intersection guide points, or it can be constantly changing, such as the front vehicle and pedestrians when the vehicle is driving, so that the historical motion trajectory of the projection object can exist in multiple states.

[0032] Moreover, during the driving process of the vehicle, the current driving position of the vehicle can be determined in real time through the vehicle sensors, speed sensors, radars, cameras, accelerometers, gyroscope sensors, Global Positioning System (GPS) modules and Dead Reckoning (DR) sensors installed on the vehicle, thereby obtaining the historical driving trajectory of the vehicle.

[0033] Then, when analyzing the relative motion state of the projection object relative to the vehicle, it is considered that the relative motion position of the projection object at the same historical moment in the historical motion trajectory will change with the driving position of the projection object relative to the vehicle at different historical moments. For example, taking the driving position of the vehicle at time t1 as the origin of the relative motion analysis, compared with taking the driving position of the vehicle at time t2 as the origin of the relative motion analysis, the relative motion position of the projection object at the same historical moment is different, so that the relative motion trajectory determined by the projection relative to the driving position of the vehicle at times t1 and t2 will also be different. Therefore, in order to uniquely analyze the relative motion state of the projection object with respect to the vehicle, in the present application, the driving position of the vehicle at any historical moment can be used as the reference position origin of the vehicle, and then the motion position at each historical moment in the historical motion trajectory of the projection object is analyzed, and the relative motion position relative to the reference position origin of the vehicle is combined. The relative motion position of the projection object at each historical moment determined at the reference position origin of the vehicle can be obtained to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle, so as to predict the subsequent motion state of the projection object at the reference position origin of the vehicle.

[0034] S120, predicting the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory.

[0035] Specifically, since AR-HUD needs to project the virtual image of the projection object in real time while the vehicle is driving, it is necessary to analyze the actual projection position of the projection object in real time. Therefore, after obtaining the relative motion trajectory of the projection object at the reference position origin of the vehicle, the position information at the current projection moment can be directly predicted from the relative motion trajectory as the relative position information of the projection object relative to the reference position origin of the vehicle at the current projection moment, so as to subsequently analyze the position information of the projection object relative to the position of the vehicle at the current projection moment, so as to use the real-time motion of the vehicle to perform motion compensation on the projection object.

[0036] S130, taking the vehicle position at the current projection moment as a new reference position origin, compensating the relative position information, and obtaining the current projection position of the projection object.

[0037] Considering that when AR-HUD displays the virtual images of each projected object in front of the vehicle, it is viewing the virtual images of each projected object from the perspective of the real-time driving status of the vehicle. Therefore, after obtaining the relative position information of the projected object relative to the reference position origin of the vehicle at the current projection moment, it is necessary to further analyze the relative motion position of the projected object at the current projection moment relative to the position of the vehicle at the current projection moment.

[0038] Specifically, the vehicle position at the current projection moment can be obtained through the vehicle sensors, speed sensors, radars, cameras, accelerometers, gyroscope sensors, GPS modules, and DR dead reckoning sensors installed on the vehicle. Then, the vehicle position at the current projection moment is used as the new reference position origin, and the relative position information of the projection object at the previous reference position origin at the current projection moment is transformed to the new reference position origin, thereby compensating the relative position information of the projection object at the current projection moment, obtaining the relative position information of the projection object at the new reference position origin, and using this to analyze the current projection position of the projection object.

[0039] It should be noted that the current projection position of the projection object can be the relative position information of the projection object at the new reference position origin obtained by motion compensation of the relative position information of the projection object at the current projection moment, and the projection object is subsequently projected according to the relative position information and the position of the vehicle at the current projection moment. On the other hand, after motion compensation is performed on the relative position information of the projection object at the current projection moment to obtain the relative position information of the projection object at the new reference position origin, the relative position information of the projection object at the new reference position origin at the current projection moment can also be inversely transformed according to the coordinate conversion relationship existing when the position of the vehicle at the current projection moment is used as the new reference position origin, so as to obtain the position information of the projection object in the original world coordinate system, and use it as the current projection position of the projection object, and the projection object can be projected directly according to the current projection position.

[0040] Exemplarily, in order to accurately analyze the current projection position of the projection object under the new reference position origin, the present application can specifically perform the following operations: transform the position of the vehicle at the current projection moment to the reference position origin of the vehicle, and obtain the relative driving position of the vehicle at the reference position origin at the current projection moment; use the position of the vehicle at the current projection moment as the new reference position origin, and use the relative driving position to determine the position transformation relationship at the current projection moment under the new reference position origin; use the position transformation relationship and relative position information at the current projection moment to calculate the current projection position of the projection object.

[0041] That is to say, since the relative position information of the projection object at the reference position origin at the current projection moment has been obtained, when transforming to the new reference position origin again to analyze the current projection position, it is first necessary to analyze the position transformation relationship between the coordinate system represented by the previous reference position origin and the new reference position origin.

[0042] Therefore, first, the position of the vehicle in the original world coordinate system at the current projection moment is transformed to the coordinate system represented by the previous reference position origin, and the relative driving position of the vehicle at the reference position origin at the current projection moment is obtained. Then, when the position of the vehicle at the current projection moment is used as the new reference position origin, the relative driving position of the vehicle at the previous reference position origin and the origin position coordinates at the new reference position origin can be used to determine the position transformation relationship corresponding to the current projection moment when the previous reference position origin is transformed to the new reference position origin. Finally, according to the position transformation relationship at the current projection moment, the relative position information of the projection object at the previous reference position origin at the current projection moment can be transformed to the new reference position origin, and the relative position information of the projection object at the new reference position origin at the current projection moment is obtained, thereby further determining the current projection position of the projection object.

[0043] The technical solution provided by the embodiment of the present application sets the reference position origin of the vehicle in advance, determines the relative motion trajectory of the projection object at the reference position origin according to the historical motion trajectory of the projection object in the AR-HUD and the historical driving trajectory of the vehicle, and predicts the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory, so as to analyze the relative motion between the projection object and the vehicle at the current projection moment, and then uses the position of the vehicle at the current projection moment as the new reference position origin to compensate for the relative position information and obtain the current projection position of the projection object, thereby realizing the projection compensation of the projection object under the movement of the vehicle, preventing the problem of offset of the projection position due to the movement of the vehicle, improving the actual fit of the projection object in the AR-HUD after projection, and eliminating the need for high computing power requirements, so that on the basis of ensuring the projection accuracy of the projection object in the AR-HUD, the projection calculation overhead in the AR-HUD is greatly reduced.

[0044] As an optional implementation scheme in the embodiment of the present application, in order to ensure the accuracy of motion compensation for the projection position of the projection object using the real-time motion state of the vehicle, the present application describes in detail the specific transformation process of the historical motion trajectory of the projection object to the relative motion trajectory at the reference position origin of the vehicle.

[0045] Figure 2 FIG. 1 is a flow chart of another motion compensation method based on AR-HUD shown in an embodiment of the present application. Figure 2 As shown, the method may specifically include the following steps:

[0046] S210, transforming the historical driving trajectory of the vehicle to the reference position origin of the vehicle, and determining the position transformation relationship at each historical moment.

[0047] Optionally, since the historical driving trajectory of the vehicle and the historical motion trajectory of the projection object are both determined in the original world coordinate system, when analyzing the relative motion trajectory of the projection object relative to the vehicle, it is first necessary to select a position information from the historical driving trajectory of the vehicle as the reference position origin of the vehicle, so as to analyze the relative motion trajectory of the projection object relative to the vehicle at each historical moment.

[0048] Among them, in this application, the reference position origin of the vehicle can be determined by the following operations: from the historical driving trajectory of the vehicle, the historical driving position of the vehicle at a preset historical moment is screened out; and the historical driving position is used as the reference position origin of the vehicle.

[0049] That is to say, considering that the historical driving trajectory of the vehicle is composed of the driving positions of the vehicle at various historical moments, the relative motion position of the projection object at the same historical moment relative to the driving position of the vehicle at different historical moments is not the same. Therefore, the present application needs to filter out the historical driving position of the vehicle at a preset historical moment from the historical driving trajectory of the vehicle, and the preset historical moment can be any historical moment in the historical driving trajectory of the vehicle. Then, the filtered historical driving position of the vehicle is used as the reference position origin of the vehicle, and the subsequent analysis of the relative motion trajectory of the projection object relative to the vehicle is compared and analyzed with the reference position origin.

[0050] After determining the reference position origin of the vehicle, the driving positions at each historical moment contained in the historical driving trajectory of the vehicle will also change relatively. Therefore, the historical driving trajectory of the vehicle can be transformed to the reference position origin of the vehicle, that is, the historical driving trajectory of the vehicle in the world coordinate system is transformed to the relative coordinate system represented by the reference position origin of the vehicle. Then, according to the position information of the vehicle before and after the transformation at each historical moment, the position transformation relationship at each historical moment can be determined, so that the position transformation relationship at each historical moment can be used later to perform motion transformation on the historical motion trajectory of the projection object relative to the vehicle.

[0051] Exemplarily, the position transformation relationship at each historical moment under the reference position origin of the vehicle is analyzed. Specifically, it can be as follows: according to the reference position origin of the vehicle, the historical driving position at each historical moment in the historical driving trajectory of the vehicle is transformed to obtain the relative driving position of the vehicle at the reference position origin at each historical moment; according to the historical driving position of the vehicle at each historical moment and the relative driving position of the vehicle at the reference position origin at that historical moment, the position transformation relationship at each historical moment is calculated.

[0052] For example, the historical driving position of the vehicle at each historical moment in the historical driving trajectory can be expressed as t1: v1 (x1, y1), t2: v2 (x2, y2) and t3: v3 (x3, y3), etc. After t1: v1 (x1, y1) is used as the reference position origin of the vehicle, the relative driving position of the vehicle at the reference position origin at each historical moment can be expressed as t1: Rv1 (0, 0), t2: Rv2 (Rx2, Ry2) and t3: Rv3 (Rx3, Ry3), etc. Then, using the historical driving position of the vehicle at each historical moment and the relative driving position after transformation at the historical moment, the position transformation relationship at each historical moment can be obtained respectively, such as the coordinate system transformation matrix M1, M2, M3, etc. at each historical moment.

[0053] Taking the scene of the preceding vehicle warning as an example, the projection object in the present application can be each vehicle in front of the vehicle during its driving process. At the same time, the historical driving positions t1: p1 (x1, y1), t2: p2 (x2, y2) and t3: p3 (x3, y3) at each historical moment in the historical motion trajectory of the preceding vehicle can be obtained, as well as the historical driving positions t1: v1 (x1, y1), t2: v2 (x2, y2) and t3: v3 (x3, y3) at each historical moment in the historical driving trajectory of the vehicle.

[0054] Then, if Figure 3 As shown, taking the vehicle's t1: v1 (x1, y1) as the reference position origin, the relative driving positions of the vehicle at the reference position origin, t1: Rv1 (0, 0), t2: Rv2 (Rx2, Ry2) and t3: Rv3 (Rx3, Ry3), etc., can be obtained, thereby creating a relative coordinate system represented by the reference position origin of the vehicle. Then, using the driving position information before and after the vehicle is transformed from the world coordinates to the relative coordinate system at each historical moment, the position transformation relationship M1, M2, M3, etc. at each historical moment can be obtained.

[0055] S220, according to the position transformation relationship at each historical moment, transform the historical motion trajectory of the projection object to the reference position origin of the vehicle, and obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle.

[0056] Optionally, after determining the position transformation relationship when transforming from the original world coordinate system to the relative coordinate system represented by the reference position origin of the vehicle at each historical moment, the motion position information at each historical moment contained in the historical motion trajectory of the projection object can be transformed into a coordinate system according to the position transformation relationship at each historical moment, thereby obtaining the relative motion trajectory of the projection object at the reference position origin of the vehicle.

[0057] Specifically, the historical motion trajectory of the projection object is transformed to the reference position origin of the vehicle, and the following operations can be specifically performed: according to the historical motion position at each historical moment in the historical motion trajectory of the projection object and the position transformation relationship at the historical moment, the relative motion position of the projection object at the reference position origin at each historical moment is determined; the relative motion position at each historical moment is curve fitted to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle.

[0058] That is to say, firstly, the historical motion position of the projection object at each historical moment is determined from the historical motion trajectory of the projection object. Then, for each historical moment, the position transformation relationship when the world coordinate system is transformed into the relative coordinate system represented by the reference position origin of the vehicle at the historical moment is used to transform the historical motion position of the projection object at the historical moment, and obtain the relative motion position of the projection object at the reference position origin of the vehicle at the historical moment. By performing the same steps at each historical moment, the relative motion position of the projection object at the reference position origin of the vehicle at each historical moment can be obtained. Finally, by curve fitting the relative motion position of the projection object at each historical moment, the relative motion trajectory of the projection object at the reference position origin of the vehicle can be obtained.

[0059] For example, taking the scenario of the preceding vehicle warning as an example, Figure 3 As shown, after obtaining the position transformation relations M1, M2, M3, etc. at each historical moment, for the historical driving positions t1: p1(x1, y1), t2: p2(x2, y2) and t3: p3(x3, y3) at each historical moment in the historical motion trajectory of the front vehicle, p'=M*p can be executed at each historical moment, so as to obtain the relative motion position t1: p1'(x1', y1'), t2: p2'(x2', y2') and t3: p3'(x3', y3') of the front vehicle at the reference position origin of the vehicle at each historical moment, and then according to each historical moment, curve fitting is performed on each relative motion position of the front vehicle to generate corresponding curve fitting equations Fx(t, x)=0 and Fy(t, y)=0 as the relative motion trajectory of the front vehicle at the reference position origin of the vehicle.

[0060] Subsequently, according to the relative motion trajectory of the vehicle in front at the reference position origin of this vehicle, the current projection moment can be input into Fx(t, x)=0 and Fy(t, y)=0, thereby predicting the relative position information of the projection object at the reference position origin at the current projection moment.

[0061] At this time, if Figure 3As shown, if the relative driving position of the vehicle position at the current projection moment at the reference position origin of the vehicle is expressed as t: Rv(Rx, Ry), and the predicted relative position information of the projection object at the reference position origin at the current projection moment is expressed as t: p'(x', y'), then according to the relative driving position t: Rv(Rx, Ry) of the vehicle at the reference position origin at the current projection moment and t: Rv(0, 0) expressed when the vehicle position at the current projection moment is used as the new reference position origin, the position transformation relationship M when the relative coordinate system represented by the previous reference position origin is transformed to the relative coordinate system represented by the new reference position origin at the current projection moment can be determined, and then the relative position information of the projection object after the relative coordinate system represented by the previous reference position origin is transformed to the relative coordinate system represented by the new reference position origin at the current projection moment is obtained as p=M*p', thereby determining the current projection position of the projection object.

[0062] In addition, considering that when analyzing the historical motion trajectory of the projection object, it is necessary to exchange data between the vehicle and the projection object to obtain the relevant information of the motion position of the projection object at each historical moment, there will be a certain transmission delay Δt1 in data acquisition and data reception, and the reception time of each position data of the projection object is usually used as the corresponding acquisition time, which leads to a certain difference between the historical motion trajectory of the projection object and the actual motion trajectory. Moreover, after inputting the current projection moment, it is necessary to further predict the relative position information of the projection object at the reference position origin at the current projection moment, and when the position of the vehicle at the current projection moment is used as the new reference position origin, the relative position information of the projection object at the previous reference position origin is compensated to calculate the current projection position of the projection object, and finally the actual projection operation is performed, that is, there is also a certain calculation delay Δt2 between inputting the current projection moment and performing the actual projection operation, so that when performing the actual projection operation, the vehicle has continued to travel a distance in the calculation delay Δt2 stage, resulting in an offset in the projection position.

[0063] Therefore, the present application needs to avoid the positional offset effects of the above two delays on the actual projection of the projection object. For example, by analyzing the data transmission frequency between the projection object and the vehicle and the calculation delay when the projection object coordinates are transformed, or by using the subsequent real delay to continuously correct the delay of the previous moment, the estimation method or the learning comparison method can be used to determine that there are two delays Δt1 and Δt2 when the AR-HUD actually projects the projection object. Then, when analyzing the historical motion trajectory of the projection object, the reception time of each position data can be subtracted from the transmission delay Δt1 as the real acquisition time of the position data, and then the historical motion trajectory of the projection object can be accurately obtained. In addition, when the current projection moment is input to calculate the current projection position of the projection object, the current input moment can be added with the calculation delay Δt2 as the current projection moment, that is, the projection position of the projection object at a future moment is calculated in advance at the current moment, and then the actual projection operation is directly performed on the projection object at that moment to avoid the influence of the calculation delay.

[0064] S230: predicting the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory.

[0065] S240: Taking the vehicle position at the current projection moment as a new reference position origin, the relative position information is compensated to obtain the current projection position of the projection object.

[0066] The technical solution provided by the embodiment of the present application sets the reference position origin of the vehicle in advance, and by analyzing the position transformation relationship when transforming from the original world coordinate system to the relative coordinate system represented by the reference position origin of the vehicle at each historical moment, the historical motion trajectory of the projection object can be transformed to the reference position origin of the vehicle, and the relative motion trajectory of the projection object at the reference position origin is obtained. According to the relative motion trajectory, the relative position information of the projection object at the reference position origin at the current projection moment is predicted to analyze the relative motion between the projection object and the vehicle at the current projection moment, and then the position of the vehicle at the current projection moment is used as the new reference position origin, and the relative position information is compensated to obtain the current projection position of the projection object, thereby realizing the projection compensation of the projection object under the movement of the vehicle, preventing the problem of offset of the projection position due to the movement of the vehicle, and improving the actual fitting degree of the projection object in the AR-HUD after projection, and there is no need to have high computing power requirements, so that on the basis of ensuring the projection accuracy of the projection object in the AR-HUD, the projection calculation overhead in the AR-HUD is greatly reduced.

[0067] Figure 4 FIG. 1 is a block diagram of a motion compensation device based on AR-HUD shown in an embodiment of the present application. Figure 4 As shown, the apparatus 400 may include:

[0068] The relative trajectory determination module 410 is used to determine the relative motion trajectory of the projection object at the reference position origin of the vehicle according to the historical motion trajectory of the projection object in the augmented reality head-up display AR-HUD and the historical driving trajectory of the vehicle;

[0069] A relative position prediction module 420, configured to predict the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory;

[0070] The motion compensation module 430 is used to use the vehicle position at the current projection moment as a new reference position origin, compensate the relative position information, and obtain the current projection position of the projection object.

[0071] Furthermore, the relative trajectory determination module 410 may include:

[0072] A vehicle position transformation unit, used to transform the historical driving trajectory of the vehicle to the reference position origin of the vehicle, and determine the position transformation relationship at each historical moment;

[0073] The projection object transformation unit is used to transform the historical motion trajectory of the projection object to the reference position origin of the vehicle according to the position transformation relationship at each historical moment, so as to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle.

[0074] Furthermore, the AR-HUD-based motion compensation device 400 may further include:

[0075] The position origin determination module is used to filter out the historical driving position of the vehicle at a preset historical moment from the historical driving trajectory of the vehicle; and use the historical driving position as the reference position origin of the vehicle.

[0076] Furthermore, the vehicle position transformation unit may be specifically used for:

[0077] According to the reference position origin of the vehicle, the historical driving position of the vehicle at each historical moment in the historical driving trajectory is transformed to obtain the relative driving position of the vehicle at the reference position origin at each historical moment;

[0078] According to the historical driving position of the vehicle at each historical moment and the relative driving position of the vehicle at the reference position origin at the historical moment, the position transformation relationship at each historical moment is calculated.

[0079] Furthermore, the projection object transformation unit may be specifically used for:

[0080] Determine the relative motion position of the projection object at each historical moment under the reference position origin according to the historical motion position at each historical moment in the historical motion trajectory of the projection object and the position transformation relationship at the historical moment;

[0081] The relative motion position at each of the historical moments is subjected to curve fitting to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle.

[0082] Furthermore, the motion compensation module 430 may be specifically used for:

[0083] The position of the vehicle at the current projection moment is transformed to the reference position origin of the vehicle to obtain the relative driving position of the vehicle at the reference position origin at the current projection moment;

[0084] Taking the vehicle position at the current projection moment as a new reference position origin, and using the relative driving position, determining the position transformation relationship at the current projection moment under the new reference position origin;

[0085] The current projection position of the projection object is calculated by using the position transformation relationship at the current projection moment and the relative position information.

[0086] Furthermore, the historical motion trajectory of the projection object and the historical driving trajectory of the vehicle are determined by using at least one of the following sensing devices:

[0087] Vehicle sensors, speed sensors, radars, cameras, accelerometers, gyroscope sensors, global positioning system GPS modules and dead reckoning DR sensors.

[0088] In the embodiment of the present application, the reference position origin of the vehicle is set in advance, and the relative motion trajectory of the projection object at the reference position origin is determined according to the historical motion trajectory of the projection object in the AR-HUD and the historical driving trajectory of the vehicle. According to the relative motion trajectory, the relative position information of the projection object at the reference position origin at the current projection moment is predicted to analyze the relative motion between the projection object and the vehicle at the current projection moment. Then, the position of the vehicle at the current projection moment is used as the new reference position origin, and the relative position information is compensated to obtain the current projection position of the projection object, thereby realizing the projection compensation of the projection object under the movement of the vehicle, preventing the problem of offset of the projection position due to the movement of the vehicle, and improving the actual fitting degree of the projection object in the AR-HUD after projection. Moreover, there is no need for high computing power requirements, so that on the basis of ensuring the projection accuracy of the projection object in the AR-HUD, the projection calculation overhead in the AR-HUD is greatly reduced.

[0089] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 4 The device 400 shown can execute any method embodiment provided in the present application, and the aforementioned and other operations and / or functions of each module in the device 400 are respectively for implementing the corresponding processes in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0090] The above describes the device 400 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0091] Figure 5 It is a schematic block diagram of an electronic device 500 provided in an embodiment of the present application.

[0092] like Figure 5 As shown, the electronic device 500 may include:

[0093] The memory 510 and the processor 520, the memory 510 is used to store the computer program and transmit the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the method in the embodiment of the present application.

[0094] For example, the processor 520 may be configured to execute the above method embodiments according to instructions in the computer program.

[0095] In some embodiments of the present application, the processor 520 may include but is not limited to:

[0096] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0097] In some embodiments of the present application, the memory 510 includes but is not limited to:

[0098] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0099] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0100] like Figure 5 As shown, the electronic device may also include:

[0101] The transceiver 530 may be connected to the processor 520 or the memory 510 .

[0102] The processor 520 may control the transceiver 530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of antennas may be one or more.

[0103] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0104] The embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the embodiment of the present application also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0105] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0106] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0108] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0109] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A motion compensation method based on AR-HUD, characterized in that: include: According to the historical motion trajectory of the projection object in the augmented reality head-up display AR-HUD and the historical driving trajectory of the vehicle, determining the relative motion trajectory of the projection object under the reference position origin of the vehicle; According to the relative motion trajectory, predict the relative position information of the projection object at the reference position origin at the current projection moment; Taking the position of the vehicle at the current projection moment as a new reference position origin, compensating the relative position information, and obtaining the current projection position of the projection object; The method of determining the relative motion trajectory of the projection object at the reference position origin of the vehicle according to the historical motion trajectory of the projection object in the augmented reality head-up display AR-HUD and the historical driving trajectory of the vehicle includes: Transforming the historical driving trajectory of the vehicle to the reference position origin of the vehicle, and determining the position transformation relationship at each historical moment; According to the position transformation relationship at each historical moment, the historical motion trajectory of the projection object is transformed to the reference position origin of the vehicle, so as to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle; The process of using the vehicle position at the current projection moment as a new reference position origin, compensating the relative position information, and obtaining the current projection position of the projection object includes: The position of the vehicle at the current projection moment is transformed to the reference position origin of the vehicle to obtain the relative driving position of the vehicle at the reference position origin at the current projection moment; Taking the vehicle position at the current projection moment as a new reference position origin, and using the relative driving position, determining the position transformation relationship at the current projection moment under the new reference position origin; The current projection position of the projection object is calculated by using the position transformation relationship at the current projection moment and the relative position information.

2. The method according to claim 1, characterized in that: Before transforming the historical driving trajectory of the vehicle to the reference position origin of the vehicle, the method further includes: Filtering out the historical driving position of the vehicle at a preset historical moment from the historical driving trajectory of the vehicle; The historical driving position is used as the reference position origin of the vehicle.

3. The method according to claim 1, characterized in that The step of transforming the historical driving trajectory of the vehicle to the reference position origin of the vehicle and determining the position transformation relationship at each historical moment includes: According to the reference position origin of the vehicle, the historical driving position of the vehicle at each historical moment in the historical driving trajectory is transformed to obtain the relative driving position of the vehicle at the reference position origin at each historical moment; According to the historical driving position of the vehicle at each historical moment and the relative driving position of the vehicle at the reference position origin at the historical moment, the position transformation relationship at each historical moment is calculated.

4. The method according to claim 1, characterized in that: The method of transforming the historical motion trajectory of the projection object to the reference position origin of the vehicle according to the position transformation relationship at each historical moment to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle includes: Determine the relative motion position of the projection object at each historical moment under the reference position origin according to the historical motion position at each historical moment in the historical motion trajectory of the projection object and the position transformation relationship at the historical moment; The relative motion position at each of the historical moments is subjected to curve fitting to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle.

5. The method according to claim 1, characterized in that The historical motion trajectory of the projection object and the historical driving trajectory of the vehicle are determined by using at least one of the following sensing devices: Vehicle sensors, speed sensors, radars, cameras, accelerometers, gyroscope sensors, global positioning system GPS modules and dead reckoning DR sensors.

6. A motion compensation device based on AR-HUD, characterized in that: include: A relative trajectory determination module is used to determine the relative motion trajectory of the projection object at the reference position origin of the vehicle according to the historical motion trajectory of the projection object in the augmented reality head-up display AR-HUD and the historical driving trajectory of the vehicle; A relative position prediction module, used to predict the relative position information of the projection object at the reference position origin at the current projection moment according to the relative motion trajectory; A motion compensation module, used to use the vehicle position at the current projection moment as a new reference position origin, to compensate the relative position information, and obtain the current projection position of the projection object; Wherein, the relative trajectory determination module is specifically used for: Transforming the historical driving trajectory of the vehicle to the reference position origin of the vehicle, and determining the position transformation relationship at each historical moment; According to the position transformation relationship at each historical moment, the historical motion trajectory of the projection object is transformed to the reference position origin of the vehicle, so as to obtain the relative motion trajectory of the projection object at the reference position origin of the vehicle; The motion compensation module is specifically used for: The position of the vehicle at the current projection moment is transformed to the reference position origin of the vehicle to obtain the relative driving position of the vehicle at the reference position origin at the current projection moment; Taking the vehicle position at the current projection moment as a new reference position origin, and using the relative driving position, determining the position transformation relationship at the current projection moment under the new reference position origin; The current projection position of the projection object is calculated by using the position transformation relationship at the current projection moment and the relative position information.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the AR-HUD-based motion compensation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the AR-HUD-based motion compensation method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the motion compensation method based on AR-HUD as described in any one of claims 1 to 5 is implemented.

Images