AR-HUD display method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention relates to an AR-HUD display method, electronic equipment and a computer readable storage medium, and relates to the technical field of vehicles. The AR-HUD display method comprises the steps that relative movement parameters of surrounding objects relative to a current vehicle and AR-HUD display parameters of the current vehicle are acquired; predicting the next moment position of the surrounding object relative to the current vehicle according to the relative movement parameter and the display parameter; and controlling the AR-HUD to display an identifier suitable for representing the surrounding object at the next moment position.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of vehicle technology, and in particular, to an AR-HUD display method, an electronic device, and a computer-readable storage medium. Background Art

[0002] AR-HUD (Augmented Reality Head Up Display) is a technology that combines AR augmented reality technology and HUD head-up display function. AR-HUD superimposes computer-generated virtual information on the real scene to achieve a combination of visual effects and real road elements. It can provide a larger field of view, a longer imaging distance, and richer and more intuitive information display.

[0003] However, the current AR-HUD system has display delays, which affects the user experience. Summary of the invention

[0004] One purpose of the embodiments of the present application is to provide an AR-HUD display method, an electronic device and a computer-readable storage medium, which have the advantage that by obtaining the relative movement parameters of the surrounding objects relative to the current vehicle and the AR-HUD display parameters of the current vehicle, the next moment position of the surrounding objects relative to the current vehicle can be predicted in advance according to the relative movement parameters and the display parameters, and then the AR-HUD can be controlled to display the identification of the surrounding objects at the next moment position, so that the identification of the surrounding objects follows the surrounding objects and moves synchronously with the surrounding objects, thereby avoiding display delays and improving user experience.

[0005] Another purpose of the embodiments of the present application is to provide an AR-HUD display method, an electronic device and a computer-readable storage medium, which have the advantage that the position at the next moment can be calculated as long as at least the current relative speed, the relative position at the previous moment and the display interval duration are obtained. In other words, the electronic device of the vehicle can calculate in advance the position of the surrounding objects relative to the current vehicle at the next moment from the current moment, thereby compensating for the time delay caused by the calculation by calculating the position at the next moment in advance.

[0006] Another purpose of the embodiments of the present application is to provide an AR-HUD display method, an electronic device and a computer-readable storage medium, which have the advantage that by calculating the last actual display time based on the acquired last display time and the pre-measured compensation duration, and calculating the display interval duration based on the last actual display time and the display frequency, the deviation between the actual display time and the theoretical display time determined according to the display frequency can be eliminated, thereby improving the accuracy of the display interval duration.

[0007] Another object of the embodiments of the present application is to provide an AR-HUD display method, an electronic device, and a computer-readable storage medium, which have the advantage of calculating the relative acceleration by utilizing multiple previous relative speeds and the time when the previous relative speeds were obtained, and then calculating the position at the next moment based on the relative acceleration, the current relative speed, the relative position at the previous moment, and the display interval duration, so that the position at the next moment can be more accurate.

[0008] Another purpose of the embodiments of the present application is to provide an AR-HUD display method, an electronic device and a computer-readable storage medium, which have the advantage of determining the relative coordinates of the surrounding objects in a relative coordinate system by acquiring multiple frames of images of the surrounding objects at different times, which not only can directly obtain the relative position of the surrounding objects relative to the current vehicle at the previous moment, in other words, the relative coordinates at the previous moment are the relative position at the previous moment, but also does not require additional auxiliary equipment. Multiple frames of images of the surrounding objects at different times can be directly obtained with the help of existing equipment in the current vehicle, such as a camera, thereby reducing costs.

[0009] Another object of the embodiments of the present application is to provide an AR-HUD display method, an electronic device, and a computer-readable storage medium, which have the advantages that by controlling the AR-HUD to display an identification box at the next moment when the current vehicle speed is greater than a preset vehicle speed threshold, the time spent on obtaining the outline can be saved, thereby further reducing the delay; and by controlling the AR-HUD to display the outline at the next moment when the current vehicle speed is not greater than the preset vehicle speed threshold, the aesthetics can be improved and the user experience can be improved while avoiding display delays.

[0010] Another object of the embodiments of the present application is to provide an AR-HUD display method, an electronic device, and a computer-readable storage medium, which have the advantages that by controlling the AR-HUD to display an identification box at the next moment when the current relative speed is greater than a preset vehicle speed threshold, the time spent on obtaining the outline can be saved, thereby further reducing the delay; and by controlling the AR-HUD to display the outline at the next moment when the current relative speed is not greater than the preset vehicle speed threshold, the aesthetics can be improved and the user experience can be improved while avoiding display delays.

[0011] To achieve at least one of the above-mentioned purposes, according to the first aspect of the present application, an embodiment of the present application provides an AR-HUD display method, including: obtaining relative movement parameters of surrounding objects relative to a current vehicle and display parameters of the AR-HUD of the current vehicle; predicting the next moment position of the surrounding objects relative to the current vehicle based on the relative movement parameters and the display parameters; and controlling the AR-HUD to display an identification suitable for characterizing the surrounding objects at the next moment position.

[0012] In some embodiments, predicting the next moment position of the surrounding objects relative to the current vehicle based on the relative movement parameters and the display parameters includes: determining the display interval duration based on at least the display frequency; and calculating the next moment position based on at least the current relative speed, the relative position at the previous moment, and the display interval duration.

[0013] In some embodiments, the display parameters also include the last display time of the AR-HUD; wherein, determining the display interval duration at least based on the display frequency includes: calculating the last actual display time based on the last display time and a pre-measured compensation duration; predicting the next predicted display time based on the display frequency and the last actual display time; and calculating the display interval duration based on the last actual display time and the next predicted display time.

[0014] In some embodiments, the relative movement parameters also include multiple previous relative speeds; wherein, before calculating the next moment position based on at least the current relative speed, the relative position at the previous moment and the display interval duration, the display method also includes: calculating the relative acceleration based on the multiple previous relative speeds and the moment when the previous relative speed is obtained; wherein, calculating the next moment position based on at least the current relative speed, the relative position at the previous moment and the display interval duration includes: calculating the next moment position based on the relative acceleration, the current relative speed, the relative position at the previous moment and the display interval duration.

[0015] In some embodiments, obtaining the relative movement parameters of surrounding objects relative to the current vehicle includes: establishing a relative coordinate system with the current vehicle as the origin; obtaining the relative coordinates of the surrounding objects in the relative coordinate system at multiple different times; and determining the current relative speed and the relative position at the previous moment based on the multiple relative coordinates and the time when the relative coordinates were obtained.

[0016] In some embodiments, the identification includes an identification box or an outline of the surrounding objects; wherein, controlling the AR-HUD to display an identification suitable for characterizing the surrounding objects at the next moment position includes: obtaining the current vehicle speed of the current vehicle; in response to the current vehicle speed being greater than a preset vehicle speed threshold, controlling the AR-HUD to display the identification box at the next moment position; and in response to the current vehicle speed being not greater than a preset vehicle speed threshold, obtaining the outline and controlling the AR-HUD to display the outline at the next moment position.

[0017] In some embodiments, the relative movement parameter includes a current relative speed; wherein, controlling the AR-HUD to display an identification suitable for characterizing the surrounding objects at the next moment position includes: in response to the current relative speed being greater than a preset vehicle speed threshold, controlling the AR-HUD to display the identification box at the next moment position; and in response to the current relative speed being not greater than a preset vehicle speed threshold, acquiring the outline and controlling the AR-HUD to display the outline at the next moment position.

[0018] According to the second aspect of the present application, an embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the AR-HUD display method described in the first aspect of the present application.

[0019] According to the third aspect of the present application, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the AR-HUD display method described in the first aspect of the present application is implemented.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, purposes and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are used to better understand the present solution and do not constitute a limitation of the present application, wherein:

[0022] Figure 1 is a partial schematic diagram of a vehicle to which the present application may be applied;

[0023] Figure 2 is a flow chart of an AR-HUD display method according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of an AR-HUD projection screen according to the present application; and

[0025] Figure 4 It is a block diagram of an electronic device used to implement the AR-HUD display method of the embodiment of the present application.

[0026] Reference numerals:

[0027] 100, surrounding objects; 110, identification box; 120, text logo; 130, current vehicle;

[0028] 140, windshield; 150, vehicle-mounted camera; 200, electronic equipment;

[0029] 201, computing unit; 202, memory (ROM); 203, memory (RAM);

[0030] 204, bus; 205, I / O interface; 206, input unit; 207, output unit;

[0031] 208. Storage unit; 209. Communication unit. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0033] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] AR-HUD superimposes computer-generated virtual information on the real scene to achieve a combination of visual effects and real road elements. It not only allows users to see relevant information without looking down, but also provides a larger field of view, a longer imaging distance, and a richer and more intuitive information display. In terms of navigation, AR-HUD usually displays indicator signs related to the surrounding objects in front on the windshield of the vehicle. However, since AR-HUD currently uses real-time calculation and real-time display, and there may be time delays in camera imaging, calculation, etc., it is very easy to cause display delays in the indicator signs, resulting in the indicator signs being unable to move synchronously with the surrounding objects. For example, AR-HUD generally presents a tail sign at the rear of the vehicle in front, but when the vehicle in front turns, the tail sign deviates from the vehicle in front due to display delays, thereby affecting the user experience.

[0035] Based on this, in order to solve the above problems, an embodiment of the present application provides an AR-HUD display method. Figure 1 A partial schematic diagram of a vehicle to which the AR-HUD display method of the present application can be applied is shown. Figure 1As shown, the current vehicle 130 may include a windshield 140 and a vehicle-mounted camera 150. The vehicle-mounted camera 150 may capture a road image in front of the current vehicle 130, and the electronic device of the current vehicle 130 may determine the relative movement parameters of the surrounding objects relative to the current vehicle 130 based on the road image captured by the vehicle-mounted camera 150, and the AR-HUD of the current vehicle 130 may display identification corresponding to the surrounding objects in front on the windshield 140 of the vehicle.

[0036] Figure 2 A schematic diagram of a process flow of an embodiment of an AR-HUD display method according to the present application is shown. Figure 3 A schematic diagram of an embodiment of the projection screen of the AR-HUD according to the present application is shown. The AR-HUD display method provided in the embodiment of the present application is generally performed by an electronic device, and the electronic device may be, but is not limited to, a vehicle-mounted electronic device such as a vehicle computer, an AR-HUD device, or a mobile device such as a mobile phone. For example, the following description takes the vehicle computer as an example. Figure 2 and Figure 3 As shown, the AR-HUD display method 1000 includes the following steps:

[0037] S100, obtaining relative movement parameters of the surrounding objects 100 relative to the current vehicle 130 and display parameters of the AR-HUD of the current vehicle 130;

[0038] S200, predicting the next moment position of the surrounding object 100 relative to the current vehicle 130 according to the relative movement parameter and the display parameter;

[0039] S300 , controlling the AR-HUD to display a mark suitable for representing the surrounding object 100 at the next moment position.

[0040] After the current vehicle 130 is started, the vehicle terminal can obtain the relative movement parameters of the surrounding objects 100 relative to the current vehicle 130 and the display parameters of the AR-HUD of the current vehicle 130. Among them, the relative movement parameters can be obtained by means of, but not limited to, sensors, radars, cameras and other devices, and this application does not limit this. The vehicle terminal can predict in advance the next moment position of the surrounding objects 100 relative to the current vehicle 130 based on the acquired relative movement parameters and display parameters, and then can control the AR-HUD to display the logo of the surrounding objects 100 at the next moment position, so that the logo of the surrounding objects 100 moves synchronously with the surrounding objects 100.

[0041] It should be noted that in the embodiment of the present application, the surrounding object 100 may be, but is not limited to, at least one of a surrounding vehicle, a surrounding non-motor vehicle, a pedestrian, and a roadblock. In addition, the next moment position mentioned above can generally be understood as the position of the surrounding object 100 relative to the current vehicle 130 after the current moment.

[0042] The following is a detailed introduction to each step of the AR-HUD display method in the implementation manner of the present application.

[0043] Step S100

[0044] In step S100, the relative movement parameters of the surrounding objects 100 relative to the current vehicle 130 and the display parameters of the AR-HUD of the current vehicle 130 are obtained. The relative movement parameters may include, but are not limited to, at least one of the current relative speed, the relative position at the last moment, and the previous relative speed. Those skilled in the art should understand that the current relative speed and the previous relative speed are both vectors, in other words, both include speed magnitude and speed direction. The display parameters may include, but are not limited to, the display frequency of the AR-HUD and / or the last display time of the AR-HUD.

[0045] As an example, the relative movement parameters include the current relative speed and the relative position at the last moment. The current relative speed and the relative position at the last moment can be obtained by: establishing a relative coordinate system with the current vehicle 130 as the origin; obtaining the relative coordinates of the surrounding objects 100 in the relative coordinate system at multiple different moments; and determining the current relative speed and the relative position at the last moment based on the multiple relative coordinates and the moment when the relative coordinates were obtained.

[0046] For example, after the vehicle-mounted terminal establishes a relative coordinate system with the current vehicle 130 as the origin, it can obtain multiple frames of images of the surrounding objects 100 at different times through image acquisition devices such as cameras, and identify each frame of the image of the surrounding objects 100 to determine the relative coordinates of the surrounding objects 100 in the relative coordinate system. Among them, the relative coordinates of the previous moment in the multiple relative coordinates are the relative positions at the previous moment. The vehicle-mounted terminal can calculate the current relative speed based on the multiple relative coordinates and the time when the relative coordinates are obtained. If the surrounding objects 100 move in a uniform straight line relative to the current vehicle 130 from a previous moment to the current moment, in other words, the connecting line of the multiple relative coordinates in the relative coordinate system is an inclined straight line, the vehicle-mounted terminal can first calculate the relative displacement based on the multiple relative coordinates, and then calculate the moving duration based on the multiple times when the relative coordinates are obtained, and finally calculate the current relative speed through the relative displacement and the moving duration. If the surrounding object 100 performs uniformly accelerated linear motion relative to the current vehicle 130 from a previous moment to the current moment, in other words, the line connecting multiple relative coordinates in the relative coordinate system is a parabola, the vehicle terminal can first calculate multiple equal moving durations based on multiple moments of obtaining relative coordinates, and then calculate multiple relative displacements based on multiple relative coordinates corresponding to the above moments, and then calculate acceleration based on multiple moving durations and corresponding relative displacements, and finally calculate the current relative speed based on acceleration, relative displacement and moving duration. The implementation method of the present application determines the relative coordinates of the surrounding object 100 in the relative coordinate system by collecting multiple frames of images of the surrounding object 100 at different moments, which can not only directly obtain the relative position of the surrounding object 100 at the previous moment relative to the current vehicle 130, in other words, the relative coordinates at the previous moment are the relative position at the previous moment, but also does not require additional auxiliary equipment. It can directly use existing equipment in the current vehicle 130, such as a camera, to obtain multiple frames of images of the surrounding object 100 at different moments, thereby reducing costs.

[0047] It should be noted that the method of obtaining the previous relative speed is similar to the method of obtaining the current relative speed, which will not be repeated here. In addition, the relative coordinates of the surrounding objects 100 can be obtained by millimeter wave radar or star flash technology in addition to image recognition, and this application does not limit this.

[0048] Step S200

[0049] In step S200, the next moment position of the surrounding object 100 relative to the current vehicle 130 is predicted based on the relative movement parameter and the display parameter. As an example, when the relative movement parameter includes the current relative speed and the relative position at the previous moment, and the display parameter includes the display frequency of the AR-HUD, the vehicle terminal may first determine the display interval duration at least based on the display frequency, and then calculate the next moment position at least based on the current relative speed, the relative position at the previous moment, and the display interval duration. Since the next moment position in the implementation mode of the present application is calculated at least based on the current relative speed, the relative position at the previous moment, and the display interval duration, the next moment position can be calculated as long as at least the current relative speed, the relative position at the previous moment, and the display interval duration are obtained, that is, the electronic device of the vehicle can calculate the next moment position of the surrounding object 100 relative to the current vehicle 130 in advance from the current moment. It can be seen that compared with the method of real-time calculation and real-time display in the prior art, that is, the method of calculating the next moment position at the next moment, the implementation mode of the present application can compensate for the time delay caused by the calculation by calculating the next moment position in advance.

[0050] It should be noted that the display interval duration can be directly determined according to the display frequency, but considering that the actual display time is different from the theoretical display time determined according to the display frequency, for example, the last display time in theory is t1, but due to the existence of display delay, the last display time in practice is t1+Δt1. Therefore, in order to improve the accuracy of the display interval duration, the vehicle terminal can determine the display interval duration according to the display frequency and the last display time of the AR-HUD. As an example, the vehicle terminal can first calculate the last actual display time according to the last display time and the pre-measured compensation time, where the compensation time is Δt1; then, the vehicle terminal predicts the next predicted display time according to the display frequency and the last actual display time, and finally calculates the display interval duration according to the last actual display time and the next predicted display time. It should be noted that before the vehicle leaves the factory, multiple actual display times can be obtained based on the timestamp, and then the theoretical display time corresponding to the actual display time is determined by the display frequency, and finally the compensation time can be calculated by multiple actual display times and the theoretical display time. For example, the compensation duration is an average of multiple target durations, and the target duration is a difference between the actual display time and the theoretical display time.

[0051] In addition, the speed of the surrounding objects 100 relative to the current vehicle 130 also affects the calculation method of the next moment position. If the surrounding objects 100 are moving in a uniform straight line relative to the current vehicle 130, the vehicle terminal can first calculate the relative displacement based on the current relative speed and the display interval duration, and then calculate the next moment position based on the relative displacement and the relative position at the previous moment. For example, the vehicle terminal can construct the following constant speed model:

[0052]

[0053] Δt=t2-t1′ (2);

[0054] t1′=t1+Δt1 (3);

[0055] Among them, x k+1 represents the x-axis coordinate of the surrounding object 100 in the relative coordinate system at the next moment, and y k+1 represents the coordinate of the surrounding object 100 on the y-axis of the relative coordinate system at the next moment, x k Indicates the x-axis coordinates of the surrounding object 100 at the last moment, y k represents the coordinate of the surrounding object 100 on the y-axis of the relative coordinate system at the last moment, v x.k+1 represents the next relative speed of the surrounding object 100 relative to the current vehicle 130 along the x-axis direction, v y.k+1 represents the next relative speed of the surrounding object 100 relative to the current vehicle 130 along the y-axis direction, v x.k represents the current relative speed of the surrounding object 100 relative to the current vehicle 130 along the x-axis direction, v y.k represents the current relative speed of the surrounding object 100 relative to the current vehicle 130 along the y-axis direction, Δt represents the display interval duration, t2 represents the next predicted display time, t1′ represents the previous actual display time, and Δt1 represents the compensation time.

[0056] According to formula (1), x k+1 =x k +v x.k *Δt;y k+1 =y k +v y.k *Δt;v x.k+1 =v x.k ;v y.k+1 =v y.k .

[0057] If the surrounding object 100 performs uniformly variable linear motion relative to the current vehicle 130, then while obtaining the current relative speed and the relative position at the previous moment, it is also necessary to obtain multiple previous relative speeds of the surrounding object 100 relative to the current vehicle 130. The vehicle terminal first calculates the relative acceleration based on multiple previous relative speeds and the moment of obtaining the previous relative speed, and then calculates the position at the next moment based on the relative acceleration, the current relative speed, the relative position at the previous moment, and the display interval duration. Those skilled in the art should understand that the relative acceleration is also a vector like the current relative speed, with magnitude and direction. If the relative acceleration is in the same direction as the current relative speed, it means that the surrounding object 100 performs accelerated linear motion relative to the current vehicle 130; if the relative acceleration is in the opposite direction to the current relative speed, it means that the surrounding object 100 performs decelerated linear motion relative to the current vehicle 130. The embodiment of the present application calculates the relative acceleration by using multiple previous relative speeds and the moment of obtaining the previous relative speed, and then calculates the position at the next moment based on the relative acceleration, the current relative speed, the relative position at the previous moment, and the display interval duration, so that the position at the next moment can be more accurate.

[0058] Step S300

[0059] In step S300, the AR-HUD is controlled to display an identifier suitable for representing the surrounding object 100 at the next moment. The identifier may include, but is not limited to, an identification frame 110 or an outline of the surrounding object 100. In addition, for easy identification, the identifier of the surrounding object 100 may also include a text identifier 120 and / or a pattern representing the object type.

[0060] Considering that the speed of the current vehicle 130 will also affect the position of the surrounding objects 100 relative to the current vehicle 130, thereby affecting the reaction time, and obtaining the vehicle outline takes a certain amount of time, in order to further reduce the delay, the vehicle terminal can determine whether the identification box 110 or the outline of the surrounding objects 100 should be displayed at the next moment based on the current speed of the current vehicle 130.

[0061] Specifically, the vehicle terminal can obtain the current speed of the current vehicle 130 through detection equipment such as a six-axis sensor, GPS, etc. If the current speed of the current vehicle 130 is greater than the preset speed threshold, it means that the position of the surrounding objects 100 relative to the current vehicle 130 may change rapidly, that is, the position at the next moment has a large change compared to the current position, so in order to save time and reduce delay, such as Figure 3The vehicle-mounted terminal shown can control the AR-HUD to display the identification frame 110 at the next moment. If the current speed of the current vehicle 130 is not greater than the preset speed threshold, it means that the position change of the surrounding objects 100 relative to the current vehicle 130 may be relatively slow, and the vehicle-mounted terminal has enough time to obtain the outline of the surrounding objects 100. In this case, the vehicle-mounted terminal can obtain the outline of the surrounding objects 100 and control the AR-HUD to display the outline at the next moment. Among them, the preset speed threshold can be but not limited to 20km / h. In addition, the preset speed threshold can be a factory preset value, or it can be a threshold set by the user according to his own needs, and of course it can also be a threshold calculated in real time by the vehicle-mounted terminal.

[0062] It should be noted that when determining whether the identification frame 110 or the outline of the surrounding object 100 should be displayed at the next moment, in addition to the current speed of the current vehicle 130, the current relative speed of the surrounding object 100 relative to the current vehicle 130 can also be used to determine. For example, if the current relative speed is greater than the preset speed threshold, it means that the position of the surrounding object 100 relative to the current vehicle 130 changes rapidly, such as Figure 3 The vehicle-mounted terminal can control the AR-HUD to display the identification frame 110 at the next moment. On the contrary, if the current relative speed is not greater than the preset speed threshold, it means that the position of the surrounding object 100 relative to the current vehicle 130 changes relatively slowly, and the vehicle-mounted terminal can obtain the outline of the surrounding object 100 and control the AR-HUD to display the outline at the next moment. The preset speed threshold corresponding to the current speed and the preset speed threshold corresponding to the current relative speed can be the same or different.

[0063] In some embodiments, the preset vehicle speed threshold may be determined in the following manner: determining the acquisition time for acquiring the contour of the surrounding object 100; and determining the preset vehicle speed threshold at least according to the next moment position of the surrounding object 100 relative to the current vehicle 130 and the above acquisition time. For example, the preset vehicle speed threshold corresponding to the current relative speed may be determined in the following manner: first determining the acquisition time for acquiring the contour of the surrounding object 100, then determining the relative distance of the surrounding object relative to the current vehicle 130 according to the next moment position of the surrounding object 100 relative to the current vehicle 130, and finally calculating the vehicle speed required to reach the next moment position within the acquisition time according to the relative distance and the acquisition time, and the vehicle speed is the preset vehicle speed threshold.

[0064] As an example, the contour of the surrounding object 100 can be obtained in the following manner: obtaining a region of interest image, the region of interest image including an image of the surrounding object 100; performing semantic segmentation on the region of interest image to segment the image of the surrounding object 100; and extracting the contour from the image of the surrounding object 100.

[0065] In addition, in order to improve the visual effect, along the speed direction of the current relative speed, the vehicle terminal can control the AR-HUD to display a trailing image behind the outline. Among them, the display parameters of the trailing image are determined based on the current relative speed, and the display parameters of the trailing image include color and / or trailing length. For example, if the current relative speed is less than the first relative speed, the trailing image is displayed behind the outline of the surrounding object 100 with the first color and / or the first length; if the current relative speed is between the first relative speed and the second relative speed, the trailing image is displayed behind the outline of the surrounding object 100 with the second color and / or the second length; if the current relative speed is greater than the second relative speed, the trailing image is displayed behind the outline of the surrounding object 100 with the third color and / or the third length. Among them, the first length, the second length and the third length increase in sequence. In order to warn the driver, the saturation of the first color, the second color and the third color increases in sequence or is different colors. In other words, the color of the trailing image becomes more and more vivid as the current relative speed increases. For example, the first color may be green, the second color may be blue, and the third color may be red.

[0066] In addition, an embodiment of the present application also provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned AR-HUD display method.

[0067] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned AR-HUD display method is implemented.

[0068] Figure 4 A schematic block diagram of an example electronic device 200 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 200 is intended to represent various forms of digital computers and may include, but is not limited to, applications in vehicles. The electronic device may also represent various forms of mobile devices that can run computing programs. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0069] like Figure 4As shown, the electronic device 200 includes a computing unit 201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 202 or a computer program loaded from a storage unit 208 into a random access memory (RAM) 203. In the RAM 203, various programs and data required for the operation of the electronic device 200 can also be stored. The computing unit 201, the ROM 202, and the RAM 203 are connected to each other via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204. The computing unit 201 can also be collectively referred to as a processor herein.

[0070] A number of components in the electronic device 200 are connected to the I / O interface 205, including: an input unit 206, such as a display screen, etc.; an output unit 207, such as various types of displays, speakers, etc.; a storage unit 208, such as a disk, etc.; and a communication unit 209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 209 allows the electronic device 200 to exchange information / data with other devices through a computer network 140 such as the Internet and / or various telecommunication networks 140.

[0071] The computing unit 201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 201 performs the various methods and processes described above, such as the AR-HUD display method. For example, in some embodiments, the AR-HUD display method may be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 208. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 200 via the ROM 202 and / or the communication unit 209. When the computer program is loaded into the RAM 203 and executed by the computing unit 201, one or more steps of the AR-HUD display method described above may be performed. Alternatively, in other embodiments, the computing unit 201 may be configured to execute the AR-HUD display method in any other appropriate manner (eg, by means of firmware).

[0072] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0073] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow diagram and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0074] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0075] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0076] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An AR-HUD display method, characterized in that: include: Obtaining relative movement parameters of surrounding objects relative to the current vehicle and display parameters of the AR-HUD of the current vehicle; Predicting the next moment position of the surrounding objects relative to the current vehicle according to the relative movement parameter and the display parameter; as well as The AR-HUD is controlled to display an identifier suitable for representing the surrounding object at the next moment position.

2. The AR-HUD display method according to claim 1, wherein: The relative movement parameters include the current relative speed and the relative position at the previous moment, and the display parameters include the display frequency of the AR-HUD.

3. The AR-HUD display method according to claim 2, wherein: Predicting the next moment position of the surrounding objects relative to the current vehicle according to the relative movement parameter and the display parameter includes: Determining a display interval duration at least according to the display frequency; and The position at the next moment is calculated at least according to the current relative speed, the relative position at the previous moment and the display interval duration.

4. The AR-HUD display method according to claim 3, wherein: The display parameters also include the last display time of the AR-HUD; Wherein, determining the display interval duration at least according to the display frequency includes: Calculating the last actual display time according to the last display time and the pre-measured compensation time; predicting a next predicted display time according to the display frequency and the previous actual display time; and The display interval duration is calculated according to the previous actual display time and the next predicted display time.

5. The AR-HUD display method according to claim 3, wherein: The relative movement parameters also include a plurality of previous relative speeds; Before calculating the position at the next moment at least according to the current relative speed, the relative position at the previous moment and the display interval duration, the display method further includes: Calculating a relative acceleration based on a plurality of the previous relative speeds and the time at which the previous relative speeds were obtained; The step of calculating the position at the next moment at least based on the current relative speed, the relative position at the previous moment and the display interval duration includes: The position at the next moment is calculated according to the relative acceleration, the current relative speed, the relative position at the previous moment and the display interval duration.

6. The AR-HUD display method according to claim 2, wherein: Obtaining the relative movement parameters of surrounding objects relative to the current vehicle includes: Establishing a relative coordinate system with the current vehicle as the origin; Acquire the relative coordinates of the surrounding objects in the relative coordinate system at multiple different times; and The current relative speed and the relative position at the previous moment are determined according to the plurality of relative coordinates and the moment when the relative coordinates are acquired.

7. The AR-HUD display method according to claim 6, wherein: Acquiring the relative coordinates of the surrounding objects in the relative coordinate system at multiple different times includes: Acquire multiple frames of images of surrounding objects at different times; and The image of each frame of the surrounding objects is recognized to determine the relative coordinates of the surrounding objects in the relative coordinate system.

8. The AR-HUD display method according to claim 1, wherein: The identification includes an identification frame or an outline of the surrounding object; Among them, controlling the AR-HUD to display a mark suitable for representing the surrounding object at the next moment position includes: Obtaining the current speed of the current vehicle; In response to the current vehicle speed being greater than a preset vehicle speed threshold, controlling the AR-HUD to display the identification frame at the next moment position; and In response to the current vehicle speed being not greater than a preset vehicle speed threshold, the contour is acquired and the AR-HUD is controlled to display the contour at the next moment position.

9. The AR-HUD display method according to claim 1, wherein: The identification includes an identification frame or an outline of the surrounding object, and the relative movement parameter includes a current relative speed; Among them, controlling the AR-HUD to display a mark suitable for representing the surrounding object at the next moment position includes: In response to the current relative speed being greater than a preset vehicle speed threshold, controlling the AR-HUD to display the identification frame at the next moment position; and In response to the current relative speed being not greater than a preset vehicle speed threshold, the contour is acquired and the AR-HUD is controlled to display the contour at the next moment position.

10. The AR-HUD display method according to claim 8 or 9, wherein: The display method further includes: Determining an acquisition time for acquiring the contour; and The preset vehicle speed threshold is determined at least according to the next moment position and the acquisition duration.

11. The AR-HUD display method according to claim 8 or 9, wherein: The relative movement parameters include current relative speed; The display method further includes: The AR-HUD is controlled to display a trailing image behind the contour along a speed direction of the current relative speed.

12. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the AR-HUD display method described in any one of claims 1 to 11.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the AR-HUD display method according to any one of claims 1 to 11 is implemented.

Images