A display method, apparatus and system

Abstract

The application discloses a display method, device and system. The target object first time prediction position and the first time measurement position are obtained, and the electronic device displays the prompt information of the target object at the first time, effectively reduces the output jitter, and improves the user experience.

Description

[0001] This application is a divisional application. The original application, application number 202180005788.3, ​​was filed on July 31, 2021. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of automotive technology, and in particular to a display method, device, and system. Background Technology

[0003] Currently, in order to enhance drivers' access to road information and achieve functions such as augmented reality (AR) navigation and AR warnings, such as... Figure 1 As shown, AR head-up display (AR-HUD) is used in vehicles. This involves transforming the information of pedestrians, vehicles, etc. detected by the camera into the real-world coordinate system and projecting it onto an electronic device in the driver's field of vision, such as the windshield of the vehicle. This integrates the projected virtual information with the real road condition information to achieve functions such as pedestrian warning and vehicle warning.

[0004] However, due to limitations in computing power provided by the vehicle, excessively high speeds, detection errors inherent in the vehicle's cameras, and vehicle vibrations during driving, such as... Figure 2 As shown, when AR-HUD is used for display, the detection results of pedestrians and vehicles detected by the driving vehicle will exhibit a relatively obvious shaking phenomenon, which can cause dizziness and affect the driver's long-term safe driving.

[0005] In summary, the current method of displaying AR-HUDs suffers from significant jitter and a poor user experience. Summary of the Invention

[0006] This application provides a display method, apparatus, and system for reducing image jitter during the display process and improving display effect.

[0007] The display method provided in this application can be executed by an electronic device. The electronic device can be abstracted as a computer system. The electronic device can be a complete system or a component within a complete system, such as a system-on-a-chip (SoC) or image processing chip. Specifically, the electronic device can be a terminal device or in-vehicle equipment such as an in-vehicle computer or vehicle infotainment system, or it can be a SoC, image processing chip, or other type of chip that can be installed in a computer system within a vehicle or in-vehicle equipment.

[0008] In a first aspect, embodiments of this application provide a display method, including: Acquire sensor information of the target object; obtain the predicted position of the target object at a first moment based on the sensor information of the target object; acquire the measured position of the target object before the first moment; and enable the display device to display prompt information of the target object at the first moment based on the predicted position and the measured position.

[0009] For example, when the above method is executed by the electronic device, the sensor information of the target object can be acquired through the acquisition device in the electronic device. The acquisition device can send the sensor information of the target object to the processing device through an interface circuit connected to the processing device in the electronic device. The processing device can process the acquired sensor information of the target object to obtain the prompt information of the target object at a first moment. The processing device can also send the prompt information of the target object at a first moment to the display device for projection through an interface circuit connected to the display device in the electronic device.

[0010] Using this method, during the display process, the predicted position of the target object at the first moment is obtained by collecting the sensor information of the target object, and the predicted position is combined with the measured position obtained before the first moment, enabling the electronic device to display the prompt information of the target object at the first moment, effectively reducing output jitter.

[0011] In one possible design, if the measured position of the target object cannot be obtained before the first time step, the display device is enabled to display a prompt message about the target object at the first time step and the display position, where the display position is related to the predicted position. Therefore, if the measured position of the target object is not obtained before the first time step, in order to better ensure the real-time nature of the output display position of the target object at the first time step, the predicted position can be directly determined as the display position of the target object at the first time step, effectively reducing latency.

[0012] For example, when the above method is performed by the electronic device, the processing device in the electronic device can determine whether the measurement position of the target object was obtained before the first time period, and if the measurement position of the target object was not obtained before the first time period, the predicted position is determined as the display position.

[0013] In one possible design, when the measured position of the target object is obtained before the first moment, the display device is enabled to display a prompt message about the target object at the first moment and the display position, where the display position is related to the predicted position and the measured position. Therefore, if the measured position of the target object is obtained before the first moment, in order to better reduce the jitter of the output content, the display position of the target object at the first moment can be determined based on the predicted position and the measured position.

[0014] In one possible design, the processing unit in the electronic device can determine the average of the predicted position and the measured position as the display position of the target object at a first moment.

[0015] In one possible design, when multiple measurement positions of the target object are acquired before the first time point, a display device is enabled to display a prompt message about the target object at the first time point and a display position. The display position is related to the average value of the multiple measurement positions and the predicted position. Therefore, after acquiring multiple measurement positions of the target object before the first time point, it is necessary to determine a measurement position to be fused with the predicted position. The display position is obtained by fusing the selected measurement position with the predicted position. For example, the average value of the multiple measurement positions can be determined as the selected measurement position for fusion.

[0016] For example, when the above method is performed by the electronic device, the average value of multiple measurement positions can be determined by the processing device in the electronic device, and the average value of the multiple measurement positions can be fused with the predicted position to obtain the display position of the target object at the first moment.

[0017] In one possible design, when multiple measurement positions of the target object are acquired before the first time point, a display device is enabled to display a prompt message about the target object at the first time point and a display position. The display position is related to the last measurement position acquired among the multiple measurement positions and the predicted position. Therefore, after acquiring multiple measurement positions of the target object before the first time point, it is necessary to determine a measurement position to be fused with the predicted position. The display position is obtained by fusing the selected measurement position with the predicted position. For example, the last measurement position among the multiple measurement positions can be determined as the selected measurement position for fusion.

[0018] For example, when the above method is performed by the electronic device, the processing unit in the electronic device can determine the last measurement position among multiple measurement positions, and fuse the last measurement position with the predicted position to obtain the display position of the target object at the first moment.

[0019] In one possible design, the display position is also related to a pre-set correction value, which is used to reduce errors caused by vehicle bumps and swaying during driving. Therefore, after obtaining the display position of the target object at the first moment, in order to further reduce jitter, the display position of the target object at the first moment can be fused and calibrated again before projection. For example, the display position can be obtained based on the predicted position, the measured position, and the correction value, so that the display position of the target object at the first moment is closer to the actual position of the target object, improving the user experience.

[0020] For example, when the above method is performed by the electronic device, the display position can be updated by the processing unit in the electronic device according to the correction value.

[0021] In one possible design, the display position is also related to the average of the display positions corresponding to multiple adjacent time points before the first time point. Therefore, after obtaining the display position of the target object at the first time point, in order to further reduce jitter, the display position of the target object at the first time point can be fused and calibrated again before projection. For example, the display position of the target object at the first time point can be obtained based on the display positions corresponding to multiple adjacent time points before the first time point, so that the display position of the target object in the first time point is closer to the actual position of the target object, thus improving the user experience.

[0022] For example, when the above method is performed by the electronic device, the average value of the display positions corresponding to multiple adjacent times before the first time can be determined by the processing device in the electronic device, and the display position of the first time can be updated according to the average value.

[0023] In one possible design, based on the correspondence between the world coordinate system and the vehicle coordinate system where the display location is located, the display device is enabled to project the prompt information of the target object in the world coordinate system onto the vehicle coordinate system at the first moment. Therefore, during the display process, in order to better fit the vehicle coordinate system, the position of the prompt information of the target object in the vehicle coordinate system at the first moment can be determined according to the correspondence between the world coordinate system and the vehicle coordinate system.

[0024] For example, when the above method is performed by the electronic device, the processing device in the electronic device can determine the position of the prompt information of the target object in the vehicle coordinate system at the first moment according to the correspondence.

[0025] In one possible design, the target object includes one or more of a vehicle, a person, an obstacle, and a traffic sign. The sensing information includes one or more of the following: the external features of the target object, the position of the target object at a first moment, and the distance between the target object and the driving vehicle at the first moment. Therefore, based on the collected sensing information of the target object, the condition of the target object in front of the driving vehicle can be known.

[0026] For example, in the embodiments of this application, when the target object includes a vehicle, the external characteristics of the target object include, but are not limited to, the vehicle model, vehicle width, vehicle length, vehicle value, and vehicle color. When the target object includes a pedestrian, the external characteristics of the target object include, but are not limited to, the pedestrian's height, gender, age group, and clothing color.

[0027] Secondly, this application provides an electronic device comprising a processing module and a communication module. The communication module is used to acquire sensing information of a target object through an interface circuit. The processing module is used to obtain a predicted position of the target object at a first moment based on the sensing information of the target object; obtain a measured position of the target object within a preset time period before the first moment; and enable a display device to display a prompt message of the target object at the first moment based on the predicted position and the measured position.

[0028] In one possible design, the processing module can be used for: When the measured position of the target object is obtained before the first moment, the display device is enabled to display the prompt information of the target object at the first moment and the display position, wherein the display position is related to the predicted position and the measured position.

[0029] In one possible design, the processing module can be used for: If the measured position of the target object cannot be obtained before the first moment, the display device is enabled to display a prompt message about the target object at the first moment and the display position, wherein the display position is related to the predicted position.

[0030] In one possible design, the processing module can be used for: When multiple measurement positions of the target object are acquired before the first moment, the display device is enabled to display a prompt message of the target object at the first moment and the display position, wherein the display position is related to the average value of the multiple measurement positions and the predicted position.

[0031] In one possible design, the processing module can be used for: When multiple measurement positions of the target object are obtained before the first moment, the display device is enabled to display the prompt information of the target object at the first moment and the display position. The display position is related to the last measurement position obtained among the multiple measurement positions and the predicted position.

[0032] In one possible design, the processing module is further configured to: The display position is updated according to the correction value; the correction value is preset to remove errors caused by the vehicle's bumps and swaying during driving.

[0033] In one possible design, the processing module is further configured to: The display position is updated based on the average of the display positions corresponding to multiple adjacent times before the first time.

[0034] In one possible design, the target object includes one or more of a vehicle, a person, an obstacle, and a traffic sign.

[0035] Thirdly, this application provides a computing device including a processor connected to a memory storing computer programs or instructions. The processor is used to execute the computer programs or instructions stored in the memory, so that the computing device performs the methods in the first aspect or any possible implementation thereof.

[0036] Fourthly, this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed, causes a computer to perform the method described in the first aspect or any possible implementation thereof.

[0037] Fifthly, this application provides a computer program product that, when executed by a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0038] In a sixth aspect, this application provides a chip connected to a memory for reading and executing computer programs or instructions stored in the memory to implement the method in the first aspect or any possible implementation of the first aspect.

[0039] In a seventh aspect, this application provides a vehicle that includes the on-board control device and actuator as described in the second aspect or any possible implementation of the second aspect, to implement the method described in the first aspect or any possible implementation of the first aspect.

[0040] Eighthly, this application provides a vehicle that includes the chip and actuator described in the sixth aspect above, to implement the method in the first aspect or any possible implementation of the first aspect.

[0041] It should be understood that, based on the technical solution provided in this application, during the display process while driving, by combining the predicted position of the target object at the first moment with the measured position of the target object before the first moment, a fusion correction is performed. This enables the display device to display the prompt information of the target object at that first moment to be closer to the actual situation of the target object, effectively reducing the jitter of the content projected on the windshield of the driving vehicle. Furthermore, during the display process, the displayed content can be updated through correction values, thereby further optimizing jitter and effectively reducing the dizziness experienced by the driver when using AR head-up display for road condition information, ensuring safe driving. Attached Figure Description

[0042] Figure 1 A schematic diagram of a vehicle warning display for an AR-HUD provided in this application; Figure 2 This application provides a schematic diagram of a display jitter scenario; Figure 3 A schematic diagram of an electronic device scenario provided in this application; Figure 4 A schematic diagram of the structure of an electronic device provided in this application; Figure 5 A schematic diagram of another electronic device provided in this application; Figure 6 A schematic diagram of another electronic device provided in this application; Figure 7 This is a flowchart illustrating the first display method provided in this application; Figure 8 This is a schematic diagram of the second display method provided in this application; Figure 9 This application provides a schematic diagram of a data collection scenario. Figure 10 This application provides a schematic diagram for obtaining the measurement position of a target object; Figure 11 This application provides a schematic diagram illustrating the first method for determining the display position of a target object. Figure 12 A schematic diagram illustrating the fusion of predicted and measured positions into a displayed position, as provided in this application; Figure 13 This application provides a schematic diagram illustrating the second method for determining the display position of the target object. Figure 14This is a schematic diagram illustrating the first method for determining the measurement location provided in this application; Figure 15 This application provides a second schematic diagram for determining the measurement location; Figure 16 This application provides a schematic diagram of a display position update scenario; Figure 17 A schematic diagram illustrating a scenario where the display position is updated via a sliding window filtering method, as provided in this application; Figure 18 A schematic diagram of the third display method provided in this application. Detailed Implementation

[0043] This application provides a display method, apparatus, and system to reduce image jitter during the display process and improve the display effect. The method and apparatus are based on the same technical concept. Since the principles by which the method and apparatus solve the problem are similar, their implementations can be referred to interchangeably, and repeated details will not be repeated.

[0044] In the method provided in this application embodiment, during the display process, the electronic device can obtain the predicted position of the target object at a first moment and the measured position of the target object before the first moment based on the sensor information of the target object obtained. By using the predicted position and the measured position, the display device can display the prompt information of the target object at the first moment, effectively reducing the jitter of the output image.

[0045] The electronic device in this application embodiment can be used to support the vehicle in implementing the method provided in this application embodiment.

[0046] Optionally, the electronic device can be integrated with the vehicle, for example, it can be installed inside the vehicle. Alternatively, the electronic device can be separate from the vehicle, for example, it can be implemented through a terminal device. This terminal device could be, for example, an AR-HUD or an in-vehicle device.

[0047] For example, such as Figure 3 As shown, taking an in-vehicle device as an example, such as a dashcam with projection function, the in-vehicle device can provide the following functions: acquiring sensor information of a target object through the dashcam; determining the predicted position of the target object at a first moment and the measured position before the first moment based on the sensor information of the target object; and displaying a prompt message of the target object at the first moment based on the predicted position and the measured position.

[0048] The vehicle in this embodiment may have autonomous driving capabilities, especially human-machine interaction (HMI) capabilities.

[0049] Furthermore, it should be understood that, depending on the actual needs of use, the vehicle may be replaced by other vehicles or means of transportation such as trains, aircraft, or mobile platforms. This application does not limit this.

[0050] For example, Figure 4 A schematic diagram of a possible electronic device is shown, which may include a processing module 410 and a data acquisition module 420. Exemplarily, Figure 4 The structure shown may be an in-vehicle device or a functional component having the electronic device shown in this application.

[0051] When the structure is an in-vehicle device or other electronic device, the acquisition module 420 may include a camera, a sensor, or other device for supporting the target object acquisition function, and the processing module 410 may be a processor, such as a central processing unit (CPU). The acquisition module 420 can communicate with the processing module 410 via Bluetooth, network connection, or interface circuit. The processing module 410 can display traffic information on the display screen 410 via projection, wired connection, or wireless connection. When the structure is a functional component of the electronic device shown in this application, the acquisition module 420 may include a camera, a sensor, or other device for supporting the target object acquisition function, and the processing module 410 may be a processor. The acquisition module 420 can communicate with the processing module 410 via interface circuit. The processing module 410 can display target object prompts on the display screen 410 via projection, wired connection, or wireless connection. When the structure is a chip or chip system, the acquisition module 420 can be one or more of a camera device and a sensing device controlled by the chip, and the processing module 410 can be the chip's processor, which may include one or more central processing units. It should be understood that the processing module 410 in the embodiments of this application can be implemented by a processor or processor-related circuit components, and the acquisition module 420 can be implemented by a camera device, a sensing device, or a related acquisition device.

[0052] For example, the processing module 410 can be used to perform all operations performed by the electronic device in any embodiment of this application, except for the acquisition operation and the projection operation. For example, based on the sensing information of the target object, it can determine the predicted position of the target object at a first moment; and based on the predicted position and the measured position of the target object, it can display the prompt information of the target object at a first moment. The acquisition module 420 can be used to perform the acquisition operation of the target object in any embodiment of this application. For example, it can acquire the sensing information of the target object through one or more of a camera device and a sensing device.

[0053] The sensing information of the target object acquired by the processing device 410 may be generated from point cloud information, sound, and one or more sensing data in the image of the target object collected by an external sensor or camera; or, the sensing information of the target object acquired by the processing device 410 may be generated from point cloud information, sound, and one or more sensing data in the image of the target object collected by its own sensor or camera.

[0054] In this embodiment, the camera device can be a monocular camera, a binocular camera, or the like. The shooting area of ​​the camera device can be the external environment of the vehicle. The sensing device is used to acquire sensor data of the target object, thereby assisting the processing device in the vehicle to analyze and determine the sensor information of the target object. For example, the sensing device in this embodiment can include lidar, millimeter-wave radar, ultrasonic radar, etc., for acquiring environmental information.

[0055] Additionally, the processing module 410 can be a functional module that can perform both the analysis of collected information and the display of traffic information on the display screen. When performing processing operations, the processing module 410 can be considered an analysis module, and when performing display operations, it can be considered a display module. For example, the processing module 410 in this embodiment can be replaced by an AR-HUD. That is, the AR-HUD in this embodiment has the functions of the aforementioned processing module 410. Alternatively, the processing module 410 can also include two functional modules, which can be considered a collective term for these two modules: an analysis module and a display module. The analysis module analyzes the traffic conditions based on the acquired sensor information of the target object, and determines the prompt information for the target object at the first moment based on the predicted position of the target object at the first moment and the measured position before the first moment. The display module displays the prompt information of the target object determined by the analysis module on the display screen.

[0056] Furthermore, in this embodiment of the application, the electronic device may also include a storage module for storing one or more programs and data information; wherein the one or more programs include instructions. The electronic device may also include a display screen, which may be a windshield in a vehicle or other in-vehicle equipment display screen.

[0057] Figure 5 A schematic diagram of another electronic device is shown, used to perform the actions performed by the electronic device provided in the embodiments of this application. This is for ease of understanding and illustration. Figure 5As shown, the electronic device may include a processor, a memory, and an interface circuit. Furthermore, the electronic device may also include at least one component selected from the following: a data acquisition device, a processing device, a display device, or a display screen. The processor is mainly used to implement the processing operations provided in the embodiments of this application, such as analyzing and processing the acquired sensing information of the target object, executing software programs, and processing data from the software programs. The memory is mainly used to store software programs and data. The data acquisition device can be used to acquire sensing information of the target object and may include a camera, millimeter-wave radar, or ultrasonic sensor. The interface circuit can be used to support communication of the electronic device; for example, after the data acquisition device acquires the sensing information of the target object at a first moment, it can send the acquired sensing information to the processor through the interface circuit. The interface circuit may include a transceiver or an input / output interface.

[0058] It should be understood that, for ease of explanation, Figure 5 Only one memory and processor are shown in the illustration. In actual electronic devices, there may be one or more processors and one or more memories. Memory may also be referred to as storage medium or storage device, etc. Memory may be set up independently of the processor or integrated with the processor; this application does not limit this.

[0059] Figure 6 The image shows another electronic device provided in an embodiment of this application. As can be seen, this electronic device may include a detection module, a tracking and fusion module, a HUD stabilization module, a HUD coordinate transformation module, and a HUD engine rendering module, etc.

[0060] The detection module uses detection algorithms to detect sensor information of target objects, such as pedestrians and vehicles, obtaining their positions and bounding box information. The specific method depends on the detection device. For example, it might use a camera to capture photos of pedestrians and vehicles and extract their information from the photos; or it could use sensors to obtain the distance and position of pedestrians and vehicles relative to the driver's vehicle. Furthermore, the detection module can also be used to detect the measured position of the target object at a given moment.

[0061] The tracking fusion module can be used to build a 3D prediction model of the detected target object based on the sensing information at the first moment, and use the model to predict and track the position of the target object at the first moment. When detection is complete, the predicted output value of the model and the detected value of the target object before the first moment are fused and updated as the output. When detection is not complete, the predicted value of the tracking algorithm is output using the MQ mechanism.

[0062] The HUD anti-shake module is used to smooth the position output of the tracking fusion module by using a sliding window filtering method, reducing its jitter and improving the stability of the 3D detection frame.

[0063] The HUD coordinate transformation module is used to transfer position information from the camera coordinate system to the vehicle coordinate system in a real vehicle HUD scenario by using a large-scale real vehicle camera attitude calibration algorithm.

[0064] The HUD engine rendering module is used to input the final output location information into the HUD rendering engine, render the corresponding pedestrian / vehicle warning information, and project it onto the windshield through the optical engine, presenting it to the driver's eyes, so that the driver can receive real-time warning information for pedestrians and vehicles, realizing real-time warnings for pedestrians and vehicles.

[0065] The system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Furthermore, those skilled in the art will understand that with the evolution of vehicle architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems. It should be understood that... Figures 4 to 6 This is a simplified diagram for ease of understanding only. The system architecture may also include other devices or other unit modules.

[0066] The following is combined with Figure 7 The method provided in the embodiments of this application will be described. This method can be executed by an electronic device. The electronic device may include a processing device and a display device. The processing device may be an in-vehicle system, a computer, or a processing device used within a HUD, etc. The electronic device may include... Figures 4 to 6 Any one or more structures shown. When implementing this display method, it can be... Figure 4 The processing module 410 shown or Figure 5 The processor shown, or Figure 6 The tracking fusion module, HUD stabilization module, HUD coordinate transformation module, and HUD engine rendering module shown implement the processing actions in the method provided in this application embodiment. It can also be implemented by... Figure 4 The acquisition module 420 shown is or Figure 5 The data acquisition device shown, or Figure 6 The detection module shown realizes the acquisition of sensor information of the target object. These interactions include, but are not limited to, acquiring sensor information of the target object.

[0067] S700: Electronic devices acquire sensor information of the target object.

[0068] S701, the electronic device obtains the predicted position of the target object at a first moment based on the sensing information of the target object.

[0069] S702, the electronic device acquires the measured position of the target object before the first moment.

[0070] S703, the electronic device enables the display device to display a prompt message for the target object at the first moment, based on the predicted position and the measured position.

[0071] It should be noted that in the embodiments of this application, the electronic device displays the target object based on the measured position and predicted position (based only on the predicted position). The display can be at that position, or it can be displayed in a nearby or related position.

[0072] For example, you can frame the target object; or display some prompts, such as indicating that there are pedestrians nearby.

[0073] In order to better illustrate the display method provided in this application, such as Figure 8 As shown, based on Figure 7 The content shown will be further detailed: S800: Electronic devices acquire sensing information of the target object.

[0074] In one optional manner described in this application, the electronic device can acquire the sensing information of the target object through an interface circuit; or, the electronic device can acquire the sensing information of the target object through wireless communication, such as through a Bluetooth connection, etc., without limitation.

[0075] Furthermore, the sensing information of the target object acquired by the electronic device can be sensing information at a single moment; or, the sensing information of the target object acquired by the electronic device can be sensing information over a period of time. In this case, after the electronic device acquires sensing information over a period of time, it can filter the acquired sensing information over that period of time to select useful sensing information.

[0076] In this application embodiment, the target objects include one or more of vehicles, pedestrians, obstacles, and traffic signs. The traffic signs may also include one or more of traffic signs, traffic lights, and road markings.

[0077] It should be understood that the target object in the embodiments of this application is not limited to the above content. Any object that is applicable to this application can be used as the target object of this application.

[0078] Furthermore, the sensing information of the target object may be one or more of the target object's external features, current position, and distance from the driving vehicle. For example, if the target object includes a vehicle, the sensing information of the target object may include one or more of the vehicle's external features, current position, and distance from the driving vehicle; the vehicle's external features may include the vehicle's model, color, length, and width, etc.

[0079] If the target object includes a pedestrian, the sensing information of that target object may include one or more of the pedestrian's external characteristics, the pedestrian's current location, and the distance to the driving vehicle; the pedestrian's external characteristics may include the pedestrian's identity, for example, the pedestrian's identity may be an adult, an elderly person, etc. Thus, the electronic device can determine, based on the pedestrian's identity, whether the pedestrian has a quick reaction ability and whether a stronger warning is needed, etc.

[0080] To better understand the S800 process, we will use the example of acquiring sensor information of the target object through a camera. For example, as... Figure 9 As shown, a camera can capture multiple consecutive images. By identifying feature pixels in the images, it determines the sensor information of the target object and then sends it to the processor in the electronic device via an interface circuit, enabling the electronic device to acquire the sensor information of the target object. For example, the camera can capture multiple consecutive images and then directly send the captured images to the processor in the electronic device via an interface circuit. The processor then identifies the feature pixels in the images to determine the sensor information of the target object, enabling the electronic device to acquire the sensor information of the target object.

[0081] S801. The electronic device obtains the predicted position of the target object at the first moment based on the sensing information of the target object.

[0082] In one optional embodiment of this application, the electronic device can obtain the predicted position of the target object at a first moment by inputting the acquired sensing information of the target object into a prediction model for determining the predicted position.

[0083] In this embodiment, a prediction model can be established based on the sensor information of a threshold number of target objects previously acquired. The prediction model's function is to predict the future position of the target object based on its current sensor information, such as acceleration, direction, distance, and location.

[0084] In this embodiment of the application, the electronic device can also provide feedback based on the error between the predicted position and the corresponding actual position of the target object over a period of time, and update and adjust the established prediction model.

[0085] It should be noted that this application does not limit the way the prediction model is established or the prediction method.

[0086] Furthermore, the electronic device in this application can also perform location prediction based on a large database corresponding to the target object.

[0087] It should be understood that the embodiments of this application are not limited to the method of obtaining the predicted position of the target object at the first moment. Any method that can be applied to this application is applicable to this application. S802: The electronic device attempts to obtain the measurement position before the first moment. If it obtains the position, execute S803; if it does not obtain the position, execute S804.

[0088] The electronic device can acquire the measurement location in various ways, and is not limited to the following: Implementation method 1: The measurement position is obtained by the electronic device before the first moment.

[0089] In other words, the electronic device only needs to ensure that it acquires the measurement position before the first moment.

[0090] Implementation method 2: The measurement position is obtained by the electronic device within a preset time period before the first moment.

[0091] In other words, the electronic device needs to ensure that it acquires the measurement position before the first moment and within a preset time period before that first moment.

[0092] For example, such as Figure 10 As shown, assuming a preset duration of 1ms, the electronic device acquires the measurement position of the target object within 1ms before the first moment.

[0093] In one optional embodiment of this application, a camera can continuously capture multiple images within a preset time period. By identifying feature pixels in the images, the sensor information of the target object within the preset time period is determined, and then sent to the electronic device through an interface circuit. This allows the electronic device to determine the measurement position of the target object based on the sensor information within the preset time period.

[0094] In another optional embodiment of this application, a camera can continuously capture multiple images within a preset time period, and then directly send the captured multiple images to an electronic device through an interface circuit. The electronic device can then identify the feature pixels in the images, determine the sensing information of the target object, and thus determine the measurement position of the target object.

[0095] It should be noted that, in the embodiments of this application, the number of measurement positions of the target object obtained by the electronic device within a preset time period before the first moment may be one, multiple, or even zero.

[0096] S803, the electronic device determines the display position of the target object at a first moment based on the predicted position and the measured position.

[0097] In one case, such as Figure 11As shown, if the electronic device determines that it has acquired the measured position of the target object within the preset time period, the electronic device merges the measured position with the predicted position to obtain the displayed position of the target object at the first moment.

[0098] For example, in this embodiment of the application, the electronic device can determine the display position by the average of the predicted position and the measured position.

[0099] It should be understood that, such as Figure 12 As shown, the embodiment of this application obtains the display position of the target object at the first moment by fusing and correcting the predicted position of the target object at the first moment and the measured position, so that the obtained display position is closer to the real trajectory of the target object, effectively reducing jitter and improving user experience.

[0100] Furthermore, in S803, if the electronic device acquires multiple measurement positions of the target object within the preset time period, the measurement position used for fusion with the predicted position can be determined in various ways, including but not limited to the following: Method 1: The electronic device determines the average value of the multiple measurement locations as the measurement location to be fused with the predicted location.

[0101] For example, such as Figure 13 As shown, assuming that three measurement locations, such as measurement locations 1 to 3, are acquired within a preset time period before the first moment, the electronic device can determine the average value of these three measurement locations, and then determine the average value of these three measurement locations as the measurement location to be fused with the predicted location.

[0102] Method 2: The electronic device determines the last measurement position among the multiple measurement positions as the measurement position to be fused with the predicted position.

[0103] For example, such as Figure 14 As shown, assuming three measurement locations, such as measurement locations 1 to 3, are acquired within a preset time period before the first moment. It should be understood that the closer the measurement location is to the first moment, the stronger its timeliness. Therefore, the electronic device can determine measurement location 3 among the three measurement locations as the measurement location used for fusion with the predicted location.

[0104] S804, the electronic device determines the display position of the target object at a first moment based on the predicted position.

[0105] Among them, such as Figure 15 As shown, if the electronic device determines that it has not acquired the measurement position of the target object within the preset time period, the electronic device will use the predicted position as the display position of the target object at the first moment.

[0106] It should be understood that when the electronic device is blocked from detecting the target object and fails to complete the detection within the preset time, directly using the predicted position of the target object at the first moment as the output can significantly improve the system's display output speed and reduce latency.

[0107] S805, the electronic device enables the display device to display a prompt message for the target object at the first moment and display position.

[0108] In one scenario, when the electronic device has projection and / or display functions, it can project the prompt information of the target object onto the windshield of the driving vehicle at a specific moment and display location.

[0109] In another scenario, when the electronic device lacks projection and / or display capabilities, it can send control commands to a connected display screen, such as an in-vehicle display or AR-HUD, via an interface circuit. This causes the receiving display screen to display the prompt information for the target object at the first moment. The control command can instruct the display screen to display corresponding content. Alternatively, the electronic device can send control commands to a connected display screen, such as an in-vehicle display or AR-HUD, via a wireless connection, such as Bluetooth. This causes the receiving display screen to display the prompt information for the target object at the first moment. The control command can also instruct the display screen to display corresponding content.

[0110] Furthermore, in the second case of embodiment S805 of this application, the electronic device enables the HUD to display the prompt information of the target object at the first moment in a manner that is not limited to the following: Display Method 1: The electronic device performs a projection coordinate system transformation and sends the transformed display position of the target object to the HUD for projection.

[0111] When the electronic device performs HUD coordinate system transformation on the display position of the target object at time i, it can be achieved in the following ways: The electronic device maps the display position in the world coordinate system to the vehicle coordinate system based on the correspondence between the world coordinate system and the vehicle coordinate system.

[0112] In one optional embodiment of this application, the electronic device can determine and establish the correspondence between the world coordinate system and the vehicle body coordinate system of the display location in the following manner, and then perform coordinate system transformation on the display location according to the correspondence.

[0113] For example, the internal parameters of the electronic device are first determined.

[0114] The internal parameters of the electronic device include, but are not limited to, the three attitudes of the electronic device: roll, yaw, and pitch.

[0115] For example, a checkerboard pattern is placed in front of the electronic device multiple times to calibrate its intrinsic parameters. Then, based on the intrinsic parameters of the electronic device, a checkerboard pattern is placed vertically in front of the HUD vehicle body, and the extrinsic parameters rotation matrix R and offset vector T of the electronic device are calibrated. The offset of the checkerboard pattern relative to the HUD vehicle body coordinate system is measured.

[0116] The offset of the HUD vehicle body coordinate system can be determined according to the following formula 1.

[0117] Formula 1

[0118] in, This represents the offset of the checkerboard pattern relative to the HUD vehicle coordinate system in the x, y, and z directions. Indicates the offset in the x-direction; Indicates the offset in the y-direction; This represents the offset in the z-direction.

[0119] Furthermore, after determining the intrinsic parameters of the electronic device and the offset between the electronic device and the HUD vehicle coordinate system, the corresponding HUD vehicle coordinate system can be obtained according to the following formula 2.

[0120] Formula 2

[0121] in, This represents the coordinate in the vehicle's coordinate system in the x-direction. This represents the coordinate in the vehicle's coordinate system along the y-direction. This represents the vehicle's coordinates in the z-direction within the vehicle's coordinate system. This represents the x-axis coordinate in the world coordinate system. This represents the coordinate in the world coordinate system along the y-direction. This represents the z-axis coordinate in the world coordinate system.

[0122] Furthermore, the electronic device sends the initial display position of the converted target object to the HUD, which then projects a prompt message about the target object onto the windshield of the vehicle based on the display position.

[0123] Projection Method 2: The electronic device sends the display position of the target object at the first moment to the HUD, and the HUD performs the projection coordinate system transformation automatically.

[0124] When the HUD in a driving vehicle transforms the initial display position of the target object into the HUD coordinate system, it can be done in the following way: Based on this correspondence, the HUD maps the display position in the world coordinate system to the vehicle body coordinate system.

[0125] In this embodiment of the application, the HUD determines and establishes the correspondence between the world coordinate system where the display position is located and the vehicle body coordinate system. For details, please refer to the above description of projection method 1. For the sake of brevity, it will not be repeated here.

[0126] Furthermore, the HUD can also render corresponding prompts for the target object based on the target object's initial display position, and project them onto the windshield via an optical engine, presenting them to the driver's eyes. This allows the driver to receive real-time warnings for pedestrians and vehicles, enabling real-time warnings for pedestrians and vehicles.

[0127] Furthermore, the determination of the display position in this embodiment may also be achieved in the following ways: Method 1: The display position is related to a preset correction value.

[0128] In this embodiment, the correction value may be preset to remove errors caused by the vehicle's bumps and swaying during driving.

[0129] In one optional embodiment of this application, the electronic device may use different correction values ​​to make corrections based on different driving scenarios.

[0130] The electronic device can determine the driving scenario based on different road conditions. For example, driving on a relatively flat asphalt road can be considered a city driving scenario; driving on a narrow and steep road section can be considered a mountain road driving scenario.

[0131] Furthermore, embodiments of this application can obtain vehicle bump and sway conditions under different driving scenarios based on big data analysis, and determine the correction values ​​corresponding to different scenarios based on the vehicle bump and sway conditions.

[0132] Understandably, on a flat asphalt road, the vehicle is relatively stable and the correction value is small; on mountainous terrain, the vehicle is more bumpy and the correction value is larger.

[0133] For example, assuming the current driving scenario is in an urban area, based on the correspondence between the driving scenario and the correction value, the correction value in this scenario is an offset of 0.5 meters towards the target object's direction of travel.

[0134] For example, if the display position is not further obtained based on the correction value, the display position of the target object is as follows: Figure 16 As shown in (a) above. When the display device further obtains the display position based on the correction value before projection, the display position of the target object after projection at the first moment is as follows. Figure 16 As shown in (b) above. Compared to Figure 16 The display position in (a) is offset by 0.5 meters.

[0135] Method 2: The display position is related to the average of the display positions corresponding to multiple adjacent times before the first time.

[0136] For example, the electronic device in this application embodiment can update the display position of the first position of the target object through sliding window filtering.

[0137] The electronic device determines a mean filter based on the display positions of at least two adjacent frames. Then, based on the mean filter and a preset step size, the electronic device performs a sliding window filter to update the display position of the target object at its first location.

[0138] The mean filtering in this embodiment can be determined using the following formula 3: Formula 3 Where k represents the location information of the target object in the k-th frame, and n represents the number of frames selected for mean filtering. This indicates mean filtering.

[0139] Assuming, such as Figure 17 As shown, by selecting three neighboring frames for mean filtering and performing sliding window filtering with a preset step size of 1, the electronic device can obtain a relatively smooth and stable display position output. Each Box represents the display position corresponding to each frame.

[0140] Furthermore, in order to effectively save system power consumption, the electronic device in this application embodiment can determine that the vehicle is in motion before displaying the information.

[0141] This application embodiment can determine whether the vehicle is in motion based on one or more of the following conditions: Condition 1: Is the engine of the vehicle running?

[0142] Condition 2: Whether the distance the vehicle travels within a certain time period is greater than the threshold distance.

[0143] Condition 3: Is the gear used when driving the vehicle P (Park) or D (Drive)?

[0144] In addition, in order to effectively improve the output speed of the electronic device in displaying the location of the target object, an optional approach in this application embodiment is to use the MQ parallel mechanism during the road condition display process.

[0145] Specifically, the modules of the entire system can be interdependent and sequential processes. The message queue (MQ) mechanism can be used to decouple the modules of the entire system, reduce the coupling between the modules, and make the modules of the entire system parallel. Other methods can also be used, which are not limited here.

[0146] For example, when detecting a target object on a road, if the updated position of the target object is not obtained within a threshold time (i.e., after detection is determined to be blocked), the detection system directly outputs the predicted value of the target object's position at the next moment obtained through the prediction model, which greatly improves the tracking output rate.

[0147] like Figure 18 As shown, when the electronic device includes Figure 6 When the detection module, tracking fusion module, HUD stabilization module, HUD coordinate transformation module, HUD engine rendering module, etc. are shown, the method provided in this application embodiment may include the following steps: S1800, the detection module detects the sensing information of the target object.

[0148] S1801, The tracking fusion module acquires the sensing information of the target object.

[0149] In one optional embodiment of this application, the tracking fusion module can acquire the sensing information of the target object detected by the detection module through an interface circuit; or the tracking fusion module can acquire the sensing information of the target object detected by the connected detection module through a wireless connection, such as a Bluetooth connection.

[0150] S1802. The tracking fusion module obtains the predicted position of the target object at the first moment based on the sensing information of the target object.

[0151] In one optional embodiment of this application, the tracking fusion module can input the acquired sensing information of the target object into a prediction model used to determine the predicted position, thereby obtaining the predicted position of the target object at the first moment.

[0152] In this embodiment, the tracking fusion module can establish a prediction model based on the sensing information of target objects with a threshold number obtained in the past.

[0153] It should be understood that in the embodiments of this application, the tracking fusion module can update and adjust the established prediction model based on the sensor information of the target object.

[0154] S1803. The tracking fusion module determines whether the measurement position has been acquired before the first moment. If yes, execute S1804; otherwise, execute S1805.

[0155] Alternatively, the measured position of the target object before the first moment can be detected by the detection module and then notified to the tracking fusion module. Another alternative is that the measured position of the target object before the first moment can be determined by the tracking fusion module using the sensor information of the target object before the first moment notified by the detection module.

[0156] The specific content of S1803 is similar to that of S802 above. For a concise description, please refer to the content of S802 above.

[0157] S1804. The tracking fusion module determines the display position of the target object at the first moment based on the predicted position and the measured position.

[0158] The tracking fusion module can determine the average value of the predicted position and the measured position as the display position of the target object at the first moment.

[0159] Furthermore, in S1804, if the tracking fusion module acquires multiple measurement positions of the target object before the first moment, it can determine the measurement position to be fused with the predicted position in various ways, including but not limited to the following: Method 1: The tracking fusion module determines the average value of the multiple measurement locations as the measurement location to be fused with the predicted location.

[0160] Method 2: The tracking fusion module determines the last measurement position among the multiple measurement positions as the measurement position to be fused with the predicted position.

[0161] The specific content of S1804 is similar to that of S803 above. For a concise description, please refer to the content of S803 above.

[0162] S1805. The tracking and fusion module determines the display position of the target object at the first moment based on the predicted position.

[0163] An alternative approach is to directly use the predicted position of the target object at the first moment as the output if the tracking fusion module has not obtained the measured position of the target object before the first moment. This can significantly improve the system's display output speed and reduce latency.

[0164] S1806. The tracking and fusion module enables the HUD engine rendering module to display the prompt information of the target object at the first moment and display position.

[0165] If the world coordinate system used by the detection module in the driving vehicle to collect information is different from the vehicle body coordinate system, then in this embodiment of the application, the display position of the target object at the first moment can be adjusted by the HUD coordinate transformation module before projection.

[0166] For example, the HUD coordinate transformation module obtains the display position of the target object at the first moment through the interface circuit, and then maps the display position in the world coordinate system to the vehicle coordinate system according to the correspondence between the world coordinate system and the vehicle coordinate system, thus obtaining the adjusted display position of the target object at the first moment. The HUD coordinate transformation module transmits the adjusted display position of the target object at the first moment to the HUD engine rendering module through the interface circuit.

[0167] Furthermore, in this embodiment of the application, before projecting the display position of the target object through the HUD engine rendering module, in order to better reduce jitter, it can be further determined by the HUD anti-shake module, specifically not limited to the following: Method 1: The HUD stabilization module obtains the display position of the target object at the first moment based on the predicted position, the measured position, and the correction value.

[0168] Method 2: The HUD anti-shake module obtains the display position of the target object at the first moment based on the predicted position, the measured position, and the average of the display positions corresponding to multiple adjacent moments before the first moment.

[0169] according to Figure 18 The process shown can be derived from... Figure 6 The electronic device shown implements the display method provided in the embodiments of this application. It should be understood that... Figure 18 The one shown is made by Figure 6 The steps implemented by the electronic device shown are exemplary embodiments of the display method provided according to the embodiments of this application. Figure 18 Some of the steps shown can be omitted or replaced by other steps. Figure 18 Some steps in the process, or the electronic device may also perform Figure 18 Some steps are not shown.

[0170] Based on the foregoing content and the same concept, this application also provides an electronic device for implementing the functions of the electronic device in the display method described in the above method embodiments, thus possessing the beneficial effects of the above method embodiments. This electronic device may include... Figures 4 to 6 Any structure in, or by Figures 4 to 6 It can be implemented by combining any number of structures.

[0171] like Figure 4 The electronic device shown can be a terminal or vehicle, or it can be a chip inside a terminal or vehicle. This electronic device can achieve, for example... Figure 8 or Figure 18 The display method and the various optional embodiments described above are shown. The electronic device may include a processing module 410 and a data acquisition module 420.

[0172] The processing module 410 can be used to execute Figure 8 S800~S805 in the method shown Figure 18 Any step in S1801-S1806 of the method can be used to perform any step in the optional embodiments described above, such as determining the measurement position of the target object, coordinate system transformation, or determining whether the measurement position of the target object has been acquired before the first moment. The acquisition module 420 is used to acquire the sensing information of the target object. For example, it can be used to perform... Figure 18 The steps S1800, etc., in the method shown may be used to perform any of the steps involving the acquisition of target object information in the optional embodiments described above. See the detailed description in the method examples for specific details, which will not be repeated here.

[0173] The processing module 410 can be used to acquire sensing information of a target object, which includes one or more of vehicles, people, obstacles, and traffic signs; obtain the predicted position of the target object at a first moment based on the sensing information of the target object; acquire the measured position of the target object before the first moment; and enable an electronic device to display a prompt message of the target object at the first moment based on the predicted position and the measured position.

[0174] It should be understood that the electronic device in this application embodiment can be implemented by software, for example, by a computer program or instructions having the above-described functions. The corresponding computer program or instructions can be stored in the terminal's internal memory, and the processor reads the corresponding computer program or instructions from the memory to implement the above-described functions of the processing module 410 and / or the acquisition module 420. Alternatively, the electronic device in this application embodiment can also be implemented by hardware. The processing module 410 can be a processor (such as a CPU or a processor in a system chip), and the acquisition module 420 can include one or more of a camera device and a sensing device.

[0175] In one alternative approach, the processing module 410 may be used for: When the measured position of the target object is obtained before the first moment, the electronic device is enabled to display a prompt message about the target object at the first moment and at the display position, wherein the display position is related to the predicted position and the measured position.

[0176] In one alternative approach, the processing module 410 may be used for: If the measured position of the target object cannot be obtained before the first moment, the electronic device is enabled to display a prompt message about the target object at the first moment and at the display position, wherein the display position is related to the predicted position.

[0177] In one alternative approach, the processing module 410 may be used for: When multiple measurement positions of the target object are acquired before the first moment, the electronic device is enabled to display a prompt message about the target object at the first moment and at a display position, wherein the display position is related to the average value of the multiple measurement positions and the predicted position.

[0178] In one alternative approach, the processing module 410 may be used for: When multiple measurement positions of the target object are acquired before the first moment, the electronic device is enabled to display a prompt message of the target object at the first moment and at the display position. The display position is related to the last measurement position acquired among the multiple measurement positions and the predicted position.

[0179] In one alternative approach, the display position is also related to a preset correction value used to reduce errors caused by vehicle bumps and swaying during driving.

[0180] In one alternative approach, the display position is related to the average of the display positions corresponding to multiple adjacent times prior to the first time.

[0181] In one alternative approach, the target object includes one or more of a vehicle, a person, an obstacle, and a traffic sign.

[0182] It should be understood that the processing details of the electronic device in the embodiments of this application can be found by referring to... Figure 8 , Figure 18 The relevant descriptions in the embodiments of the method of this application will not be repeated here.

[0183] like Figure 5 The electronic device shown can be a terminal or vehicle, or it can be a chip inside a terminal or vehicle. This vehicle control device can achieve, for example... Figure 8 or Figure 18 The vehicle control method and the various optional embodiments described above are illustrated. The electronic device may include at least one of a processor, memory, interface circuitry, or human-machine interface device. It should be understood that, although... Figure 5 The diagram shows only one processor, one memory, one interface circuit, and one (or one) acquisition device. The electronic device may include other numbers of processors and interface circuits.

[0184] Interface circuits are used to connect electronic devices to other components of a terminal or vehicle, such as memory or other processors, or projection devices. Processors can be used to interact with other components via interface circuits. Interface circuits can be the processor's input / output interfaces.

[0185] For example, a processor can read computer programs or instructions from a coupled memory via an interface circuit, and then decode and execute these computer programs or instructions. It should be understood that these computer programs or instructions may include the aforementioned functional programs, or they may include the functional programs of the aforementioned electronic device. When the corresponding functional program is decoded and executed by the processor, the electronic device can implement the display method scheme provided in the embodiments of this application.

[0186] Optionally, these functional programs are stored in external memory of the electronic device, in which case the electronic device may not include a memory. When the aforementioned functional programs are decoded and executed by the processor, part or all of the contents of the aforementioned functional programs are temporarily stored in the memory.

[0187] Optionally, these functional programs are stored in the internal memory of the electronic device. When the aforementioned functional programs are stored in the internal memory of the electronic device, the electronic device can be installed in the electronic device of this application embodiment.

[0188] Optionally, these functional programs are stored in memory outside the road condition television device, while other parts of these functional programs are stored in memory inside the electronic device.

[0189] It should be understood that the aforementioned processor can be a chip. For example, the processor can be a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0190] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as execution by a hardware decoding processor, or as a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0191] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0192] It should be understood that, through Figure 5 When the structure shown implements this electronic device, a memory can store computer programs or instructions, and a processor can execute the computer programs or instructions stored in the memory to perform operations via... Figure 4 The structure shown implements the actions performed by the processing module 410 when the electronic device is in operation. These actions can also be performed by the acquisition device 420 when the data is transmitted through… Figure 4 The structure shown enables the electronic device to acquire sensor information from the target object. Optionally, it can be... Figure 5 The processor and memory implementation shown Figure 4 The processing module 410 shown, or rather, Figure 4 The processing module 410 shown includes a processor and a memory; in other words, the processor executes the computer program or instructions stored in the memory to implement the above-mentioned functions. Figure 4 The actions performed by the processing module 410 shown. And / or, may be performed by... Figure 5 The data acquisition device shown achieves Figure 4 The acquisition module 420 shown, or rather, Figure 4 The processing module 410 shown includes Figure 5 The data acquisition device shown, or rather, the data acquisition device that performs the above... Figure 4 The actions performed by the acquisition device 420 shown.

[0193] In passing Figure 6 When the structure shown is used to implement this electronic device, it can be executed by one or more of the following modules: detection module, tracking fusion module, HUD stabilization module, HUD coordinate transformation module, and HUD engine rendering module. Figure 4 The structure shown implements the actions performed by the processing module 410 when the electronic device is in operation. These actions can also be performed by the detection module upon passing through... Figure 4 The structure shown enables the electronic device to perform actions via the acquisition module 420. (The sentence is incomplete and likely refers to a separate process.) Figure 6 When the structure shown implements this electronic device, the actions performed by the detection module, tracking fusion module, HUD stabilization module, HUD coordinate transformation module, and HUD engine rendering module can be referred to [reference needed]. Figure 17 The explanations in the process shown will not be repeated here.

[0194] It should be understood that Figures 4 to 6 The structures of any of the electronic devices shown can be combined with each other. Figures 4 to 6 The design details of any of the electronic devices shown and the various alternative embodiments can be referenced interchangeably or further. Figures 4 to 6 The display method shown and the design details of each optional embodiment are not repeated here.

[0195] Based on the above content and the same concept, this application provides a computing device, including a processor connected to a memory, the memory for storing computer programs or instructions, and the processor for executing the computer program stored in the memory, so that the computing device performs the method in the above method embodiment.

[0196] Based on the above content and the same concept, this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed, causes a computing device to perform the methods described in the above method embodiments.

[0197] Based on the above content and the same concept, this application provides a computer program product that, when executed by a computer, causes a computing device to perform the methods described in the above method embodiments.

[0198] Based on the above content and the same concept, this application provides a chip connected to a memory for reading and executing computer programs or instructions stored in the memory, so that a computing device performs the methods in the above method embodiments.

[0199] Based on the above content and the same concept, this application provides an apparatus comprising a processor and an interface circuit, wherein the interface circuit is configured to receive a computer program or instructions and transmit them to the processor; the processor executes the computer program or instructions to perform the method described in the above method embodiments.

[0200] It should be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0201] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0202] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0203] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0204] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0205] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A display method, characterized in that, include: Acquire sensor information of the target object; The sensing information is input into a prediction model used to determine the predicted location of the target object, and the predicted location is obtained. At least based on the predicted position, the display device is enabled to display a prompt message about the target object at the display position; The displayed position is related to the predicted position and / or the measured position of the target object.

2. The method according to claim 1, characterized in that, The step of enabling the display device to display the prompt information of the target object at the display position based at least on the predicted position includes: If the measured position is not obtained within the threshold time period, the display position is determined based on the predicted position obtained through the prediction model.

3. The method according to claim 1, characterized in that, The step of enabling the display device to display the prompt information of the target object at the display position based at least on the predicted position includes: When the vehicle is in motion, the display device is enabled to display the prompt information at the display location.

4. The method according to claim 3, characterized in that, The state of being in motion includes at least one of the following: The engine is running, the distance traveled within a certain time exceeds a threshold distance, and the gear is in D mode.

5. The method according to claim 1, characterized in that, include: When the measured position of the target object is obtained, the displayed position is related to the predicted position and the measured position.

6. The method according to claim 1, characterized in that, include: When the measured position of the target object cannot be obtained, the displayed position is related to the predicted position.

7. The method according to claim 1, characterized in that, include: When multiple measurement locations of the target object are obtained, the displayed location is related to the average value of the multiple measurement locations and the predicted location.

8. The method according to claim 1, characterized in that, include: When multiple measurement positions of the target object are obtained, the displayed position is related to the last measurement position obtained among the multiple measurement positions and the predicted position.

9. The method according to any one of claims 1 to 8, characterized in that, The display position is also related to a preset correction value, which is used to reduce errors caused by vehicle bumps and swaying during driving.

10. The method according to claims 1 to 8, characterized in that, The display position is also related to the average of the display positions corresponding to multiple adjacent times before the first time.

11. The method according to any one of claims 1 to 10, characterized in that, The target objects include one or more of vehicles, people, obstacles, and traffic signs.

12. An electronic device, characterized in that, Includes a processing module and an acquisition module: The acquisition module is used to acquire the sensing information of the target object; The processing module is configured to input the sensing information into a prediction model for determining the predicted position of the target object, obtain the predicted position, and enable the display device to display a prompt message of the target object at a display position, at least based on the predicted position. The displayed position is related to the predicted position and / or the measured position of the target object.

13. The electronic device according to claim 12, characterized in that, The processing module is also used for: If the measured position is not obtained within the threshold time period, the display position is determined based on the predicted position obtained through the prediction model.

14. The electronic device according to claim 12, characterized in that, The processing module is also used for: When the vehicle is in motion, the display device is enabled to display the prompt information at the display location.

15. The electronic device according to claim 14, characterized in that, The state of being in motion includes at least one of the following: the engine is running, the distance traveled within a certain period of time is greater than a threshold distance, and the gear is in D gear.

16. The electronic device according to claim 12, characterized in that, The processing module is used for: When the measured position of the target object is obtained, the display device is enabled to display a prompt message of the target object at the display position, which is related to the predicted position and the measured position.

17. The electronic device according to claim 12, characterized in that, The processing module is used for: If the measured position of the target object cannot be obtained before the first moment, the display device is enabled to display a prompt message about the target object at the first moment and the display position, wherein the display position is related to the predicted position.

18. The electronic device according to claim 12, characterized in that, The processing module is used for: When multiple measurement positions of the target object are obtained, the display device is enabled to display a prompt message of the target object at the display position, which is related to the average value of the multiple measurement positions and the predicted position.

19. The electronic device according to claim 12, characterized in that, The processing module is used for: When multiple measurement positions of the target object are obtained, the display device is enabled to display a prompt message of the target object at the display position, which is related to the last measurement position obtained among the multiple measurement positions and the predicted position.

20. The electronic device according to any one of claims 12 to 19, characterized in that, The display position is also related to a preset correction value, which is used to reduce errors caused by vehicle bumps and swaying during driving.

21. The electronic device according to any one of claims 12 to 19, characterized in that, The display position is also related to the average of the display positions corresponding to multiple adjacent times before the first time.

22. The electronic device according to any one of claims 12 to 21, characterized in that, The target objects include one or more of vehicles, people, obstacles, and traffic signs.

23. A computing device, characterized in that, The device includes a processor connected to a memory that stores computer programs or instructions, the processor executing the computer programs or instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 11.

25. A chip, characterized in that, Includes at least one processor and interface; The interface is used to provide computer programs, instructions or data to the at least one processor; The at least one processor is configured to execute the computer program or instructions such that the method as claimed in any one of claims 1 to 11 is performed.

Images