Information prompting method and device applied to vehicle, equipment and storage medium

Abstract

The invention discloses an information prompting method and device applied to a vehicle, equipment and a storage medium. According to the specific scheme, under the condition that a target vehicle is in an auxiliary driving mode, to-be-used data of the target vehicle under multiple collection dimensions are obtained; analyzing and processing the to-be-used data based on an intelligent driving perception model, and determining traffic participation element information in a driving environment where the target vehicle is located and element relative type information of traffic participation elements and the target vehicle; determining target prompt information corresponding to the traffic participation element information based on the element relative type information; wherein the target prompt information comprises relative information of the traffic participation elements and the target vehicle, driving guidance information and guidance basis information of the driving guidance information. Through the driving guidance information and the guidance basis information, the credibility of the driving user can be improved, the interactive guidance for the driving user is improved, and the safe driving of the vehicle is ensured.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an information prompting method, device, equipment and storage medium used in a vehicle. Background Art

[0002] Currently, the interaction between cars and drivers is mainly achieved through the vehicle's head-up display and the vehicle's center console. The vehicle's head-up display and center console display the results of autonomous driving decisions, allowing the vehicle to drive according to the results of the autonomous driving decisions.

[0003] However, existing technologies primarily rely on a single sensor to acquire data and output only the autonomous driving decision results. This can obscure the boundaries of autonomous driving for drivers, making it difficult for them to capture key information while the vehicle is in motion and to determine the reliability of the autonomous driving decision results, leading to a loss of trust. Furthermore, relying on data from a single sensor can easily lead to misjudgments that could result in collision risks, compromising driving safety. Summary of the Invention

[0004] The present invention provides an information prompt method, device, equipment and storage medium for use in a vehicle, which can improve the driver's trust through driving guidance information and guidance basis information, improve the interactive guidance for the driver, and ensure the safe driving of the vehicle.

[0005] According to one aspect of the present invention, there is provided an information prompting method for a vehicle, the method comprising:

[0006] When the target vehicle is in the assisted driving mode, obtaining the target vehicle's data to be used in multiple collection dimensions;

[0007] Based on the intelligent driving perception model, the data is analyzed and processed to determine the traffic participating elements in the driving environment of the target vehicle, as well as the relative types of traffic participating elements and elements of the target vehicle;

[0008] Based on the relative type information of the elements, determining the target prompt information corresponding to the traffic participation element information;

[0009] The target prompt information includes relative information between traffic participating elements and target vehicles, driving guidance information, and guidance basis information of the driving guidance information.

[0010] According to another aspect of the present invention, there is provided an information prompting device for use in a vehicle, the device comprising:

[0011] A data acquisition module is used to acquire the target vehicle's data to be used in multiple acquisition dimensions when the target vehicle is in the assisted driving mode;

[0012] An element information determination module is used to analyze and process the usage data based on the intelligent driving perception model to determine the traffic participation element information in the driving environment of the target vehicle, as well as the relative type information of the traffic participation elements and the elements of the target vehicle;

[0013] A prompt information determination module is used to determine target prompt information corresponding to traffic participation element information based on element relative type information;

[0014] The target prompt information includes relative information between traffic participating elements and target vehicles, driving guidance information, and guidance basis information of the driving guidance information.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the information prompt method applied to a vehicle according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for enabling a processor to implement the information prompt method applied to a vehicle according to any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the information prompt method applied in a vehicle according to any embodiment of the present invention.

[0021] The technical solution of the embodiment of the present invention obtains the target vehicle's data to be used under multiple collection dimensions when the target vehicle is in assisted driving mode. Data collection is performed through multiple collection dimensions, which ensures the comprehensiveness of data collection and is conducive to improving the accuracy of subsequent prompt information. The intelligent driving perception model analyzes and processes the data to be used, determines the traffic participation element information in the driving environment where the target vehicle is located, and the element relative type information of the traffic participation element and the target vehicle, so as to determine the target prompt information corresponding to the traffic participation element information through the element relative type information. The present invention enables the driving user to determine whether the driving guidance information is reliable through the relative information, driving guidance information and guidance basis information in the target prompt information, solves the problem of lack of trust of driving users in the prior art, and improves the trust of driving users. Determining the target prompt information through the data to be used under multiple collection dimensions can ensure the accuracy of the target prompt information, avoid the problem of vehicle collision risk due to misjudgment, improve the interactive guidance for the driving user, and ensure the safe driving of the target vehicle.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flow chart of an information prompt method applied in a vehicle provided by an embodiment of the present invention;

[0025] Figure 2 This is an example diagram of a process for determining and displaying prompt information provided by an embodiment of the present invention;

[0026] Figure 3 This is a flow chart of an information prompt method applied in a vehicle provided by an embodiment of the present invention;

[0027] Figure 4 This is a flowchart illustrating an information prompting method applied to a vehicle provided by an embodiment of the present invention;

[0028] Figure 5 This is a schematic structural diagram of an information prompting device for use in a vehicle, provided by an embodiment of the present invention;

[0029] Figure 6 The present invention is a schematic structural diagram of an electronic device for implementing the information prompt method applied in a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] Figure 1 This is a flowchart of an information prompt method applied in a vehicle provided by the first embodiment of the present invention. This embodiment can be applied to determine the relative information of the target vehicle, driving guidance information, and guidance basis information of the driving guidance information through the data to be used in multiple acquisition dimensions, so as to prompt the driving user through the above prompt information to ensure the safety of the driving user's automatic driving decision. This method can be executed by an information prompt device applied in a vehicle. The information prompt device applied in a vehicle can be implemented in the form of hardware and / or software. The information prompt device applied in a vehicle can be configured in electronic devices such as mobile phones, computers or servers. Figure 1 As shown, the method includes:

[0034] S110 : When the target vehicle is in the assisted driving mode, obtain data to be used of the target vehicle in multiple collection dimensions.

[0035] Assisted driving mode can be a technology that provides additional safety assistance and driving convenience to the driver of the target vehicle. This can alleviate the driver's burden to a certain extent and improve the safety of the target vehicle. The target vehicle can be a vehicle that is currently in motion and requires prompting for autonomous driving decisions.

[0036] Multiple acquisition dimensions include two or more of the following: camera acquisition dimension deployed on the target vehicle; lidar acquisition dimension; millimeter wave radar acquisition dimension; inertial measurement unit acquisition dimension; high-precision map data acquisition dimension.

[0037] The data to be used in the camera acquisition dimension may be a forward-looking image of the target vehicle's direction of travel, acquired by a camera deployed in the target vehicle. The forward-looking image may include traffic elements that affect the target vehicle's travel. Traffic elements can be understood as various entities and factors that directly or indirectly participate in traffic activities related to the target vehicle and have an impact on the target vehicle's travel. Optionally, traffic elements may include pedestrians, vehicles, and other objects in the target vehicle's environment. The data to be used in the lidar acquisition dimension may be three-dimensional point cloud data of traffic elements in the target vehicle's environment, acquired by light detection and ranging (LiDAR). The data to be used in the millimeter-wave radar acquisition dimension may include data such as the relative distance, relative speed, angle, and travel direction between the target vehicle and the traffic elements. The data to be used in the inertial measurement unit acquisition dimension may include the target vehicle's position, speed, and direction information acquired by an inertial measurement unit (IMU). The data to be used in the high-precision map data collection dimension can be road data and map information collected by the Global Positioning System (GPS) or high-precision map collection equipment.

[0038] Specifically, when the target vehicle is in assisted driving mode, data collection devices corresponding to at least two of the following collection dimensions: camera, lidar, millimeter-wave radar, inertial measurement unit, and high-precision map data, are used to obtain the target vehicle's available data in those dimensions. Collecting data across multiple dimensions ensures comprehensiveness and improves the accuracy of subsequent prompts.

[0039] For example, see Figure 2 , Figure 2This is an example diagram of the process for determining and displaying prompt information. When the target vehicle is driving in assisted driving mode, the camera collects the corresponding image data, the LiDAR collects the three-dimensional point cloud data of the traffic elements in the target vehicle's environment, the millimeter-wave radar (Radar) collects the relative distance, relative speed, angle, and driving direction between the target vehicle and the traffic elements, the IMU collects the target vehicle's position information, speed information, and direction information, and the GPS or high-precision map collection equipment collects the road data and map information. The collected image data, point cloud data, vehicle body data (position information, speed information, and direction information), and positioning data are used for subsequent data processing.

[0040] S120. Analyze and process the usage data based on the intelligent driving perception model to determine the traffic participation element information in the driving environment of the target vehicle, as well as the relative type information between the traffic participation elements and the elements of the target vehicle.

[0041] Optionally, the intelligent driving perception model may include: a data processing module, a perception module, a planning and control module, and a multi-source data fusion module. The data processing module is used to perform data preprocessing and time alignment on the data to be used. The perception module is used to determine the traffic participant element information, perception evaluation value, and relative element type information in the driving environment of the target vehicle. The planning and control module and the multi-source data fusion module are used to determine target prompt information based on the data information determined by the perception module and the processed data to be used.

[0042] The driving environment of the target vehicle may be an area determined by taking the target vehicle as the center and a preset distance as the radius. Traffic participation element information may be perception information corresponding to traffic participation elements that directly or indirectly participate in the driving of the target vehicle. For example, traffic participation element information may include: location information of other vehicles in the driving environment to which the target vehicle belongs. Element relative type information may be used to characterize the relationship type between traffic participation elements and the target vehicle. Optionally, element relative type information may include two types, one is: the traffic participation element is the camera or lidar of the target vehicle itself, and the other is: the traffic participation element belongs to other vehicles or pedestrians in the driving environment to which the target vehicle belongs.

[0043] Specifically, the data to be used is pre-processed and time-aligned by the data processing module in the intelligent driving perception model to obtain the processed data to be used. The processed data to be used is perceived by the perception module in the intelligent driving perception model to determine the traffic participation element information in the driving environment of the target vehicle, as well as the relative types of the traffic participation elements and the elements of the target vehicle. Optionally, the perception module in the intelligent driving perception model can also be used to determine the perception evaluation value corresponding to the traffic participation element information. For example, the traffic participation element information can be that the camera of the target vehicle detects dirt, and the perception evaluation value can be 60%. That is, the camera occlusion of the target vehicle reaches 60%, which is abnormally dirty.

[0044] For example, in conjunction with the above examples, see Figure 2 After the data processing module of the intelligent driving perception model receives the data to be used under multiple acquisition dimensions, the data processing module can perform data preprocessing and time alignment processing on the data to be used under all acquisition dimensions to obtain the processed data to be used. The perception module performs perception processing on the processed data to be used to determine the perception result, that is, to determine the traffic participation element information in the driving environment where the target vehicle is located, the element relative type information of the traffic participation element and the target vehicle, and the perception evaluation value corresponding to the traffic participation element. For example, the traffic participation element information is that the camera is dirty. Since the camera is a component in the target vehicle, the element relative type information belongs to the first type mentioned above, that is, the traffic participation element is the camera or lidar of the target vehicle itself. Correspondingly, the perception evaluation value can be the proportion of the dirty part of the camera to the entire display part of the camera. For example, the perception evaluation value can be 60%. That is, the element relative type information belongs to the first type, the camera is blocked by 60%, and it is abnormally dirty.

[0045] S130. Determine target prompt information corresponding to the traffic participation element information based on the element relative type information.

[0046] The target prompt information includes relative information between the traffic participant elements and the target vehicle, driving guidance information, and guidance basis information for the driving guidance information. The relative information between the traffic participant elements and the target vehicle can be used to characterize the relative positional relationship between the traffic participant elements and the target vehicle. The driving guidance information can be execution logic information used to guide the driver or the assisted driving system in the target vehicle to control the target vehicle's driving. It should be noted that the driving guidance information includes a quantified risk value corresponding to controlling the target vehicle in accordance with the driving guidance information. The quantified risk value represents the probability of success when the target vehicle follows the driving guidance information. For example, the driving guidance information may be: Control the target vehicle to change lanes to the left with an 80% success rate. The guidance basis information for the driving guidance information can be determined based on the relative information between the traffic participant elements and the target vehicle, the traffic participant element information, the relative element type information between the traffic participant elements and the target vehicle, and the perception evaluation value corresponding to the traffic participant element information. For example, the guidance basis information may be: The vehicle ahead of the target vehicle in the direction of travel has overturned and occupies 60% of the target vehicle's lane.

[0047] Specifically, after determining the relative type information of the elements, the relative information, driving guidance information, and guidance basis information of the driving guidance information corresponding to the traffic participation element information can be determined based on the relative type information of the elements. Optionally, a complete logical chain for controlling the driving of the target vehicle can be generated based on the relative information, driving guidance information, and guidance basis information of the driving guidance information, and the complete logical chain can be displayed on the head-up display (HUD) or central control display of the target vehicle to improve the interactive friendliness between the driving user and the target vehicle, so that the driving user can establish trust with the assisted driving mode and ensure that the vehicle can drive normally and safely.

[0048] Exemplarily, in combination with the above example, the guidance basis information is determined based on the relative type information of the elements, the traffic participation element information and the perception evaluation value output by the perception module. The planning and control module of the intelligent driving perception model processes the relative type information of the elements, the traffic participation element information and the perception evaluation value to determine the relative information between the traffic participation element and the target vehicle and the driving guidance information. According to the multi-source data fusion module (AR-Creator), the guidance basis information, the relative information between the traffic participation element and the target vehicle, and the driving guidance information are combined with the processed data to be used to generate target prompt information to display the target prompt information in the HUD of the target vehicle.

[0049] Optionally, after determining the target prompt information corresponding to the traffic participation element information, the method also includes: displaying the target prompt information on the display interface of the target vehicle, and in response to an event of detecting a target operation, updating and processing the data to be used based on the operation information corresponding to the target operation to obtain data to be processed; processing the data to be processed based on the intelligent driving perception model, determining the updated target prompt information and displaying it.

[0050] Among them, the display interface can be an interface corresponding to the head-up display or central control display screen located in the target vehicle. The event of the target operation can be the operation performed by the driving user of the target vehicle after receiving the target prompt information. The operation information corresponding to the target operation can be the driving behavior data information when the driving user performs the target operation. For example, the driving user controls the target vehicle to change lanes to the left. The data to be processed can be the data obtained after the data to be used is updated by the data obtained under multiple acquisition dimensions after the event of the target operation is detected. The updated target prompt information can be the prompt information obtained after the data to be processed is analyzed and processed by the intelligent driving perception model.

[0051] Specifically, after determining the target prompt information, the target prompt information can be displayed in the display interface corresponding to the head-up display or central control display screen in the target vehicle. After the target prompt information is displayed, if it is detected that the driver of the target vehicle performs a target operation after receiving the target prompt information, in response to the event of detecting the target operation, and in the process of detecting the execution of the target operation, data under multiple acquisition dimensions are obtained and based on these data, the data to be used is updated and processed to obtain updated data to be used, that is, data to be processed. The data to be processed is analyzed and processed by multiple modules of the intelligent driving perception model to obtain updated target prompt information and display it on the display interface of the target vehicle.

[0052] It should be noted that after determining the target prompt information, a three-dimensional scene video can be generated based on the target prompt information and the front view image in the data to be used, and the three-dimensional scene video and the target prompt information can be displayed on the display interface of the target vehicle.

[0053] The technical solution of this embodiment obtains the target vehicle's data to be used under multiple collection dimensions when the target vehicle is in assisted driving mode. Data collection is performed through multiple collection dimensions, which ensures the comprehensiveness of data collection and is conducive to improving the accuracy of subsequent prompt information. The intelligent driving perception model analyzes and processes the data to be used, determines the traffic participation element information in the driving environment where the target vehicle is located, and the element relative type information of the traffic participation element and the target vehicle, so as to determine the target prompt information corresponding to the traffic participation element information through the element relative type information. The present invention enables the driving user to determine whether the driving guidance information is reliable through the relative information, driving guidance information and guidance basis information in the target prompt information, solves the problem of lack of trust of driving users in the prior art, and improves the trust of driving users. Determining the target prompt information through the data to be used under multiple collection dimensions can ensure the accuracy of the target prompt information, avoid the problem of vehicle collision risk due to misjudgment, improve the interactive guidance for the driving user, and ensure the safe driving of the target vehicle.

[0054] Example 2

[0055] Figure 3 This is a flowchart of an information prompt method applied to a vehicle provided by the second embodiment of the present invention. This embodiment is a preferred embodiment of the above embodiment. Its specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here. Figure 3 As shown, the method includes:

[0056] S210: When the target vehicle is in the assisted driving mode, obtain data to be used of the target vehicle in multiple collection dimensions.

[0057] S220: Analyze and process the usage data based on the intelligent driving perception model to determine the traffic participant element information that interacts with the target vehicle in the driving environment of the target vehicle.

[0058] Specifically, the data processing module of the intelligent driving perception model performs data preprocessing and time alignment on the target data to obtain the processed target data. The perception module of the intelligent driving perception model performs perception processing on the processed target data to determine the traffic participant element information that interacts with the target vehicle in the driving environment.

[0059] S230: Determine relative type information of the traffic participating elements based on the traffic participating element information and the relative position information of the target vehicle.

[0060] Relative position information can be used to characterize the relative positional relationship between traffic elements and the target vehicle. This information can be used to determine whether a traffic element is a component of the target vehicle itself, or another vehicle or pedestrian in the driving environment.

[0061] Specifically, relative position information is determined based on the position information corresponding to the traffic participating element in the traffic participating element information and the position information of the target vehicle. Based on the relative position information, element relative type information of the traffic participating element is determined, and the text displayed in the target prompt information is determined based on the element relative type information.

[0062] In an embodiment of the present invention, the method of determining the relative type information of the element through the traffic participation element information and the relative position information can be: determining the relative position information based on the element position information of the traffic participation element information and the vehicle position information of the target vehicle; and determining the element relative type information of the traffic participation element based on the relative position information.

[0063] Among them, the element relative type information includes a first type in which the traffic participation element belongs to the target vehicle, or a second type in which the traffic participation element does not belong to the target vehicle. The traffic participation element information may include multiple aspects of information corresponding to the traffic participation element, that is, including element location information. The element location information is used to characterize the geographical location information corresponding to the traffic participation element. The vehicle location information can be used to characterize the geographical location information corresponding to the target vehicle. It should be noted that the element location information and the vehicle location information can be represented by coordinate data under a unified coordinate system. The first type in the element relative type information may be: the traffic participation element is a component such as a camera or lidar of the target vehicle itself. The second type in the element relative type information may be: the traffic participation element belongs to other vehicles or pedestrians in the driving environment where the target vehicle is located.

[0064] Specifically, the relative position information between the traffic participating element and the target vehicle is determined based on the element position information of the traffic participating element information and the vehicle position information of the target vehicle. Based on the relative position information, it is determined whether the traffic participating element belongs to the first type or the second type in the element relative type information.

[0065] S240. When the relative type information of the element belongs to the first type, the target prompt information is the first type description text; when the relative type information of the element belongs to the second type, the target prompt information is the second type description text.

[0066] The first type of description text includes the cause of the abnormality of a traffic element and the method for handling the abnormality. The abnormality cause can be the factor that caused the abnormality of the traffic element. For example, the camera is dirty, resulting in a camera occlusion of 60%. The method for handling the abnormality cause can be the method for handling the abnormality cause. For example, if the abnormality cause is a camera occlusion of 60% due to camera dirt, the method for handling the abnormality cause may be to control the target vehicle to drive to the target location and then handle the camera dirt at the target location.

[0067] The second type of description text contains the cause of the abnormality of a traffic element and the complete logic chain for controlling the target vehicle's driving based on the abnormality. The cause of the abnormality of a traffic element in the second type of description text can be: factors affecting the normal driving of the target vehicle due to the abnormality of the traffic element. This refers to the guidance basis information in the target prompt information mentioned in the above embodiment. For example, the vehicle ahead of the target vehicle in the direction of travel has overturned and occupies 60% of the target vehicle's lane. The complete logic chain can be the logic chain for controlling the target vehicle's driving based on the abnormality cause, that is, the driving guidance information mentioned in the above embodiment. For example, the complete logic chain can be: the vehicle ahead of the target vehicle in the direction of travel has overturned and occupies 60% of the target vehicle's lane -> the target vehicle has a straight-ahead collision probability of 90% -> a left lane change success probability of 80% -> a left lane change will be made. For example, the complete logic chain can also be: the vehicle ahead of the target vehicle in the direction of travel has overturned and occupies 60% of the target vehicle's lane -> a lane-maintaining success probability of 40% -> no lane change will be made.

[0068] The cause of the anomaly can be determined by the perception module of the intelligent driving perception model. The method for handling the cause of the anomaly can be determined by the planning and control module of the intelligent driving perception model. The complete logical chain can be determined by the multi-source data fusion module of the intelligent driving perception model based on the cause of the anomaly, the method for handling the cause of the anomaly, and the processed data to be used.

[0069] Specifically, when the relative type information of the element belongs to the first type, the abnormal cause of the traffic participation element is determined according to the relative type information of the element, the traffic participation element information and the perception evaluation value corresponding to the traffic participation element, and the processing method of the abnormal cause is determined through the intelligent driving perception model, that is, the target prompt information of the first type of descriptive text is obtained.

[0070] When the relative type information of the elements belongs to the second type, the abnormal cause of the traffic participation element is determined by the intelligent driving perception model for the relative type information of the elements, the traffic participation element information, and the perception evaluation value corresponding to the traffic participation element. The processing method of the abnormal cause is determined by the planning and control module of the intelligent driving perception model, and the complete logical chain is determined by processing the abnormal cause, the processing method of the abnormal cause, and the processed data to be used according to the multi-source data fusion module of the intelligent driving perception model. That is, the target prompt information of the second type of descriptive text is obtained. After obtaining the target prompt information, the target prompt information can be displayed on the display interface of the target vehicle to prompt the driving user of the target vehicle and enhance the driving user's trust in the autonomous driving decision.

[0071] The technical solution of this embodiment obtains the target vehicle's data to be used under multiple collection dimensions when the target vehicle is in assisted driving mode. Data collection is performed through multiple collection dimensions, which ensures the comprehensiveness of data collection and is conducive to improving the accuracy of subsequent prompt information. Based on the intelligent driving perception model, the data to be used is analyzed and processed to determine the traffic participation element information that interacts with the target vehicle in the driving environment to which the target vehicle belongs. According to the traffic participation element information and the relative position information of the target vehicle, the element relative type information of the traffic participation element is determined. According to the type corresponding to the element relative type information, the target prompt information of the corresponding type description text is determined. The present invention can enable the driving user to determine whether the driving guidance information is reliable through the target prompt information, solve the problem of lack of trust of the driving user in the existing technology, and improve the trust of the driving user. By determining the target prompt information through the data to be used under multiple collection dimensions, the accuracy of the target prompt information can be guaranteed, the problem of vehicle collision risk caused by misjudgment is avoided, the interactive guidance for the driving user is improved, and the safe driving of the target vehicle is ensured.

[0072] Example 3

[0073] Figure 4 This is a flowchart of an information prompt method for a vehicle provided by an embodiment of the present invention. This embodiment is an example of the above embodiment. For its specific implementation, please refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here. Figure 4 As shown, the method includes:

[0074] S310: When the target vehicle is in the assisted driving mode, obtain data to be used of the target vehicle in multiple collection dimensions.

[0075] This step corresponds to S110 and S210 mentioned in the above embodiment.

[0076] For example, when the target vehicle is traveling at 120 kilometers per hour in assisted driving mode, data can be collected through the forward-looking camera, side-looking camera, LiDAR, forward-facing radar and other devices in the target vehicle to obtain point cloud data and image data waiting to be used.

[0077] S320. Perform data preprocessing and time alignment processing on the data to be used through the data processing module in the intelligent driving perception model to obtain processed data to be used.

[0078] Specifically, since the data to be used collected from multiple collection dimensions have inconsistent data ranges or time ranges, the data processing module in the intelligent driving perception model can be used to perform data preprocessing and time alignment on the data to be used to obtain the processed data to be used.

[0079] S330: Perform perception processing on the processed data to be used through the perception module in the intelligent driving perception model to obtain a perception result.

[0080] The perception results include: information on traffic participating elements in the driving environment of the target vehicle, relative types of traffic participating elements and elements of the target vehicle, and perception evaluation values.

[0081] The traffic participation element information and the relative element types of the traffic participation elements and the target vehicle in the perception results correspond to the traffic participation element information and relative element types mentioned in the above embodiment. The perception evaluation value in the perception result can be the perception evaluation value corresponding to the traffic participation element information determined by the perception module in the intelligent driving perception model. For example, the traffic participation element information can be that the target vehicle's camera detects dirt, and the perception evaluation value can be 60%. In other words, the target vehicle's camera is blocked by 60%, indicating abnormal dirt.

[0082] Specifically, the perception module of the intelligent driving perception model performs perception processing on the processed data to be used to obtain a perception result, which facilitates the subsequent planning and control of the target vehicle's driving based on the perception result. It should be noted that the guidance basis information mentioned in the above embodiment can be determined through the perception result. In addition, determining the perception evaluation value can ensure the integrity of the subsequent complete logical chain and the comprehensiveness of the prompt information, so as to improve the driver's trust based on the prompt information.

[0083] S340: Process the perception results through the planning and control module in the intelligent driving perception model to determine control logic information corresponding to the perception results. The control logic information includes information for controlling the driving of the target vehicle and the corresponding risk quantification value.

[0084] Among them, the control logic information corresponds to the driving guidance information mentioned in the above embodiment.

[0085] Specifically, the perception results are processed through the planning and control module in the intelligent driving perception model to determine the information for controlling the driving of the target vehicle and the corresponding risk quantification value, so as to generate a complete logic chain based on the control logic information.

[0086] S350. The control logic information, perception results and processed data to be used are processed through the multi-source data fusion module in the intelligent driving perception model to determine the abnormal cause of the traffic participating elements and the complete logical chain for controlling the driving of the target vehicle based on the abnormal cause, and the target prompt information is determined based on the abnormal cause and the complete logical chain.

[0087] The complete logic chain can be used to represent the execution logic chain of controlling the target vehicle's driving. For example, the complete logic chain can be: the vehicle ahead of the target vehicle in the direction of travel has rolled over, occupying 60% of the target vehicle's lane -> the target vehicle has a straight collision probability of 90% -> the probability of successfully changing lanes to the left is 80% -> the lane change to the left is about to begin. The multi-source data fusion module can be Figure 2 AR-Creator mentioned in .

[0088] Specifically, the control logic information, perception results and processed data to be used are processed according to the multi-source data fusion module in the intelligent driving perception model to determine the abnormal cause of the traffic participating elements and the complete logical chain for controlling the driving of the target vehicle based on the abnormal cause. Target prompt information is generated based on the abnormal cause and the complete logical chain to prompt the target user through the target prompt information. It should be noted that the control logic information, perception results and processed data to be used can also be processed by the multi-source data fusion module in the intelligent driving perception model to determine the planned route corresponding to the complete logical chain, and generate target prompt information based on the planned route, abnormal cause and complete logical chain.

[0089] After the target prompt information is determined, the target prompt information can be sent to the HUD of the target vehicle in the form of low-voltage differential signaling (LVDS), so that the target prompt information can be displayed through the HUD.

[0090] S360: Generate a three-dimensional scene video based on the target prompt information and the processed forward-view image in the data to be used, and display the three-dimensional scene video and the target prompt information on a display interface of the target vehicle.

[0091] The 3D scene video can be a 3D simulated scene video of the target vehicle's surroundings changing over time as the target vehicle follows the driving guidance information in the complete logical chain. The front view image can be an image of the front of the target vehicle in the direction of its travel, captured by a forward-looking camera deployed on the target vehicle.

[0092] Specifically, a 3D scene video is generated based on the target prompt information and the processed forward-view image of the data to be used. This 3D scene video and target prompt information are then displayed on the target vehicle's display interface. Based on this, under complex road conditions such as curves and bumpy roads, the target prompt information collected from multiple dimensions is used to establish a position + time space-time coordinate system, generating a 3D simulated scene video. This dynamically aligns the target prompt information with the actual scene, increasing the driver's confidence in the autonomous driving decisions under complex road conditions.

[0093] S370. After the target prompt information and the three-dimensional scene image are displayed on the display interface, in response to the event of detecting the target operation, the data under multiple acquisition dimensions are reacquired to obtain the data to be processed, and based on the data to be processed and the operation information of the target operation, the target prompt information and the three-dimensional scene image are updated and displayed.

[0094] Specifically, if the target prompt information and the three-dimensional scene image are displayed on the display interface, and the driving user performs a target operation based on the displayed target prompt information and the three-dimensional scene image, the data under multiple acquisition dimensions are reacquired to obtain the data to be processed, and the target prompt information and the three-dimensional scene image are updated and displayed based on the data to be processed and the operation information of the target operation.

[0095] For example, the complete logical chain in the target prompt information might be: the vehicle ahead, in the target vehicle's direction of travel, has overturned, occupying 60% of the target vehicle's lane -> 90% probability of a straight-ahead collision -> 80% probability of a successful left lane change -> an impending left lane change. If the driver, after receiving this complete logical chain of the target prompt information, takes control of the target vehicle and changes lanes to the right, data from multiple acquisition dimensions is reacquired and, combined with the target vehicle's speed and tire steering angle, the data to be used is updated in real time to produce the processed data. Based on this processed data and the driver's driving behavior data of the target vehicle changing lanes to the right, the target prompt information and 3D scene image are updated and displayed, achieving a closed-loop feedback loop that precisely aligns with the virtual reality experience.

[0096] The technical solution of this embodiment acquires data to be used from a target vehicle across multiple acquisition dimensions while the target vehicle is in assisted driving mode. Data collection across multiple acquisition dimensions ensures comprehensive data collection and improves the accuracy of subsequent prompt information. The data processing module in the intelligent driving perception model performs data preprocessing and time alignment on the data to be used, obtaining processed data to be used. The perception module in the intelligent driving perception model performs perception processing on the processed data to be used, obtaining a perception result. The planning and control module in the intelligent driving perception model processes the perception result and determines control logic information corresponding to the perception result. The multi-source data fusion module in the intelligent driving perception model processes the control logic information, the perception result, and the processed data to be used, determining the cause of an abnormality in a traffic element and the complete logic chain for controlling the target vehicle's movement based on the abnormality. Target prompt information is then determined based on the abnormality cause and the complete logic chain. A three-dimensional scene video is generated from the target prompt information and the forward-view image. The three-dimensional scene video and target prompt information are then displayed on the target vehicle's display interface, visualizing the dynamic control logic. Displaying the complete logic chain enhances driver confidence and improves interactive guidance for the driver. After the target prompt information and the three-dimensional scene image are displayed on the display interface, in response to the event of detecting the target operation, the data under multiple acquisition dimensions are reacquired to obtain the data to be processed, and based on the data to be processed and the operation information of the target operation, the target prompt information and the three-dimensional scene image are updated and displayed, thereby achieving a precise fit between the driving user feedback loop and the virtual reality. The present invention can enable the driving user to determine whether the driving guidance information is reliable through the target prompt information, solve the problem of lack of trust of the driving user in the prior art, and improve the trust of the driving user. By determining the target prompt information through the data to be used under multiple acquisition dimensions, the accuracy of the target prompt information can be guaranteed, and the problem of vehicle collision risk due to misjudgment can be avoided, thereby improving the interactive guidance for the driving user and ensuring the safe driving of the target vehicle.

[0097] Example 4

[0098] Figure 5 This is a structural diagram of an information prompt device used in a vehicle provided by the fourth embodiment of the present invention. Figure 5 As shown, the device includes: a data acquisition module 410, an element information determination module 420 and a prompt information determination module 430.

[0099] The data acquisition module 410 is used to obtain the to-be-used data of the target vehicle in multiple collection dimensions when the target vehicle is in the assisted driving mode; the element information determination module 420 is used to analyze and process the to-be-used data based on the intelligent driving perception model, and determine the traffic participation element information in the driving environment where the target vehicle is located, as well as the element relative type information of the traffic participation elements and the target vehicle; the prompt information determination module 430 is used to determine the target prompt information corresponding to the traffic participation element information based on the element relative type information; wherein the target prompt information includes the relative information between the traffic participation elements and the target vehicle, the driving guidance information, and the guidance basis information of the driving guidance information.

[0100] The technical solution of this embodiment obtains the target vehicle's data to be used under multiple collection dimensions when the target vehicle is in assisted driving mode. Data collection is performed through multiple collection dimensions, which ensures the comprehensiveness of data collection and is conducive to improving the accuracy of subsequent prompt information. The intelligent driving perception model analyzes and processes the data to be used, determines the traffic participation element information in the driving environment where the target vehicle is located, and the element relative type information of the traffic participation element and the target vehicle, so as to determine the target prompt information corresponding to the traffic participation element information through the element relative type information. The present invention enables the driving user to determine whether the driving guidance information is reliable through the relative information, driving guidance information and guidance basis information in the target prompt information, solves the problem of lack of trust of driving users in the prior art, and improves the trust of driving users. Determining the target prompt information through the data to be used under multiple collection dimensions can ensure the accuracy of the target prompt information, avoid the problem of vehicle collision risk due to misjudgment, improve the interactive guidance for the driving user, and ensure the safe driving of the target vehicle.

[0101] Based on the above embodiment, optionally, the multiple acquisition dimensions include two or more of the following: camera acquisition dimension deployed on the target vehicle; lidar acquisition dimension; millimeter wave radar acquisition dimension; inertial measurement unit acquisition dimension; high-precision map data acquisition dimension.

[0102] Optionally, the element information determination module includes: a traffic participation element information determination unit, which is used to analyze and process the usage data based on the intelligent driving perception model to determine the traffic participation element information that interacts with the target vehicle in the driving environment to which the target vehicle belongs; and a relative type information determination unit, which is used to determine the element relative type information of the traffic participation element based on the traffic participation element information and the relative position information of the target vehicle.

[0103] Optionally, a relative type information determination unit is used to determine relative position information based on element position information of the traffic participation element information and vehicle position information of the target vehicle; and determine element relative type information of the traffic participation element based on the relative position information; wherein the element relative type information includes a first type of traffic participation element belonging to the target vehicle, or a second type of traffic participation element not belonging to the target vehicle.

[0104] Optionally, the prompt information determination module is used to determine that when the element relative type information belongs to the first type, the target prompt information is the first type description text; when the element relative type information belongs to the second type, the target prompt information is the second type description text.

[0105] Optionally, the first type of description text is the abnormal cause of the traffic participating element and the handling method of the abnormal cause; the second type of description text is the abnormal cause of the traffic participating element and the complete logical chain of controlling the target vehicle's driving based on the abnormal cause.

[0106] Optionally, the device also includes: a prompt information display and update module, which is used to display target prompt information on the display interface of the target vehicle, and in response to the event of detecting a target operation, updates the data to be used based on the operation information corresponding to the target operation to obtain data to be processed; processes the data to be processed based on the intelligent driving perception model, determines the updated target prompt information and displays it.

[0107] The information prompt device for use in a vehicle provided by an embodiment of the present invention can execute the information prompt method for use in a vehicle provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0108] Example 5

[0109] Figure 6 1 is a structural diagram of an electronic device provided in Embodiment 5 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0110] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0112] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the information prompt method used in a vehicle.

[0113] In some embodiments, the information prompting method applied in a vehicle can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the information prompting method applied in the vehicle described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the information prompting method applied in the vehicle by any other appropriate means (for example, by means of firmware).

[0114] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs for implementing the in-vehicle information prompting method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that, when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication unit 19, or installed from the storage unit 18, or installed from the ROM 12. When the computer program is executed by the processor 11, the above-mentioned functions defined in the method of the embodiment of the present invention are performed.

[0117] Example 6

[0118] Embodiment 6 of the present invention further provides a computer-readable storage medium storing computer instructions, the computer instructions being used to cause a processor to execute an information prompt method applied to a vehicle, the method comprising:

[0119] When the target vehicle is in the assisted driving mode, the data to be used of the target vehicle in multiple collection dimensions is obtained; based on the intelligent driving perception model, the data to be used is analyzed and processed to determine the traffic participation element information in the driving environment of the target vehicle, as well as the element relative type information of the traffic participation elements and the target vehicle; based on the element relative type information, the target prompt information corresponding to the traffic participation element information is determined; wherein, the target prompt information includes the relative information between the traffic participation elements and the target vehicle, the driving guidance information, and the guidance basis information of the driving guidance information.

[0120] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0122] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0123] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0124] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0125] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An information prompting method used in a vehicle, characterized in that: include: When the target vehicle is in the assisted driving mode, obtaining data to be used of the target vehicle in multiple collection dimensions; Analyzing and processing the data to be used based on the intelligent driving perception model to determine traffic participant element information in the driving environment of the target vehicle, and relative type information between the traffic participant elements and the elements of the target vehicle; Determining target prompt information corresponding to the traffic participation element information based on the element relative type information; The target prompt information includes relative information between traffic participating elements and the target vehicle, driving guidance information, and guidance basis information of the driving guidance information.

2. The method according to claim 1, characterized in that The multiple acquisition dimensions include two or more of the following: The camera acquisition dimensions deployed on the target vehicle; LiDAR acquisition dimensions; Millimeter-wave radar acquisition dimensions; Inertial measurement unit acquisition dimensions; High-precision map data collection dimensions.

3. The method according to claim 1, characterized in that The analyzing and processing the data to be used based on the intelligent driving perception model to determine the traffic participation element information in the driving environment of the target vehicle and the relative type information between the traffic participation elements and the elements of the target vehicle include: Analyzing and processing the data to be used based on the intelligent driving perception model to determine information on traffic participation elements that interact with the target vehicle in the driving environment of the target vehicle; The element relative type information of the traffic participating element is determined according to the traffic participating element information and the relative position information of the target vehicle.

4. The method according to claim 3, characterized in that The determining, based on the traffic participation element information and the relative position information of the target vehicle, the element relative type information of the traffic participation element includes: Determining the relative position information based on the element position information of the traffic participation element information and the vehicle position information of the target vehicle; Determining element relative type information of the traffic participating element according to the relative position information; The element relative type information includes whether the traffic participation element belongs to a first type of the target vehicle, or whether the traffic participation element does not belong to a second type of the target vehicle.

5. The method according to claim 1, wherein The determining, based on the element relative type information, target prompt information corresponding to the traffic participation element information includes: When the relative type information of the element belongs to the first type, the target prompt information is the first type description text; When the relative type information of the element belongs to the second type, the target prompt information is the second type description text.

6. The method according to claim 5, characterized in that The first type of description text is the abnormal cause of the traffic participating element and the handling method of the abnormal cause; the second type of description text is the abnormal cause of the traffic participating element and the complete logical chain of controlling the driving of the target vehicle based on the abnormal cause.

7. The method according to claim 1, characterized in that After determining the target prompt information corresponding to the traffic participation element information, the method further includes: Displaying the target prompt information on a display interface of the target vehicle, and in response to detecting an event of a target operation, updating the data to be used based on the operation information corresponding to the target operation to obtain data to be processed; The data to be processed is processed based on the intelligent driving perception model, and updated target prompt information is determined and displayed.

8. An information prompting device used in a vehicle, characterized in that: include: A data acquisition module is used to acquire the to-be-used data of the target vehicle in multiple acquisition dimensions when the target vehicle is in the assisted driving mode; an element information determination module, configured to analyze and process the data to be used based on an intelligent driving perception model to determine traffic participation element information in the driving environment of the target vehicle, and relative type information between the traffic participation elements and the elements of the target vehicle; a prompt information determining module, configured to determine target prompt information corresponding to the traffic participation element information based on the element relative type information; The target prompt information includes relative information between traffic participating elements and the target vehicle, driving guidance information, and guidance basis information of the driving guidance information.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the information prompt method for use in a vehicle according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the information prompt method applied in a vehicle according to any one of claims 1 to 7 when executed.

Images