Method and device for presenting assisted driving decision-making process

Abstract

The present invention provides a method and device for presenting an assisted driving decision-making process. The method comprises: obtaining decision-related information corresponding to an assisted driving module of a vehicle with the assisted driving function enabled, the decision-related information being used to influence the decision-making process of the assisted driving module; and outputting the decision-related information within the vehicle's cabin so that passengers within the cabin perceive the decision-making process.

Description

Technical Field

[0001] The present invention relates to the field of vehicle assisted driving, and in particular to a method and device for presenting an assisted driving decision process. Background Art

[0002] Currently, assisted driving features are widely used in various vehicles. However, with the increasing development of assisted driving technology, the logic behind it is becoming increasingly opaque and its interpretability is gradually decreasing. Users often struggle to understand the reasons behind and the specific decision-making process for certain assisted driving decisions, leading to a decrease in trust and willingness to use these features. This urgently requires improvement.

[0003] Related technologies often issue reminders to drivers when boundary conditions (such as sensor failure or functional failure) are triggered. However, this approach only informs users of the cause or result of the failure, and still does not allow them to understand the specific decision-making process of assisted driving. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for presenting an assisted driving decision process, in order to solve the deficiencies in the related art by outputting decision-related information of the assisted driving module.

[0005] Specifically, the present invention is achieved through the following technical solutions:

[0006] According to a first aspect of the present invention, a decision-related information method is provided, comprising:

[0007] When the assisted driving function is enabled on the vehicle, obtaining decision-related information corresponding to the assisted driving module of the vehicle, wherein the decision-related information is used to influence the decision-making process of the assisted driving module;

[0008] The decision-related information is output in a cockpit of the vehicle so that passengers in the cockpit can perceive the decision-making process.

[0009] According to a second aspect of the present invention, a device for presenting an assisted driving decision process is provided, comprising:

[0010] An information acquisition unit, configured to acquire decision-making related information corresponding to the assisted driving module of the vehicle when the assisted driving function is enabled on the vehicle, wherein the decision-making related information is used to influence the decision-making process of the assisted driving module;

[0011] An information output unit is used to output the decision-related information in the cabin of the vehicle so that passengers in the cabin can perceive the decision-making process.

[0012] According to a third aspect of the present invention, there is provided a vehicle comprising:

[0013] A processor, a memory for storing instructions executable by the processor, and an assisted driving module for providing assisted driving functions;

[0014] The processor implements the method as described in the first aspect above by running the executable instructions.

[0015] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] The technical solutions provided by the embodiments of the present invention may have the following beneficial effects:

[0017] It can be seen from the above embodiments that, when the vehicle has turned on the assisted driving function, this solution obtains decision-related information that affects the decision-making process of the assisted driving module of the vehicle, and outputs the information in the cabin so that the occupants in the cabin can perceive the decision-making process of the assisted driving module.

[0018] It is understood that this decision-related information influences the decision-making process of the assisted driving module, namely, affects the module's completion of at least one key step in the decision-making chain (such as perception, prediction, and regulation). Therefore, by outputting this decision-related information to the occupants, not only is the interpretability of the decision-making process of the assisted driving module improved, but the occupants can also accurately and comprehensively understand the decision-making process presented based on this information. This helps to improve the transparency of the assisted driving module's decisions and reduce the difficulty for occupants to understand, thereby helping to increase occupants' trust in and willingness to use the assisted driving function. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0020] Figure 1 1 is a schematic diagram of the architecture of an assisted driving system according to an embodiment of the present invention;

[0021] Figure 2 is a flow chart of a method for presenting an assisted driving decision process according to an embodiment of the present invention;

[0022] Figure 3 is a flowchart of another method for presenting an assisted driving decision process according to an embodiment of the present invention;

[0023] Figure 4This is a schematic diagram showing the display effect of decision-related information according to an embodiment of the present invention;

[0024] Figure 5 1 is a schematic diagram of a camera corresponding to an image outside a vehicle shown in an embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of a current installation status evaluation method shown in an embodiment of the present invention;

[0026] Figure 7 is a schematic diagram of a path scoring process according to an embodiment of the present invention;

[0027] Figure 8 is a schematic structural diagram of a vehicle shown in an embodiment of the present invention;

[0028] Figure 9 This is a block diagram of a device for presenting an assisted driving decision process shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of the present invention.

[0030] The terms used in this invention are for the purpose of describing specific embodiments only and are not intended to limit the invention. The singular forms "a," "the," and "the" used in this invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms "first," "second," "third," etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0032] Currently, assisted driving features are widely used in various vehicles. However, with the advancement of assisted driving technology, the logic behind these features is becoming increasingly opaque and its interpretability is gradually decreasing. Users often struggle to understand the reasons behind and detailed process for assisted driving decisions. In other words, they lack visibility into the complete decision-making chain, encompassing key steps such as perception, prediction, and regulation (i.e., planning and control). This results in a decrease in user trust and willingness to use assisted driving features, hindering their widespread adoption and application, and urgently requires improvement.

[0033] Related technologies often only alert the driver when a system boundary condition (such as a sensor failure or malfunction) is triggered. However, this approach only informs the user of the cause or outcome of the failure, but remains unable to understand the specific decision-making process of assisted driving, that is, the complete decision-making chain. Therefore, it still does not effectively solve the aforementioned technical problems.

[0034] To this end, this application proposes a new solution for presenting the assisted driving decision-making process. This solution, by acquiring and proactively displaying decision-related information from the assisted driving module to the occupants, allows users to accurately and comprehensively understand the specific decision-making process of the assisted driving module when implementing the assisted driving function. This improves the explainability and transparency of assisted driving, and further enhances occupants' trust in and willingness to use the assisted driving function. This solution is described in detail below with reference to the accompanying drawings and examples.

[0035] Figure 1 This is a schematic diagram of the hardware architecture of an assisted driving system shown in an embodiment of the present application. Figure 1 As shown, from a hardware perspective, the system can include only vehicle 11 or both vehicle 11 and server 13. If the assisted driving system only includes vehicle 11, the assisted driving module (such as the neural network or visual language model (VLM) described below) can be deployed in vehicle 11, such as in the domain controller of vehicle 11. If the assisted driving system includes both vehicle 11 and server 13, the assisted driving module can be deployed in server 13, and no further details will be given.

[0036] In addition to domain control, the vehicle 11 may also be equipped with various types of sensors for collecting data about the external environment. The number of any type of sensor may be one or more, and the embodiments of the present application do not limit the number of the above sensors and their installation locations. For example, the sensors may include cameras (such as a front-view camera 111, a side-view camera 112, a rear-view camera 113, etc.), a lidar 114, a millimeter-wave radar, an ultrasonic radar, etc., and may also include a radiometer (for detecting light intensity), a rain gauge (for detecting current rainfall), etc., which will not be repeated here. Of course, the vehicle may also be equipped with at least one in-vehicle sensor (such as an in-vehicle camera, an in-vehicle microphone, an in-vehicle biosensor, an in-vehicle odor sensor, etc.) at an appropriate location to implement corresponding in-vehicle functions, and the embodiments of the present application do not limit this.

[0037] In addition, the vehicle can establish a network connection with a remote server 13 through a wireless communication module to exchange data with the server 13. For example, if the assisted driving module is deployed in the server 13, the vehicle 11 can send the environmental data collected by the aforementioned sensor to the server 13, and receive the decision-related information and vehicle control instructions returned by it. Among them, the server 13 can be a physical server containing an independent host, or it can also be a virtual server, cloud server, etc. hosted by a host cluster. In addition, the embodiment of the present application does not limit the number, type, specific interaction method with the vehicle, etc. of the server 13. As for the network 10 for interaction between the vehicle 11 and the server 13, the corresponding type of wireless network can be selected to achieve communication based on the communication method supported by the corresponding device, and the present application does not limit this.

[0038] Furthermore, the vehicle described in this application (e.g., vehicle 11) can be a pickup truck, sedan, SUV (Sport Utility Vehicle), RV, or van, among others, in terms of its functional form. It can be a fuel-powered vehicle or a new energy vehicle (e.g., a hybrid vehicle, a pure electric vehicle, a hydrogen-powered vehicle, a methanol-powered vehicle, etc.). The present application does not limit the specific form of the vehicle. Furthermore, the cabin occupant 12 can be the driver seated in the driver's seat, or at least one passenger seated elsewhere. This application also does not limit the number of occupants or their seating positions. Of course, the assisted driving function described in this application is used to assist the driver in driving the vehicle, and will not be further described.

[0039] The assisted driving module described in the embodiment of the present application can be used to make assisted driving decisions based on relevant data (such as environmental data collected by the aforementioned sensors), and its complete decision-making chain can include key steps such as perception, prediction, and regulation. For example, in the perception stage, at least one driving sensitive factor such as lane lines, traffic lights, pedestrians, etc. can be identified; in the prediction stage, high-risk factors such as pedestrian intrusion risk, vehicle collision risk, slippery road risk, etc. can be predicted based on the above-mentioned driving sensitive factors and their operating parameters; in the regulation stage, path planning can be performed, and the vehicle can be controlled to travel along the planned path (specifically, it may include controlling vehicle turning, acceleration, deceleration, etc.). This solution is used to output decision-related information corresponding to the assisted driving module to the vehicle occupants, which is used to influence its decision-making process (that is, one or more information that affects at least one of the above-mentioned key steps), so that the occupants can understand the specific decision-making process of the assisted driving module through this information.

[0040] Figure 2 20 is a flowchart of a method for presenting an assisted driving decision process according to an embodiment of the present invention. The method is applied to a vehicle and includes the following steps 202-204.

[0041] Step 202: When the assisted driving function is enabled on the vehicle, decision-making related information corresponding to the assisted driving module of the vehicle is obtained, where the decision-making related information is used to influence the decision-making process of the assisted driving module.

[0042] In one embodiment, decision-related information can be acquired in real time and continuously output throughout the entire time period after the assisted driving function is enabled, thereby presenting the entire decision-making process of the assisted driving function. Alternatively, to prevent the decision-related information output by this solution from disturbing the occupants, a function to disable the output of decision-related information can be provided to the occupants (i.e., the users of this solution), allowing them to disable the output of decision-related information provided by this solution through screen operation, gesture operation, or voice control. Obviously, in the latter case, this solution only outputs the decision-related information to the occupants when the output function is enabled, and will not be further described.

[0043] In one embodiment, the assisted driving module described herein can be built based on any neural network framework and trained in a supervised or unsupervised manner. For example, the assisted driving module can utilize a Vision-Language Model (VLM), a multimodal artificial intelligence model that combines computer vision (CV) and natural language processing (NLP) capabilities. This model can simultaneously understand and process images (or videos) and text information, establishing connections between the two to enable more complex tasks. The assisted driving module implemented using the VLM can analyze and reason from multiple dimensions based on multimodal data (e.g., environmental data of corresponding types collected by multiple types of sensors), ultimately achieving comprehensive decisions accurately and efficiently.

[0044] In related technologies, after a vehicle turns on its assisted driving function, it usually outputs information about the vehicle's surrounding environment in real time. For example, a real-time three-dimensional image of the space in which the vehicle is located is displayed on the central control screen. This image will display other vehicles, pedestrians, traffic lights and other driving-sensitive factors around the vehicle in real time in the form of a 3D model, so as to enhance the passengers' (especially the driver's) perception of the vehicle's surrounding environment and facilitate their safe driving.

[0045] In this regard, it is important to note that the decision-related information output by this solution is not the above-mentioned surrounding environment information, but rather higher-dimensional, richer information that has a certain impact on the decision-making process of the assisted driving module. For example, Figure 4 The vehicle's screen displays content. The dashed-lined environment display area 401 displays 3D models of various objects in the vehicle's surroundings, essentially identical to displays in related technologies. More importantly, the dashed-lined information display area 402 displays real-time camera footage, a description of the current safety status, and multiple alternative routes rendered in real time, among other decision-making information. This information is used to inform the occupants of the driver assistance module's decision-making process.

[0046] Step 204 : outputting the decision-related information in the cabin of the vehicle so that passengers in the cabin can perceive the decision-making process.

[0047] After acquiring the decision-making information corresponding to the assisted driving module, the assisted driving system can output this information to the occupants in the cabin, allowing them to accurately understand the decision-making process of the assisted driving module as presented by this information. This decision-making information can be output to any occupant in the cabin, for example, only to the driver to assist in safe driving; or only to rear passengers to enable them to promptly perceive the vehicle's motion trends and rhythm to avoid motion sickness; or to the co-pilot to enable the co-pilot to view the information if the driver (i.e., the primary driver) is unable to view the information and promptly alert the driver, etc. This will not be further elaborated.

[0048] In one embodiment, the decision-related information can be output within the vehicle's cabin in a variety of ways to meet different scenario requirements. For example, a display device installed within the cabin can be used to display the decision-related information. This information can be in the form of text, images, or video. The display device can be a screen, such as the instrument panel in front of the driver, the central control screen in the center of the vehicle's center console, or a screen positioned to the left behind the front row facing the rear passengers. It can also be a head-up display (HUD) device, such as a combined HUD (C-HUD), a windshield HUD (W-HUD), or an augmented reality HUD (AR-UHD). The following embodiments primarily focus on this embodiment.

[0049] For another example, an audio device installed in the cockpit can be used to play a voice message corresponding to the decision-related information. For example, the in-cockpit audio system can be used to read the decision-related information in text format, so that passengers (especially the driver) can still obtain the decision-related information even when it is difficult to view the screen or HUD.

[0050] For another example, when a network connection is established between the vehicle and the passenger's mobile terminal, the mobile terminal is called to output the decision-related information. The mobile terminal can be any electronic device, such as the passenger's mobile phone, tablet device, laptop computer, PDA (Personal Digital Assistant), wearable device (such as smart glasses, smartwatch, etc.), VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Based on the wired or wireless connection established between the mobile terminal and the vehicle, the vehicle can send the decision-related information to the device for output, such as displaying information in the form of text, images, or videos, or playing corresponding audio, etc., which will not be further described.

[0051] In one embodiment, the decision-related information corresponding to the assisted driving module may include at least one of the following: environmental data, lane data, and / or path data to be input into the assisted driving module; driving-sensitive factors related to vehicle driving identified (i.e., predicted) by the assisted driving module; the driving path formulated (i.e., planned) by the assisted driving module, and the vehicle control instructions output by the assisted driving module for the driving path. It is understood that the richer the decision-related information output, the more accurate and comprehensive it will be for passengers to understand the decision-making process of the assisted driving module. However, excessive types or amounts of information may cause interference to passengers. Therefore, the specific information to be output and the form in which each type of information is output can be comprehensively considered based on a variety of factors, such as the functional positioning of the vehicle, the identities and number of passengers in the vehicle, and the volume of human voices in the vehicle, and the embodiments of the present application are not limited in this regard.

[0052] In one embodiment, when acquiring decision-related information, the high-risk object identified by the auxiliary driving module can be determined, and at least the vehicle exterior image captured by the vehicle's external camera containing the high-risk object can be acquired. Determining the high-risk object actually means acquiring descriptive information of the high-risk object, such as the object's location, size, shape, color, type, degree of danger, and other information. When acquiring exterior images, only exterior images containing the object can be acquired based on the location information of the high-risk object (e.g., if vehicle B is overtaking vehicle A in the left lane and is a high-risk object, only the exterior image containing vehicle B captured by the left camera can be acquired) and displayed. Of course, exterior images captured by all cameras can also be acquired and displayed uniformly, which will not be elaborated on.

[0053] Accordingly, when outputting decision-related information, the image outside the vehicle can be displayed, and the high-risk objects can be marked at the corresponding positions on the image outside the vehicle. It can be understood that the above-mentioned image outside the vehicle can be captured and obtained in real time, so the process of displaying the image outside the vehicle is actually the process of playing the video outside the vehicle captured in real time by the camera. In this way, passengers can view the above-mentioned image outside the vehicle to understand the real-time environment outside the vehicle, thereby enhancing their perception of the external environment. By marking the high-risk objects, passengers can more accurately and quickly perceive the dangerous factors in the external environment and their locations, thereby helping them to provide early warnings (such as turning the steering wheel or braking, etc.) to avoid accidents.

[0054] like Figure 3 As shown, after the assisted driving function is turned on, the passenger can see the assisted driving AI+ logo information (the display effect can be seen in Figure 4410 is shown. Subsequently, based on the location information of the high-risk object, at least one corresponding target camera can be queried from the assisted driving decision data associated with the assisted driving module (i.e., the "determining the target camera number"), and then the image or video outside the vehicle captured by the target camera can be obtained and displayed. Based on the location information of the high-risk object, the location of the high-risk object can be mapped to a corresponding location in the image or video outside the vehicle and identified.

[0055] The high-risk objects can be marked in a variety of ways. For example, since the high-risk object is already included in the image or video outside the vehicle, the display parameters of the high-risk object in the image or video can be adjusted to highlight the high-risk object, or a danger mark can be displayed at the location of the high-risk object to highlight the high-risk object. The degree of highlighting of the high-risk object can be positively correlated with its degree of danger, so as to convey the danger and urgency of the object.

[0056] like Figure 5 The following table shows the corresponding system behaviors and cameras for several typical assisted driving intents. For example, when the system behavior indicates that a lateral command is about to be responded to, the corresponding cameras are the front wide-angle camera and the two side rear cameras. When the system behavior indicates that a longitudinal command is about to be responded to, the corresponding camera is the front wide-angle camera. These details are not repeated here.

[0057] like Figure 5 As shown in the flowchart below, after acquiring multiple video streams captured by multiple cameras, the perception results of high-risk objects can be obtained. Based on these results, ID tracking, target recognition, and coordinate system conversion can be performed to ultimately determine the exact location of the high-risk object in the image outside the vehicle. The corresponding high-risk object is accurately labeled at the corresponding location during the video stream rendering process.

[0058] Continuing with the previous example, let's assume that vehicle C (a truck) is overtaking vehicle A (an SUV) in the right lane. There are also multiple oncoming vehicles in the left lane, with vehicle D being the closest. Therefore, vehicles C and D are considered high-risk targets relative to vehicle A. In this case, the information display area 402 on the right side of vehicle A's central control screen displays a video 405 of vehicle A's exterior (the three small windows displaying the videos captured by the front, left, and right cameras, respectively). Vehicle C is labeled with a dangerous thermal sign 406c in the exterior video captured by the front and right cameras, respectively. Vehicle D is labeled with a dangerous thermal sign 406d in the exterior video captured by the left camera. Furthermore, because vehicle C is closer to vehicle A and poses a higher risk, sign 406d is darker and larger than sign 406c.

[0059] In addition, avoidance instructions for the aforementioned high-risk objects can be further displayed on the vehicle exterior image to instruct passengers (such as the driver of the vehicle) to avoid the aforementioned high-risk objects, thereby improving risk response efficiency. Figure 4 As shown, the screen may further display text such as "The vehicle on the right is too close, please keep to the left in the current lane to avoid" for the aforementioned vehicle C, or play corresponding voice, or display text such as "The oncoming vehicle is driving too fast and has crossed the double yellow line, please flash your lights to remind and keep to the right in the current lane to avoid" for the aforementioned vehicle D, or play corresponding voice. Of course, in view of the fact that the aforementioned assisted driving function has been turned on, the assisted driving module may also directly output avoidance instructions for the above-mentioned high-risk objects, so as to directly control the vehicle to avoid, further shorten the response path and delay, and further improve the risk response efficiency. In addition, the avoidance instruction information may also be an indicator mark on the current path, such as Figure 4 The offset path displayed in the environment display area 401 or the turning arrow displayed in the information display area are not described in detail.

[0060] In one embodiment, when acquiring decision-related information, external environment data (this data describes the vehicle's current external environment) to be input into the assisted driving module can be obtained, and the current safety status of the external environment can be assessed based on this external environment data. Alternatively, the assessment result of the current safety status output by the assisted driving module after reasoning based on the external environment data can be directly acquired. Accordingly, when displaying decision-related information, a description of the current safety status can be output, so that the occupant can understand whether the external environment is safe (or dangerous) and the degree of safety (or danger) based on this information.

[0061] Among them, the external environment data can be used to describe the external environment in which the vehicle is currently located from multiple dimensions. For example, the external environment data can include natural environment data (such as the time period of the current moment, the current weather, the current lighting, etc.), lane data (such as lane width, lane type, direction of travel, etc.) and / or path data (such as traffic congestion, distribution of obstacles on the road, etc.). Based on this, when evaluating the current safety status of the external environment based on the external environment data, a weighted calculation can be performed based on the external environment data from the multiple dimensions, and the safety level (such as excellent, good, average, poor, etc.) and / or safety score (specific safety score, such as 0 to 100 points, the larger the score, the safer the external environment) used to characterize the current safety status can be determined according to the calculation results. Figure 3 shown.

[0062] like Figure 6 As shown in (a), weighted calculations can be performed based on external environmental data such as natural environment data, lane data, and path data, and the calculation results can be mapped to any of the three safety levels: excellent, good, and general. Figure 6 As shown in (b), the current safety level can be determined based on lane and path data (e.g., "System Status: Good," as shown in the figure). The current safety level can also be determined based on natural environment data, and so on. This is not detailed here. Clearly, the safety level is positively correlated with weather conditions, lane quality, and path congestion.

[0063] In addition, when the status description information of the current safety status is a status description text, when obtaining the status description text, if the auxiliary driving module is a VLM, the status description text of the current safety status output by the auxiliary driving module can be received, that is, the current safety status can be highly summarized and the corresponding status description text can be output by leveraging the text generation capability of the VLM. Alternatively, based on the evaluation result of the current safety status, the status description text of the current safety status can be generated according to a preset description text template. In this case, the preset description text template is used as a fallback strategy to generate the status description text to ensure that the generated text is not empty.

[0064] Similar to the safety level of the aforementioned current safety state, when obtaining decision-related information, the external environment data to be input into the auxiliary driving module (this data is used to describe the external environment in which the vehicle is currently located) can be obtained, and the current driving scene of the vehicle can be identified based on the external environment data. Alternatively, the recognition result of the current driving scene output by the auxiliary driving module after reasoning based on the external environment data can be directly obtained. Accordingly, when displaying decision-related information, the scene description information of the current driving scene can be output. Figure 3 As shown, the scene description text can be generated according to the preset scene corpus (i.e., the preset description text template).

[0065] Among them, when the scene description information of the current driving scene is a scene description text, when obtaining the scene description text, when the auxiliary driving module is a VLM, the scene description text of the current driving scene output by the auxiliary driving module can be received. Alternatively, the scene description text of the current driving scene can be generated according to a preset description text template based on the recognition result of the current driving scene, which will not be repeated. Exemplarily, the above-mentioned scene description information can be used to describe static (suspended) obstacles, other vehicles in the traffic flow, VRU (Vulnerable Road User, vulnerable road users such as pedestrians, cyclists and motorcyclists. These road users are often vulnerable to injury in the event of a traffic accident because they do not have the protection of a metal shell like a car), etc., which will not be repeated.

[0066] like Figure 4 As shown, the scene description information of the current driving scene displayed in the information display area 402 is "Backlight scene, please drive carefully", and the status description text of the current safety status displayed is "The current driving environment safety level is good: the road ahead is open, the speed of the vehicle on the right is low, and there are few target vehicles."

[0067] In one embodiment, when obtaining decision-related information, the assisted driving module may output multiple alternative routes and a path score for each alternative route (in this case, the assisted driving module performs a comprehensive score for each alternative route planned by the assisted driving module). Alternatively, the assisted driving module outputs multiple alternative routes and calculates a path score for each alternative route (in this case, the assisted driving system performs a comprehensive score for each alternative route planned by the assisted driving module). Based on this, when outputting decision-related information, the multiple alternative routes and the path score for each alternative route may be displayed in descending order of path score. Since the vehicle has activated the assisted driving function, its current speed may not be zero, or other driving-sensitive factors around it may change, the alternative routes may vary at different times. To address this issue, the assisted driving system may render each alternative route planned by the assisted driving module in real time and display the path score for each alternative route in sequence, so that the occupant can accurately understand the relative merits of each alternative route by viewing the score.

[0068] When calculating the path score for each alternative path, the path score can be comprehensively calculated based on at least two dimensions: driving safety (e.g., which can be characterized by the magnitude of the collision probability), compliance with regulations (whether red lights are run, whether the road is crossed), comfort (which can be characterized by the magnitude of lateral, longitudinal, and / or vertical acceleration), driving efficiency (which can be characterized by the expected time to reach the destination), and anthropomorphism (i.e., whether the driving behavior conforms to human driving habits). For example, for any alternative path, the path's safety score, regulatory score, comfort score, efficiency score, and anthropomorphism score can be calculated, and the path score for the path can be comprehensively calculated (e.g., weighted) based on these scores.

[0069] like Figure 7 As shown, each candidate path can first be safety-verified to identify safe alternative paths with no safety risk or safety risk below a threshold. Each safe alternative path is then comprehensively scored based on the five dimensions mentioned above to obtain a corresponding path score. The safe alternative paths are then sorted by their path scores. Finally, the paths are rendered and displayed in real time according to the sorting results, such as the optimal path with a score of 98 and alternative safe path 1 with a score of 90.

[0070] like Figure 4 As shown, multiple alternative routes 408 and the route score of each alternative route are displayed in order below the information display area 402. In addition, the scores of the highest-scoring alternative routes in various dimensions are also displayed. For example, the safety score, comfort score, and efficiency score of the optimal route with a route score of 98 are displayed in the form of a radar chart. This will not be repeated here.

[0071] In one embodiment, given that vehicle sensors may contain errors or even false detections, the assisted driving module may make decisions that are inappropriate for the current driving environment. For example, due to radar blind spots, the highest-scoring alternative path may not be the optimal path, and even driving along that path could result in a traffic accident. Given that occupants (particularly the driver) may have superior observation capabilities and perspectives to sensors, there may be high-risk objects that the occupants observe but the assisted driving module fails to perceive. To address this, when the assisted driving function is enabled, occupants (e.g., the driver) may be allowed to intervene in the decision-making process, such as by allowing the user to select a path from among the multiple alternative paths based on their own judgment. For example, in response to the driver selecting any of the multiple alternative paths, the vehicle may be controlled to travel along that alternative path.

[0072] In one embodiment, when obtaining decision-related information, the vehicle control command output by the auxiliary driving module can be obtained, such as Figure 3The lateral instructions shown (such as lane change instructions, avoidance instructions, detour instructions, left / right turn instructions, etc.), longitudinal deceleration instructions (such as follow instructions, follow stop instructions, yellow flash deceleration instructions, red light brake stop instructions, CUTIN intrusion deceleration instructions, etc.) or longitudinal acceleration instructions (such as follow stop start instructions, green light start instructions, etc.). In this regard, when outputting decision-related information, the instruction description information of the vehicle control instructions can be output, such as when the auxiliary driving module issues a red light brake stop instruction, a voice such as "red light brake" is broadcast; when the auxiliary driving module issues a green light start instruction, the edge of the control screen flashes three green spots and / or plays a voice such as "green light start", which will not be repeated. In this way, the occupants can be fully informed of the vehicle control instructions issued by the auxiliary driving module, so that the occupants can know the decision results of the auxiliary driving module.

[0073] In addition, for the vehicle control instructions output by the assisted driving module, before executing the instruction or outputting the instruction description information of the instruction, the instruction can also be verified (such as keyword verification, etc.) to ensure that the instructions output by the assisted driving module are consistent with the current driving environment, thereby ensuring "doing it right and saying it right", that is, the vehicle control instructions that the assisted driving module can output are consistent with the current driving environment and the assisted driving system can correctly output the corresponding instruction description information.

[0074] It can be seen from the above embodiments that, when the vehicle has turned on the assisted driving function, this solution obtains decision-related information that affects the decision-making process of the assisted driving module of the vehicle, and outputs the information in the cabin so that the occupants in the cabin can perceive the decision-making process of the assisted driving module.

[0075] It is understood that this decision-related information influences the decision-making process of the assisted driving module, namely, affects the module's completion of at least one key step in the decision-making chain (such as perception, prediction, and regulation). Therefore, by outputting this decision-related information to the occupants, not only is the interpretability of the decision-making process of the assisted driving module improved, but the occupants can also accurately and comprehensively understand the decision-making process presented based on this information. This helps to improve the transparency of the assisted driving module's decisions and reduce the difficulty for occupants to understand, thereby helping to increase occupants' trust in and willingness to use the assisted driving function.

[0076] Figure 8 This is a schematic structural diagram of a vehicle according to an embodiment of the present invention. Figure 8At the hardware level, the vehicle includes a processor 801, a network interface 802, a memory 803, a non-volatile memory 804, and an internal bus 805. Of course, it may also include hardware required for other services. One or more embodiments of the present invention can be implemented based on software, such as the processor 801 reading the corresponding computer program from the non-volatile memory 804 into the memory 803 and then running it. Of course, in addition to software implementation, one or more embodiments of the present invention do not exclude other implementation methods, such as logic devices or a combination of software and hardware. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0077] Figure 9 A block diagram of a device for presenting an assisted driving decision process is shown in an embodiment of the present invention. Figure 8 , the device can be used for Figure 8 In the vehicle shown, the technical solution of the present invention is implemented. The device includes:

[0078] An information acquisition unit 901 is configured to acquire decision-making related information corresponding to an assisted driving module of the vehicle when the assisted driving function is enabled, wherein the decision-making related information is used to influence the decision-making process of the assisted driving module;

[0079] The information output unit 902 is configured to output the decision-related information in the cabin of the vehicle so that passengers in the cabin can perceive the decision-making process.

[0080] Optionally, the information output unit 902 is specifically configured to perform at least one of the following:

[0081] Invoking a display device installed in the cockpit to display the decision-related information;

[0082] Invoking an audio device installed in the cockpit to play a voice corresponding to the decision-related information;

[0083] In a case where a network connection is established between the vehicle and the occupant's mobile terminal, the mobile terminal is called to output the decision-related information.

[0084] Optional,

[0085] The information acquisition unit 901 is specifically configured to: determine the high-risk object identified by the auxiliary driving module, and at least obtain an external image of the vehicle containing the high-risk object captured by an external camera of the vehicle;

[0086] The information output unit 902 is specifically configured to display the vehicle exterior image and mark the high-risk object at a corresponding position on the vehicle exterior image.

[0087] Optionally, the information output unit 902 is specifically configured to:

[0088] Adjust the display parameters of the high-risk object to highlight the high-risk object, or display a danger mark at the location of the high-risk object to highlight the high-risk object; wherein the degree of highlighting of the high-risk object is positively correlated with its degree of danger.

[0089] Optionally, the system further includes an avoidance instruction unit 903, configured to:

[0090] Avoidance instruction information for the high-risk object is displayed on the vehicle exterior image.

[0091] Optional,

[0092] The information acquisition unit 901 is specifically used to:

[0093] Obtaining external environment data to be input into the auxiliary driving module, the data being used to describe the external environment in which the vehicle is currently located, and evaluating the current safety status of the external environment and / or identifying the current driving scene of the vehicle based on the external environment data; and / or,

[0094] Obtaining an evaluation result for the current safety status and / or a recognition result for the current driving scene output by the auxiliary driving module;

[0095] The information output unit 902 is specifically configured to output status description information of the current safety status and / or scene description information of the current driving scene.

[0096] Optionally, the external environment data is used to describe the external environment in which the vehicle is currently located from multiple dimensions, and the information acquisition unit 901 is specifically used to:

[0097] A weighted calculation is performed from the multiple dimensions based on the external environment data, and a security level and / or security score for characterizing the current security status is determined according to the calculation result.

[0098] Optionally, the status description information of the current security status is a status description text, and the information acquisition unit 901 is specifically configured to:

[0099] The state description information of the current safety state is a state description text, and obtaining the state description text includes: when the auxiliary driving module is a visual language model (VLM), receiving the state description text of the current safety state output by the auxiliary driving module; or, based on the evaluation result of the current safety state, generating the state description text of the current safety state according to a preset description text template; and / or,

[0100] The scene description information of the current driving scene is a scene description text, and obtaining the scene description text includes: when the assisted driving module is a visual language model VLM, receiving the scene description text of the current driving scene output by the assisted driving module; or, based on the recognition result of the current driving scene, generating the scene description text of the current driving scene according to a preset description text template.

[0101] Optional,

[0102] The information acquisition unit 901 is specifically configured to: acquire multiple alternative paths output by the assisted driving module and a path score of each alternative path; or acquire multiple alternative paths output by the assisted driving module and calculate a path score of each alternative path;

[0103] The information output unit 902 is specifically configured to display the multiple candidate paths and the path score of each candidate path in order from high to low path scores.

[0104] Optionally, the information acquisition unit 901 is specifically configured to:

[0105] The path score of each alternative path is comprehensively calculated based on at least two dimensions of driving safety, compliance with regulations, comfort, driving efficiency, and degree of anthropomorphism.

[0106] Optionally, a vehicle control unit 904 is further included, which is used to:

[0107] In response to the driver of the vehicle selecting any one of the multiple alternative routes, the vehicle is controlled to travel along the any one of the alternative routes.

[0108] Optional,

[0109] The information acquisition unit 901 is specifically used to: acquire the vehicle control command output by the auxiliary driving module;

[0110] The information output unit 902 is specifically configured to output instruction description information of the vehicle control instruction.

[0111] Accordingly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for presenting the assisted driving decision-making process as described in any of the above embodiments.

[0112] Accordingly, this specification also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for presenting the assisted driving decision-making process as described in any of the above embodiments.

[0113] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0114] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0115] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0116] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

Claims

1. A method for presenting an assisted driving decision process, characterized in that: include: When the assisted driving function is enabled on the vehicle, obtaining decision-related information corresponding to the assisted driving module of the vehicle, wherein the decision-related information is used to influence the decision-making process of the assisted driving module; The obtaining of decision-related information corresponding to the assisted driving module of the vehicle includes: obtaining external environment data to be input into the assisted driving module, the data being used to describe the external environment in which the vehicle is currently located, and evaluating a current safety state of the external environment and / or identifying a current driving scene of the vehicle based on the external environment data; and / or obtaining an evaluation result of the current safety state and / or an identification result of the current driving scene output by the assisted driving module; Outputting the decision-related information in the cabin of the vehicle so that passengers in the cabin can perceive the decision-making process; outputting the decision-related information includes: outputting state description information of the current safety state and / or scene description information of the current driving scene; Wherein, when the status description information of the current safety status is a status description text, obtaining the status description text includes: when the auxiliary driving module is a visual language model VLM, receiving the status description text of the current safety status output by the auxiliary driving module; or, based on the evaluation result of the current safety status, generating the status description text of the current safety status according to a preset description text template; and / or, when the scene description information of the current driving scene is a scene description text, obtaining the scene description text includes: when the auxiliary driving module is a visual language model VLM, receiving the scene description text of the current driving scene output by the auxiliary driving module; or, based on the recognition result of the current driving scene, generating the scene description text of the current driving scene according to a preset description text template.

2. The method according to claim 1, characterized in that Outputting the decision-related information in the vehicle cabin includes at least one of the following: Invoking a display device installed in the cockpit to display the decision-related information; Invoking an audio device installed in the cockpit to play a voice corresponding to the decision-related information; In a case where a network connection is established between the vehicle and the occupant's mobile terminal, the mobile terminal is called to output the decision-related information.

3. The method according to claim 1, characterized in that The obtaining of decision-related information corresponding to the auxiliary driving module of the vehicle includes: determining a high-risk object identified by the auxiliary driving module, and obtaining at least an image outside the vehicle including the high-risk object captured by an external camera of the vehicle; The outputting of the decision-related information includes: displaying the image outside the vehicle, and marking the high-risk object at a corresponding position on the image outside the vehicle.

4. The method according to claim 3, characterized in that The marking of the high-risk object at the corresponding position on the external image of the vehicle includes: Adjust the display parameters of the high-risk object to highlight the high-risk object, or display a danger mark at the location of the high-risk object to highlight the high-risk object; wherein the degree of highlighting of the high-risk object is positively correlated with its degree of danger.

5. The method according to claim 3 or 4, characterized in that Also includes: The avoidance instruction information for the high-risk object is displayed on the vehicle exterior image.

6. The method according to claim 1, characterized in that The external environment data is used to describe the external environment in which the vehicle is currently located from multiple dimensions, and the evaluating the current safety status of the external environment based on the external environment data includes: A weighted calculation is performed from the multiple dimensions based on the external environment data, and a security level and / or security score for characterizing the current security status is determined according to the calculation result.

7. The method according to claim 1, characterized in that The obtaining of decision-related information corresponding to the assisted driving module of the vehicle includes: obtaining multiple alternative paths output by the assisted driving module and a path score for each alternative path; or obtaining multiple alternative paths output by the assisted driving module and calculating a path score for each alternative path; Outputting the decision-related information includes: displaying the multiple candidate paths and the path score of each candidate path in order from high to low path scores.

8. The method according to claim 7, characterized in that Calculating the path score of each candidate path includes: The path score of each alternative path is comprehensively calculated based on at least two dimensions of driving safety, compliance with regulations, comfort, driving efficiency, and degree of anthropomorphism.

9. The method according to claim 7, characterized in that Also includes: In response to the driver of the vehicle selecting any one of the multiple alternative routes, the vehicle is controlled to travel along the any one of the alternative routes.

10. The method according to claim 1, characterized in that The obtaining of decision-related information corresponding to the auxiliary driving module of the vehicle includes: obtaining a vehicle control instruction output by the auxiliary driving module; The outputting of the decision-related information includes: outputting instruction description information of the vehicle control instruction.

11. A device for presenting an assisted driving decision process, characterized in that: include: an information acquisition unit configured to, when the assisted driving function is enabled on the vehicle, acquire decision-related information corresponding to the assisted driving module of the vehicle, the decision-related information being used to influence the decision-making process of the assisted driving module; the information acquisition unit being specifically configured to: acquire external environment data to be input into the assisted driving module, the data being used to describe the external environment currently located by the vehicle, and, based on the external environment data, assess the current safety status of the external environment and / or identify the current driving scenario of the vehicle; and / or, obtaining an evaluation result for the current safety status and / or a recognition result for the current driving scene output by the auxiliary driving module; an information output unit, configured to output the decision-related information in the cabin of the vehicle so that passengers in the cabin can perceive the decision-making process; the information output unit is specifically configured to: output state description information of the current safety state and / or scene description information of the current driving scene; Wherein, when the state description information of the current safety state is a state description text, the information output unit is specifically configured to receive the state description text of the current safety state output by the auxiliary driving module when the auxiliary driving module is a visual language model VLM; or, based on the evaluation result of the current safety state, generate the state description text of the current safety state according to a preset description text template; and / or, In the case where the scene description information of the current driving scene is a scene description text, the information output unit is specifically used to receive the scene description text of the current driving scene output by the assisted driving module when the assisted driving module is a visual language model VLM; or, based on the recognition result of the current driving scene, generate the scene description text of the current driving scene according to a preset description text template.

12. A vehicle comprising: A processor, a memory for storing instructions executable by the processor, and an assisted driving module for providing assisted driving functions; The processor implements the method according to any one of claims 1 to 10 by running the executable instructions.

13. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images