Vehicle auxiliary driving method and device and electronic equipment

Abstract

The invention provides a vehicle auxiliary driving method and device and electronic equipment. The vehicle auxiliary driving method comprises the steps of obtaining a real-time image of a current working scene of a vehicle, wherein the real-time image comprises a preset area of a to-be-displayed target object outside and / or inside the vehicle; analyzing the preset area, determining a target object corresponding to the preset area, and superposing a virtual model of the target object on the real-time image to generate an augmented reality image; acquiring real-time state information of the target object; generating prompt information of the target object according to the current working scene and the real-time state information of the target object; and superposing the prompt information on the augmented reality image to obtain a superposed image, and displaying the superposed image through the augmented reality wearable device. According to the vehicle auxiliary driving method, the real-time state information of the target object can be displayed, a user can sense the real-time change situation inside and outside the vehicle, the user does not need to distract attention to see third-party auxiliary equipment, and the driving safety is improved.

Description

Technical Field

[0001] The present application relates to the field of augmented reality technology, and in particular to a vehicle assisted driving method, device and electronic equipment. Background Art

[0002] With the development of vehicle technology, vehicles have become more intelligent and convenient, and there are more and more vehicle driving assistance devices, such as HUD (Head Up Display), electronic rearview mirrors, etc. These devices can improve the intelligence of the car and provide greater convenience for users.

[0003] In the related art, when a user is driving, due to driving safety requirements, the user needs to pay more attention, and the user can occasionally look at the instrument screen, car HUD head-up display or central control screen and other third-party auxiliary devices. However, the information displayed by the third-party auxiliary devices is fixed and limited, and cannot display the real-time changes inside and outside the car during the user's driving. Summary of the invention

[0004] The present application provides a vehicle assisted driving method, device and electronic equipment, which can display real-time changes inside and outside the vehicle.

[0005] One aspect of the present application provides a vehicle assisted driving method, which is applied to a vehicle, wherein the vehicle includes an augmented reality wearable device for a user to wear, and the vehicle assisted driving method includes:

[0006] Acquire a real-time image of a current working scene of the vehicle, wherein the real-time image includes a preset area of ​​a target object to be displayed outside and / or inside the vehicle;

[0007] Analyzing the preset area, determining a target object corresponding to the preset area, and superimposing a virtual model of the target object on the real-time image to generate an augmented reality image;

[0008] Acquire real-time status information of the target object;

[0009] Generate prompt information of the target object according to the current working scene and the real-time status information of the target object;

[0010] The prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

[0011] The vehicle assisted driving method provided in the present application realizes the mutual complementation between real data and virtual images through augmented reality wearable devices, and displays the real-time status information of target objects inside and / or outside the vehicle, so that the user can perceive the real-time changes inside and outside the vehicle. In addition, the augmented reality wearable device moves with the user's line of sight, and the user does not need to distract his attention to look at third-party auxiliary devices, thereby improving driving safety.

[0012] Furthermore, the vehicle assisted driving method further includes:

[0013] When the movement speed in the real-time terminal information of the preset terminal of the vehicle is greater than or equal to a speed threshold, determining that the current working scene of the vehicle is a driving scene;

[0014] When the current working scene of the vehicle is the driving scene, obtaining line of sight movement data obtained by dynamically tracking the line of sight of the user by the augmented reality wearable device;

[0015] Based on the eye movement data, determining whether the user's sight is within a safe driving field of view;

[0016] When the duration during which the user's line of sight is not within the driving safety field of view is greater than or equal to the preset line of sight separation time, an automatic driving instruction is output to control the vehicle to enter an automatic driving state.

[0017] Further, the target object includes a driving safety component; the driving safety component includes one or more of a driving unit, a braking unit and a steering unit;

[0018] The generating prompt information of the target object according to the current working scene and the real-time status information of the target object includes:

[0019] determining whether the driving safety component is in an abnormal state according to the state information of the driving safety component in the real-time state information of the target object;

[0020] When the current working scene of the vehicle is a driving scene and the driving safety component is in the abnormal state, generating driving safety prompt information corresponding to the driving safety component;

[0021] The step of superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device includes:

[0022] The driving safety prompt information corresponding to the driving safety component is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

[0023] Further, the target object includes a target object outside the vehicle;

[0024] The generating prompt information of the target object according to the current working scene and the real-time status information of the target object includes:

[0025] Determining whether the target object outside the vehicle is an obstacle according to the target object information outside the vehicle in the real-time status information of the target object;

[0026] When the current working scene of the vehicle is a driving scene and the external target object is the obstacle, generating obstacle prompt information;

[0027] The step of superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device includes:

[0028] The obstacle prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

[0029] Further, determining whether the external target object is an obstacle according to the external target object information in the real-time status information of the target object includes:

[0030] Generate a motion path of the external target according to the external target position, the external target size, the external target movement speed and the external target movement direction in the external target information;

[0031] In a case where the current working scene of the vehicle is the driving scene, determining whether the motion path of the external target object and the driving path of the vehicle at least partially overlap;

[0032] When the movement path of the external object and the driving path of the vehicle at least partially overlap and the external object is located outside the user visible area, the external object is determined to be an obstacle.

[0033] Further, the target object includes a plurality of target objects; the augmented reality wearable device includes a display module; superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device, comprises: determining, according to the current working scene and the plurality of target objects, display priorities corresponding to the plurality of target objects for displaying the prompt information; superimposing the prompt information on the augmented reality image in sequence according to the display priority levels to obtain a superimposed image, and displaying the superimposed image through the display module;

[0034] or,

[0035] The target object includes multiple target objects; the augmented reality wearable device includes a display module, and the display module includes multiple display partitions; superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device, including: according to the current working scene and the multiple target objects, determining the display partitions corresponding to the multiple target objects for displaying the prompt information, superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the corresponding display partitions.

[0036] Furthermore, the vehicle assisted driving method further includes:

[0037] Obtain the user's vehicle driving behavior;

[0038] In the case where the vehicle driving behavior affects the driving safety of the vehicle, generating correction prompt information for the vehicle driving behavior;

[0039] The correction prompt information is superimposed on the superimposed image to obtain a re-superimposed image, and the re-superimposed image is displayed through the augmented reality wearable device.

[0040] Furthermore, obtaining the user's vehicle driving behavior includes:

[0041] Determine the vehicle driving behavior to be performed by the user in the future based on the user's driving habits and the vehicle's real-time driving data;

[0042] In the case where the vehicle driving behavior affects the driving safety of the vehicle, generating correction prompt information of the vehicle driving behavior includes:

[0043] In the case where the driving behavior of the vehicle to be operated affects the driving safety of the vehicle, generating correction prompt information of the driving behavior of the vehicle to be operated in advance before the future time;

[0044] The step of superimposing the correction prompt information on the superimposed image to obtain a re-superimposed image, and displaying the re-superimposed image through the augmented reality wearable device includes:

[0045] Before the future moment, the correction prompt information is superimposed on the superimposed image in advance to obtain a re-superimposed image, and the re-superimposed image is displayed through the augmented reality wearable device.

[0046] Furthermore, before determining the vehicle driving behavior to be operated by the user in the future according to the user's driving habits and the real-time driving data of the vehicle, the method further includes:

[0047] In the case where the user drives the vehicle for the first time, obtaining vehicle information and user information;

[0048] According to the vehicle information and the user information, obtaining desensitized driving habits of other users matching the vehicle and the user as the user driving habits;

[0049] The vehicle information includes the vehicle model.

[0050] Further, in the case where the vehicle driving behavior affects the driving safety of the vehicle, generating correction prompt information of the vehicle driving behavior includes: obtaining the real-time distance between the vehicle and the vehicle in front of the vehicle; in the case where the real-time distance is less than the overtaking safety distance and the overtaking operation behavior in the vehicle driving behavior to be operated causes the risk of collision between the vehicle and the vehicle in front, generating overtaking correction prompt information;

[0051] and / or

[0052] The method of generating correction prompt information for the vehicle driving behavior when the vehicle driving behavior affects the driving safety of the vehicle includes: obtaining a road curve in the driving road condition information of the vehicle; and generating turning correction prompt information when the steering wheel operation behavior in the vehicle driving behavior to be operated causes a risk of collision of the vehicle on the road curve.

[0053] Furthermore, the vehicle assisted driving method further includes:

[0054] Detect the real-time light intensity of the ambient light;

[0055] When the real-time light intensity is greater than or equal to a light intensity threshold, the transmittance of the lens assembly in the augmented reality wearable device is adjusted.

[0056] Another aspect of the present application provides a vehicle assisted driving device, which is applied to an augmented reality wearable device worn by a user, and the vehicle assisted driving device includes:

[0057] An acquisition unit, used to acquire a real-time image of a current working scene of the vehicle, the real-time image including a preset area to be displayed virtually, and to acquire real-time status information of a target object;

[0058] An image generation unit is used to parse the preset area, determine the target object, and superimpose a virtual model of the target object on the real-time image to generate an augmented reality image, and generate prompt information of the target object according to the current working scene and the real-time status information of the target object, superimpose the prompt information on the augmented reality image to obtain a superimposed image, and display the superimposed image through the augmented reality wearable device.

[0059] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned vehicle assisted driving methods when executing the program. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0061] Figure 1 The figure is a flow chart of an embodiment of the vehicle assisted driving method of the present application;

[0062] Figure 2 Shown Figure 1 The flowchart of step S400 of the vehicle assisted driving method is shown in FIG.

[0063] Figure 3 Shown Figure 1 The flowchart of step S400 of another embodiment of the vehicle assisted driving method shown;

[0064] Figure 4 Shown Figure 3 The flowchart of step S430 of the vehicle assisted driving method is shown in FIG.

[0065] Figure 5 This is a schematic structural diagram of an embodiment of a vehicle driving assistance device of the present application;

[0066] Figure 6 for Figure 5 The structure diagram of the electronic device in which the vehicle auxiliary driving device is located is shown. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0068] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. Unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which this application belongs. The words "first", "second" and similar words used in the specification and claims of this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate a quantitative limit, but indicate that there is at least one. "Multiple" or "several" means at least two. Unless otherwise specified, words such as "front", "rear", "lower" and / or "upper" are only for the convenience of explanation and are not limited to a position or a spatial orientation. Words such as "include" or "comprise" mean that the elements or objects appearing in front of "include" or "comprise" include the elements or objects listed after "include" or "comprise" and their equivalents, and do not exclude other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, and can include electrical connections, whether direct or indirect.

[0069] The singular forms "a", "said" and "the" used in this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0070] The vehicle assisted driving method, device and electronic device of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0071] Figure 1 The flowchart of an embodiment of the vehicle assisted driving method of the present application is shown. The vehicle assisted driving method of the embodiment of the present application is applied to a vehicle, and the vehicle includes an augmented reality wearable device for a user to wear. The vehicle assisted driving method of the embodiment of the present application includes steps S100 to S500:

[0072] In step S100, a real-time image of a current working scene of a vehicle is acquired, where the real-time image includes a preset area of ​​a target object to be displayed outside and / or inside the vehicle.

[0073] In step S200, the preset area is analyzed, the target object corresponding to the preset area is determined, and a virtual model of the target object is superimposed on the real-time image to generate an augmented reality image.

[0074] In step S300, real-time status information of the target object is obtained.

[0075] In step S400, prompt information of the target object is generated according to the current working scene and the real-time status information of the target object.

[0076] In step S500, the prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

[0077] The vehicle assisted driving method of the embodiment of the present application realizes the mutual complementation between real data and virtual images through augmented reality wearable devices, and displays the real-time status information of target objects inside and / or outside the vehicle, so that the user can perceive the real-time changes inside and outside the vehicle. In addition, the augmented reality wearable device moves with the user's line of sight, and the user does not need to distract his attention to look at third-party auxiliary devices, thereby improving driving safety.

[0078] The above-mentioned augmented reality wearable device may include AR (Augmented Reality) glasses. AR glasses may include an eye tracking module to track the eye movement trajectory of the user when wearing AR glasses, so that the AR glasses display an augmented reality image that matches the user's line of sight. When the user's line of sight moves to an object in a preset area, it can be determined whether the object is a target object based on the preset range of the object. The target object may include a target object in the vehicle, a vehicle body, and a target object outside the vehicle. Among them, the vehicle body may include vehicle parts such as doors and wheels. Target objects outside the vehicle may include other vehicles and personnel, obstacles, etc. The real-time status information of the target object may include the position of the target object and the current state of the target object. The current state of the target object may include a normal state and an abnormal state. Taking the target object as a wheel as an example, the current state of the wheel can be determined based on the tire pressure of the tire. When the tire pressure is within the normal pressure range, the current state of the wheel is a normal state; when the tire pressure is not within the normal pressure range, the current state of the wheel is an abnormal state.

[0079] Optionally, the prompt information may include text content, may include non-text content such as images, expressions, etc., or may include a combination of text content and non-text content.

[0080] The above-mentioned vehicles include but are not limited to sedans, SUVs, off-road vehicles, pickup trucks or other power-driven non-track-borne vehicles. The vehicles may be new energy vehicles, including pure electric vehicles, hybrid vehicles, plug-in hybrid vehicles and other vehicles that use on-board power batteries as the main power source or one of the power sources, or they may be traditional fuel vehicles. The vehicle may include cameras and sensors installed inside and / or outside the vehicle to obtain real-time images of the current working scene of the vehicle.

[0081] In step S200, the geometry and texture of the virtual model may be generated based on the target object according to a predetermined rule set, and the virtual model of the target object may be geometrically constructed. The predetermined rule set may include rules assigned to one or more of the following: object color, background color, texture, line style, line thickness, and spatial arrangement.

[0082] In some embodiments, the vehicle assisted driving method of the present application may further include the following steps:

[0083] When the movement speed in the real-time terminal information of the preset terminal of the vehicle is greater than or equal to the speed threshold, it is determined that the current working scene of the vehicle is a driving scene.

[0084] When the current working scene of the vehicle is a driving scene, the line of sight movement data obtained by dynamically tracking the line of sight of the user by the augmented reality wearable device is obtained.

[0085] Based on the gaze movement data, determine whether the user's gaze is within the safe driving field of view.

[0086] When the duration when the user's line of sight is not within the driving safety field of view is greater than or equal to the preset line of sight separation time, an automatic driving instruction is output to control the vehicle to enter the automatic driving state.

[0087] Through the above method, during driving, when the user's line of sight leaves the front of the vehicle within a certain range, the user's attention is diverted to other places, and the vehicle can enter the automatic driving state, and the vehicle can be controlled in a short time. In this way, even if the user's attention is diverted, the vehicle will not lose control while driving, reducing the possibility of vehicle accidents. The above duration can be in seconds, and this application does not limit it.

[0088] Optionally, the preset terminal may include a motion speed sensor disposed on the vehicle body, which can obtain the real-time driving speed of the vehicle. The above speed threshold is a standard for determining whether the vehicle is moving, which can be 0 km / h, or a speed greater than 0 km / h, such as 10 km / h. When the motion speed is less than the speed threshold, it is determined that the current working scene of the vehicle is a parking scene, that is, the vehicle is in a stationary state.

[0089] Figure 2 Shown Figure 1 The process diagram of step S400 of the vehicle assisted driving method shown in the figure is an embodiment. In some embodiments, the target object includes a driving safety component. The driving safety component includes one or more of a drive unit, a brake unit, and a steering unit. Step S400 of the vehicle assisted driving method of the present application may include steps S410 to S420:

[0090] In step S410, it is determined whether the driving safety component is in an abnormal state according to the state information of the driving safety component in the real-time state information of the target object.

[0091] In step S420, when the current working scene of the vehicle is a driving scene and the driving safety component is in an abnormal state, driving safety prompt information corresponding to the driving safety component is generated.

[0092] In this embodiment, step S500 of the vehicle assisted driving method of the present application may include:

[0093] The driving safety prompt information corresponding to the driving safety component is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

[0094] The drive unit may include an engine and wheels, the brake unit may include a brake assembly, and the steering unit may include a steering wheel and a steering shaft. It is understandable that the driving safety component has a high degree of influence on driving safety. Therefore, when the vehicle is driving, if the above-mentioned driving safety component is abnormal, it will seriously affect driving safety. The vehicle assisted driving method of the embodiment of the present application can display relevant prompt information according to the status of the driving safety component to remind the user of factors that may threaten driving safety, so that the user can eliminate driving safety risk factors in a timely manner.

[0095] Figure 3 Shown Figure 1 The process diagram of another embodiment of step S400 of the vehicle assisted driving method shown in the figure. In some embodiments, the target object includes an object outside the vehicle. Step S400 of the vehicle assisted driving method of the present application may include steps S430 to S440:

[0096] In step S430, it is determined whether the target object outside the vehicle is an obstacle according to the target object information outside the vehicle in the real-time status information of the target object.

[0097] In step S440, when the current working scene of the vehicle is a driving scene and the target object outside the vehicle is an obstacle, obstacle prompt information is generated.

[0098] In this embodiment, step S500 of the vehicle assisted driving method of the present application may include:

[0099] The obstacle prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

[0100] It should be noted that the above-mentioned external target object refers to a target object other than the vehicle, which may be another vehicle other than the vehicle, or a person other than the vehicle, or other objects other than the vehicle.

[0101] Through the above method, obstacles in the blind spot that the user cannot see can be displayed through the augmented reality wearable device to expand the user's field of vision. The above blind spot can include at least one of the front blind spot, the side blind spot, the bottom blind spot of the vehicle and the parking blind spot.

[0102] Figure 4 Shown Figure 3 FIG. 1 is a flow chart of an embodiment of the vehicle assisted driving method step S430. Figure 4 As shown, step S430 of the vehicle assisted driving method of the present application may include steps S431 to S433:

[0103] In step S431, a movement path of the external target is generated according to the external target position, the external target size, the external target movement speed and the external target movement direction in the external target information.

[0104] In step S432, when the current working scene of the vehicle is a driving scene, it is determined whether the motion path of the external target object and the driving path of the vehicle at least partially overlap.

[0105] In step S433, when the movement path of the external object and the driving path of the vehicle at least partially overlap and the external object is located outside the user's visible area, the external object is determined to be an obstacle.

[0106] Through the above method, it is possible to more accurately determine whether the target object outside the vehicle is an obstacle under the current driving state of the vehicle.

[0107] Based on this embodiment, whether the external object is an obstacle can be determined according to the height of the vehicle chassis. If the height of the external object is less than the height of the vehicle chassis, it is determined that the external object is not an obstacle; otherwise, it is determined that the external object is an obstacle.

[0108] In some other embodiments, when a vehicle is following too closely behind the vehicle, the user does not need to look at the rearview mirror from the driving perspective, and the relevant prompt information of the rear of the vehicle can be displayed in the augmented reality wearable device. When the user forgets to lock the car door, before the vehicle starts, the user does not need to look at the vehicle screen from the driving perspective, and the relevant prompt information of the door unlocking can be displayed in the augmented reality wearable device.

[0109] In some embodiments, the target object includes a plurality of target objects. The augmented reality wearable device includes a display module.

[0110] In this embodiment, step S500 of the vehicle assisted driving method of the present application may include:

[0111] According to the current working scene and the multiple target objects, the display priority levels for displaying prompt information corresponding to the multiple target objects are determined.

[0112] According to the display priority level, the prompt information is sequentially superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the display module.

[0113] It should be noted that the display priority level indicates the display priority of the prompt information in the current working scene of the vehicle. When the vehicle is currently in the driving scene, the prompt information of the driving safety component and the obstacle prompt information have a higher display priority and are displayed first on the augmented reality wearable device.

[0114] In this embodiment, according to the display priority of the prompt information, the prompt information with a high priority level can be displayed on the augmented reality wearable device in sequence from the prompt information with a low priority level, so as to facilitate the user to identify more important prompt information from multiple prompt information.

[0115] Furthermore, other parameters such as the dwell time and scrolling speed of the prompt information may also be determined according to the display priority level.

[0116] In some other embodiments, the target object includes multiple target objects. The augmented reality wearable device includes a display module, and the display module includes multiple display partitions.

[0117] In this embodiment, step S500 of the vehicle assisted driving method of the present application may include:

[0118] According to the current working scene and multiple target objects, the corresponding display partitions for displaying prompt information of the multiple target objects are determined, and the prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the corresponding display partitions.

[0119] In this embodiment, the fonts and sizes of the positions and / or areas and / or corresponding text contents of multiple display partitions may be inconsistent. In the current working scenario of the vehicle, the position of the display partition corresponding to the target object that has a greater impact on the vehicle may be closer to the center of the display module relative to other display partitions, and its area and text size may also be relatively larger to highlight the prompt information corresponding to the target object.

[0120] In some embodiments, the vehicle assisted driving method of the present application may further include steps S600 to S800:

[0121] In step S600, the vehicle driving behavior of the user is obtained.

[0122] In step S700, when the vehicle driving behavior affects the driving safety of the vehicle, correction prompt information for the vehicle driving behavior is generated.

[0123] In step S800, the correction prompt information is superimposed on the superimposed image to obtain a re-superimposed image, and the re-superimposed image is displayed through the augmented reality wearable device.

[0124] In this way, users can correct their bad driving behaviors and / or driving habits in real time according to the correction prompt information to further improve driving safety.

[0125] It should be noted that the driving safety of the vehicle corresponds to the driving risk of the vehicle. If the driving risk is high, the corresponding driving safety is low; otherwise, the driving safety is high. When it is determined that the vehicle may have accidents such as collision, scratching, rollover, etc. according to the user's vehicle driving behavior, or the user violates traffic rules, it is determined that the driving risk of the vehicle is high, that is, the driving safety of the vehicle will be threatened. In some embodiments, the real-time driving data and driving target of the vehicle in the current state can be obtained, and based on the driving target, the historical road condition information and historical operation information corresponding to the driving target are analyzed to predict the driving risk of the vehicle. Based on the real-time driving data and driving risk, at least one driving strategy displayed on the superimposed image can be determined for the user to view. Among them, the driving habit can be obtained by extracting historical driving features from the historical operation information, and according to the historical driving features, the driving habit model is used to obtain the user's driving habits. Among them, the historical operation information includes at least one of driving speed, braking information, steering information, lane change and overtaking information, and traffic rule violations. The driving habit model can include a model formed by the vehicle operation data of multiple users in the same area within a certain period of time.

[0126] Based on the above embodiment, step S600 of the vehicle assisted driving method of the present application may include step S610:

[0127] In step S610, the vehicle driving behavior to be performed by the user in the future is determined according to the user's driving habits and the real-time driving data of the vehicle.

[0128] In this embodiment, step S700 of the vehicle assisted driving method of the present application may include step S710:

[0129] In step S710, when the driving behavior of the vehicle to be operated affects the driving safety of the vehicle, correction prompt information of the driving behavior of the vehicle to be operated is generated in advance before the future time.

[0130] In this embodiment, step S800 of the vehicle assisted driving method of the present application may include step S810:

[0131] In step S810, before the future moment, the correction prompt information is superimposed on the superimposed image in advance to obtain a re-superimposed image, and the re-superimposed image is displayed through the augmented reality wearable device.

[0132] In this embodiment, the user's future driving behavior can be predicted based on the user's driving habits, and the user can be informed in advance of the problems with future driving behavior and the correct driving behavior. For example, based on the user's driving habits at the zebra crossing, information is obtained that the user is more likely to not give way to pedestrians at the zebra crossing, and it is predicted that the user will not give way to pedestrians at the next zebra crossing. Then, before the vehicle reaches the next zebra crossing, a correction prompt message "Please give way to pedestrians at the next zebra crossing" is displayed. The time to display the prompt message in advance can be determined based on the actual road conditions and vehicle speed, and this application does not impose any restrictions. The above-mentioned future moment represents a future time point or time period during the user's driving process.

[0133] In other embodiments, the vehicle assisted driving method of the present application can determine the driving risk of the current vehicle driving behavior based on the user's current vehicle driving behavior and display correction prompt information. For example, if the user currently has a driving behavior that violates traffic regulations such as running a red light, then at the next traffic light, the user can be prompted to wait until the green light comes on before starting the vehicle.

[0134] On the basis of the above embodiment, the vehicle assisted driving method of the present application may further include the following steps before step S610:

[0135] When the user drives the vehicle for the first time, vehicle information and user information are obtained.

[0136] Based on the vehicle information and the user information, desensitized driving habits of other users matching the vehicle and the user are obtained as the user's driving habits.

[0137] Among them, the vehicle information includes the vehicle model.

[0138] In this way, when a user drives a vehicle for the first time, the driving habits of users who drive the same vehicle as the user and / or similar users can be obtained, so as to predict the driving behavior of the vehicle to be operated in the future in a short time, thereby improving the efficiency of assisted driving.

[0139] In some embodiments, step S710 of the vehicle assisted driving method of the present application may include steps S711 to S712:

[0140] In step S711, the real-time distance between the vehicle and the vehicle ahead of the vehicle is acquired.

[0141] In step S712, when the real-time distance is less than the overtaking safety distance and the overtaking operation behavior in the vehicle driving behavior to be operated causes a risk of collision between the vehicle and the vehicle in front, overtaking correction prompt information is generated.

[0142] The above-mentioned safe overtaking distance can be determined according to general driving conditions. For example, when overtaking on urban roads, at least two parking spaces must be kept, and the safe overtaking distance is about 10m. If the real-time distance between the vehicle and the vehicle in front is less than 10m, and based on the user's driving habits, it is predicted that the user will overtake at this time, the real-time distance, the difference between the real-time distance and the safe overtaking distance, and a prompt to remind the user not to overtake at this time will be displayed on the augmented reality wearable device. If the real-time distance is less than or equal to the safe overtaking distance, only the real-time distance can be displayed.

[0143] The overtaking prompt information may also include whether the steering wheel is used in advance or delayed and whether the steering wheel angle is insufficient or excessive.

[0144] In some embodiments, step S710 of the vehicle assisted driving method of the present application may include steps S713 to S714:

[0145] In step S713, the road curve in the driving road condition information of the vehicle is obtained.

[0146] In step S714, when the steering wheel operation behavior in the vehicle driving behavior to be operated causes the vehicle to have a risk of collision on a road curve, a turn correction prompt information is generated.

[0147] When turning, the user turning the steering wheel in advance or in a delayed manner may bring about a collision risk. Displaying turning correction prompt information on the augmented reality wearable device can reduce the collision risk to a certain extent. For example, if the user is accustomed to turning the steering wheel in advance at a bend, the turning correction prompt information can be displayed to the user before the predicted moment when the user turns the steering wheel in advance; if the user is accustomed to turning the steering wheel in a delayed manner at a bend, the turning correction prompt information can be displayed to the user before the predicted moment when the user turns the steering wheel in a delayed manner.

[0148] In some embodiments, the vehicle assisted driving method of the present application may further include the following steps:

[0149] Detect the real-time light intensity of the ambient light.

[0150] When the real-time light intensity is greater than or equal to a light intensity threshold, the light transmittance of a lens component in the augmented reality wearable device is adjusted.

[0151] The augmented reality wearable device may further include a light intensity detection unit and a lens assembly transmittance adjustment unit. When the light intensity of the high beam of the oncoming vehicle is large and / or the light intensity of the street lamp is large, the driver will not be able to see the road ahead and objects on the road clearly, and the user may be blinded for a short time. The vehicle assisted driving method of this embodiment can adjust the transmittance of the lens assembly in time, such as when the transmittance is 0, filter out strong light, avoid the influence of strong light on the user's vision, and further improve driving safety. Strong light can be weakened or filtered through AR display technologies such as brightness / coverage / blurring.

[0152] In some embodiments, the augmented reality wearable device may include an audio processing unit and an audio output unit. In this embodiment, the audio inside and outside the car can be adjusted by the audio processing unit to reduce the impact of the sound on the user's attention when driving, while highlighting the sounds required for safe driving, such as horns, and weakening other surrounding sounds, such as human noise. The adjusted audio can be output to the user through the audio output unit.

[0153] Figure 5 This is a schematic diagram of the structure of an embodiment of the vehicle auxiliary driving device of the present application. Figure 5 As shown, the vehicle assisted driving device 100 of an embodiment of the present application is applied to an augmented reality wearable device for a user to wear, and includes an acquisition unit 110 and an image generation unit 120. Specifically, the acquisition unit 110 is used to acquire a real-time image of the current working scene of the vehicle, the real-time image includes a virtual preset area to be displayed, and acquires the real-time status information of the target object. The image generation unit 120 is used to parse the preset area, determine the target object, and superimpose the virtual model of the target object on the real-time image to generate an augmented reality image, and generate prompt information of the target object according to the current working scene and the real-time status information of the target object, superimpose the prompt information on the augmented reality image, obtain the superimposed image, and display the superimposed image through the augmented reality wearable device.

[0154] The electronic device of the embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the vehicle assisted driving method is implemented when the processor executes the program. Specifically, the electronic device may include a processor of an augmented reality wearable device, such as a processor of AR glasses. Figure 6 for Figure 5The structure diagram of the electronic device where the vehicle auxiliary driving device is located is shown. Figure 6 At the hardware level, the electronic device includes a processor 210, an internal bus 220, a network interface 230, a memory 240, and a non-volatile memory 250, and may also include hardware required for other services. One or more embodiments of the present application can be implemented based on software, such as the processor 210 reading the corresponding computer program from the non-volatile memory 250 into the memory 240 and then running it. Of course, in addition to the software implementation, one or more embodiments of the present application do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0155] The device implementation described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the present application. Those of ordinary skill in the art can understand and implement it without creative work.

[0156] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0157] The computer-readable storage medium of the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, a vehicle assisted driving method is implemented.

[0158] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0159] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0160] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0161] As for the method embodiment, since it basically corresponds to the device embodiment, the relevant parts can be referred to the partial description of the device embodiment. The method embodiment and the device embodiment complement each other.

[0162] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A vehicle assisted driving method, applied to a vehicle, wherein the vehicle includes an augmented reality wearable device for a user to wear, characterized in that: The vehicle assisted driving method comprises: Acquire a real-time image of a current working scene of the vehicle, wherein the real-time image includes a preset area of ​​a target object to be displayed outside and / or inside the vehicle; Analyzing the preset area, determining a target object corresponding to the preset area, and superimposing a virtual model of the target object on the real-time image to generate an augmented reality image; Acquire real-time status information of the target object; Generate prompt information of the target object according to the current working scene and the real-time status information of the target object; The prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

2. The vehicle assisted driving method according to claim 1, characterized in that: The vehicle assisted driving method further includes: When the movement speed in the real-time terminal information of the preset terminal of the vehicle is greater than or equal to a speed threshold, determining that the current working scene of the vehicle is a driving scene; When the current working scene of the vehicle is the driving scene, obtaining line of sight movement data obtained by dynamically tracking the line of sight of the user by the augmented reality wearable device; Based on the eye movement data, determining whether the user's sight is within a safe driving field of view; When the duration that the user's line of sight is not within the driving safety field of view is greater than or equal to the preset line of sight separation time, an automatic driving instruction is output to control the vehicle to enter the automatic driving state.

3. The vehicle assisted driving method according to claim 1, characterized in that: The target object includes a driving safety component; the driving safety component includes one or more of a driving unit, a braking unit and a steering unit; The generating prompt information of the target object according to the current working scene and the real-time status information of the target object includes: determining whether the driving safety component is in an abnormal state according to the state information of the driving safety component in the real-time state information of the target object; When the current working scene of the vehicle is a driving scene and the driving safety component is in the abnormal state, generating driving safety prompt information corresponding to the driving safety component; The step of superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device includes: The driving safety prompt information corresponding to the driving safety component is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

4. The vehicle assisted driving method according to claim 1, characterized in that: The target object includes a target object outside the vehicle; The generating prompt information of the target object according to the current working scene and the real-time status information of the target object includes: Determining whether the target object outside the vehicle is an obstacle according to the target object information outside the vehicle in the real-time status information of the target object; When the current working scene of the vehicle is a driving scene and the external target object is the obstacle, generating obstacle prompt information; The step of superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device includes: The obstacle prompt information is superimposed on the augmented reality image to obtain a superimposed image, and the superimposed image is displayed through the augmented reality wearable device.

5. The vehicle assisted driving method according to claim 4, characterized in that: The determining whether the external target object is an obstacle according to the external target object information in the real-time status information of the target object includes: Generate a motion path of the external target according to the external target position, the external target size, the external target movement speed and the external target movement direction in the external target information; In a case where the current working scene of the vehicle is the driving scene, determining whether the motion path of the external target object and the driving path of the vehicle at least partially overlap; When the movement path of the external object and the driving path of the vehicle at least partially overlap and the external object is located outside the user visible area, the external object is determined to be an obstacle.

6. The vehicle assisted driving method according to claim 1, characterized in that: The target object includes a plurality of target objects; the augmented reality wearable device includes a display module; superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device, comprising: determining, according to the current working scene and the plurality of target objects, display priorities corresponding to the plurality of target objects for displaying the prompt information; superimposing the prompt information on the augmented reality image in sequence according to the display priorities to obtain a superimposed image, and displaying the superimposed image through the display module; or, The target object includes multiple target objects; the augmented reality wearable device includes a display module, and the display module includes multiple display partitions; superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the augmented reality wearable device, including: according to the current working scene and the multiple target objects, determining the display partitions corresponding to the multiple target objects for displaying the prompt information, superimposing the prompt information on the augmented reality image to obtain a superimposed image, and displaying the superimposed image through the corresponding display partitions.

7. The vehicle assisted driving method according to claim 1, characterized in that: The vehicle assisted driving method further includes: Obtain the user's vehicle driving behavior; In the case where the vehicle driving behavior affects the driving safety of the vehicle, generating correction prompt information for the vehicle driving behavior; The correction prompt information is superimposed on the superimposed image to obtain a re-superimposed image, and the re-superimposed image is displayed through the augmented reality wearable device.

8. The vehicle assisted driving method according to claim 7, characterized in that: The obtaining of the user's vehicle driving behavior includes: Determine the vehicle driving behavior to be performed by the user in the future based on the user's driving habits and the vehicle's real-time driving data; In the case where the vehicle driving behavior affects the driving safety of the vehicle, generating correction prompt information of the vehicle driving behavior includes: In the case where the driving behavior of the vehicle to be operated affects the driving safety of the vehicle, generating correction prompt information of the driving behavior of the vehicle to be operated in advance before the future time; The step of superimposing the correction prompt information on the superimposed image to obtain a re-superimposed image, and displaying the re-superimposed image through the augmented reality wearable device includes: Before the future moment, the correction prompt information is superimposed on the superimposed image in advance to obtain a re-superimposed image, and the re-superimposed image is displayed through the augmented reality wearable device.

9. The vehicle assisted driving method according to claim 8, characterized in that: Before determining the vehicle driving behavior to be operated by the user in the future according to the user's driving habits and the real-time driving data of the vehicle, the method further includes: In the case where the user drives the vehicle for the first time, obtaining vehicle information and user information; According to the vehicle information and the user information, desensitized driving habits of other users matching the vehicle and the user are obtained as the user driving habits.

10. The vehicle assisted driving method according to claim 8, characterized in that: The generating of correction prompt information of the vehicle driving behavior when the vehicle driving behavior affects the driving safety of the vehicle includes: obtaining the real-time distance between the vehicle and the vehicle in front of the vehicle; generating overtaking correction prompt information when the real-time distance is less than the overtaking safety distance and the overtaking operation behavior in the vehicle driving behavior to be operated causes a risk of collision between the vehicle and the vehicle in front; and / or The method of generating correction prompt information for the vehicle driving behavior when the vehicle driving behavior affects the driving safety of the vehicle includes: obtaining a road curve in the driving road condition information of the vehicle; and generating turning correction prompt information when the steering wheel operation behavior in the vehicle driving behavior to be operated causes a risk of collision of the vehicle on the road curve.

11. The vehicle assisted driving method according to claim 1, characterized in that: The vehicle assisted driving method further includes: Detect the real-time light intensity of the ambient light; When the real-time light intensity is greater than or equal to a light intensity threshold, the transmittance of the lens assembly in the augmented reality wearable device is adjusted.

12. A vehicle auxiliary driving device, applied to an augmented reality wearable device worn by a user, characterized in that: The vehicle auxiliary driving device comprises: An acquisition unit, used to acquire a real-time image of a current working scene of the vehicle, the real-time image including a preset area to be displayed virtually, and to acquire real-time status information of a target object; An image generation unit is used to parse the preset area, determine the target object, and superimpose a virtual model of the target object on the real-time image to generate an augmented reality image, and generate prompt information of the target object according to the current working scene and the real-time status information of the target object, superimpose the prompt information on the augmented reality image to obtain a superimposed image, and display the superimposed image through the augmented reality wearable device.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the vehicle assisted driving method described in any one of claims 1 to 11 is implemented.

Images