Projection prompting method and device and vehicle

Abstract

The invention discloses a projection prompting method and device and a vehicle, and belongs to the technical field of vehicles. The projection prompting method can comprise the following steps: identifying a projection scene to which collected environmental data around a vehicle belongs; determining a ground projection area matched with the projection scene; generating a control signal for indicating a projection direction and a projection angle according to the projection area; and controlling a projection module of the vehicle to project a prompt graph matched with the projection scene in a projection area according to the control signal. According to the method, the obvious projection prompt can be automatically given according to the environment where the vehicle is located, so that the driving safety of the vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular to a projection prompting method, device and vehicle. Background Art

[0002] Whether it is for the vehicle's autonomous driving scenario or the driver's driving scenario, unclear lane lines or oncoming vehicles' deviation will pose a safety risk to the vehicle. Currently, the main method is to adjust the brightness of the vehicle's headlights to provide sufficient light for the vehicle or driver to identify the lane lines. However, in many cases, it is still impossible to accurately identify the lane lines and the driving conditions of oncoming vehicles. Therefore, there is still a lot of room for improvement in vehicle driving safety. Summary of the invention

[0003] In view of this, the present invention provides a projection prompt method, device and vehicle, which can automatically give relatively obvious projection prompts according to the environment in which the vehicle is located, so as to improve the driving safety of the vehicle.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] In a first aspect, the present invention provides a projection prompting method, comprising:

[0006] Identify the projection scene to which the collected environmental data around the vehicle belongs;

[0007] Determining a ground projection area that matches the projection scene;

[0008] generating a control signal indicating a projection direction and a projection angle according to the ground projection area;

[0009] According to the control signal, the projection module of the vehicle is controlled to project a prompt graphic matching the projection scene in the ground projection area.

[0010] Optionally, the identifying of the projection scene satisfied by the collected environmental data around the vehicle includes:

[0011] Identifying lane lines located on both sides of the vehicle from an environment image included in the environment data;

[0012] Analyzing whether the clarity of the lane line included in the environment image is lower than a preset clarity threshold;

[0013] When the analysis result indicates that the clarity of the lane line is lower than the preset clarity threshold, it is determined that the environmental data belongs to a projection scene indicating abnormal lane line recognition.

[0014] Optionally, the identifying of the projection scene satisfied by the collected environmental data around the vehicle further includes:

[0015] extracting driving features related to the oncoming vehicle from the environmental data, and inputting the driving features related to the oncoming vehicle into a pre-trained model;

[0016] Predicting, using the pre-trained model, whether the oncoming vehicle has a risk of yaw that crosses a lane line;

[0017] In a case where the oncoming vehicle has the risk of deviation, it is determined that the environmental data belongs to a projection scene indicating that the oncoming vehicle is deviating.

[0018] Optionally, the driving characteristics related to the oncoming vehicle include multiple of the following characteristics:

[0019] The speed of the oncoming vehicle, the wheel steering angle, the steering wheel movement, the head state of the driver of the oncoming vehicle, and the eye deviation field of view of the driver of the oncoming vehicle.

[0020] Optionally, the projection prompt method further includes:

[0021] Extracting historical driving features related to the oncoming vehicle from the historical environmental data of the vehicle;

[0022] The historical driving characteristics are used to train a model or to modify a pre-trained model.

[0023] Optionally, determining a ground projection area matching the projection scene includes:

[0024] In a case where the projection scene indicates that the oncoming vehicle is yawed, determining that the road surface in front of the oncoming vehicle is the ground projection area;

[0025] When the projection scene indicates that the lane line recognition is abnormal, the lane line area on one side or both sides of the vehicle is determined as the ground projection area.

[0026] Optionally, determining the lane line area on one side or both sides of the vehicle as a ground projection area includes:

[0027] Using machine vision to identify the lane line position on one side or both sides of the vehicle from the environmental image included in the environmental data;

[0028] or,

[0029] Based on the central axis of the vehicle and the width of the vehicle, the lane line positions on one side or both sides of the vehicle are located.

[0030] Optionally, the projection module controlling the vehicle projects a prompt graphic matching the projection scene in the ground projection area, including:

[0031] When the projection scene indicates that the oncoming vehicle is yawed, a warning pattern is projected on the road surface in front of the oncoming vehicle, and the display color of the warning pattern is adjusted according to the yaw condition of the oncoming vehicle;

[0032] When the projection scene indicates lane line recognition abnormality, solid lane lines or dashed lane lines are projected on the lane line area on one side or both sides of the vehicle, and the display color of the solid lane lines or the dashed lane lines is adjusted according to the environmental data.

[0033] In a second aspect, an embodiment of the present invention provides a projection prompting device, comprising: an identification module, a signal generation module and a control module, wherein:

[0034] The recognition module is used to recognize the projection scene to which the collected environmental data around the vehicle belongs;

[0035] The signal generating module is used to determine a ground projection area that matches the projection scene identified by the identifying module; and generate a control signal indicating a projection direction and a projection angle according to the ground projection area;

[0036] The control module is used to control the projection module of the vehicle to project a prompt graphic matching the projection scene in the ground projection area according to the control signal.

[0037] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0038] one or more processors;

[0039] a storage device for storing one or more programs,

[0040] When the one or more programs are executed by the one or more processors, the one or more processors implement the projection prompt method provided in the above-mentioned first aspect embodiment.

[0041] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having a computer program for implementing a visualization-assisted lane change method stored thereon.

[0042] When the computer program is executed by the vehicle-mounted processor, the projection prompt method provided in the first aspect embodiment described above is implemented.

[0043] In a fifth aspect, an embodiment of the present invention provides a vehicle, which implements the projection prompting method provided by the embodiment of the first aspect or includes the projection prompting device provided by the embodiment of the second aspect.

[0044] The technical solution of the above invention has the following advantages or beneficial effects:

[0045] The technical solution provided by the embodiment of the present invention determines a ground projection area matching the projection scene by identifying the projection scene to which the collected environmental data around the vehicle belongs, and controls the projection module of the vehicle to project a prompt graphic matching the projection scene in the projection area through a control signal indicating the projection direction and the projection angle generated according to the projection area. By coordinating the projection area and the prompt graphic with the projection scene, differentiated prompt graphics can be projected in the projection area according to different projection scenes, so that the prompt graphics can be clearly observed by the driver of the vehicle or the drivers of other vehicles in the projection area, that is, a relatively obvious projection prompt can be automatically given according to the environment in which the vehicle is located, so as to improve the driving safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0047] Figure 2 It is a schematic diagram of the main flow of the projection prompting method provided in an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of a projection scene of lane line recognition anomaly according to an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of a projection scenario in which an oncoming vehicle has a risk of yaw across a lane line, provided according to an embodiment of the present invention;

[0050] Figure 5 It is a schematic diagram of a projection scene provided according to an embodiment of the present invention in which lane line recognition anomalies exist and an oncoming vehicle has a risk of yaw across the lane line;

[0051] Figure 6 is a schematic diagram of a projection scene of a vehicle lane change prompt provided according to an embodiment of the present invention;

[0052] Figure 7 is a schematic diagram of a projection scene for a vehicle deceleration prompt provided according to an embodiment of the present invention;

[0053] Figure 8 It is a schematic diagram of the main process of projection prompting for a projection scene with abnormal lane line recognition provided by an embodiment of the present invention;

[0054] Fig. 9 It is a schematic diagram of the main process of projection prompting for a projection scene in which an oncoming vehicle has a risk of yaw across a lane line provided according to an embodiment of the present invention;

[0055] Fig.10It is a schematic diagram of the main process of projection prompting for a projection scene in which there is a lane line recognition anomaly and an oncoming vehicle has a risk of yaw across the lane line, provided according to an embodiment of the present invention;

[0056] Fig.11 is a partial structural schematic diagram of a projection prompting device provided according to an embodiment of the present invention;

[0057] Fig.12 is a partial structural schematic diagram of a projection prompting system provided according to an embodiment of the present invention;

[0058] Fig.13 It is a schematic diagram of the structure of a computer system suitable for implementing projection prompts according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0060] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0061] The following describes a technical scenario to which the technical solution provided by the embodiment of the present invention is applicable based on the system architecture on which the technical solution provided by the embodiment of the present invention relies.

[0062] Figure 1 An exemplary system architecture 100 is shown, to which the projection prompting method or projection prompting device according to an embodiment of the present invention can be applied.

[0063] like Figure 1 As shown, the vehicle system architecture 100 may include various systems, such as a driving control system 101, a power system 102, a sensor system 103, a control system 104, an auxiliary driving system 105, one or more peripheral devices 106, a power supply 107, a computer system 108, and a user interface 109, wherein the projection prompt method provided in the embodiment of the present invention may be implemented by interacting with each of the above systems. Optionally, the vehicle system architecture 100 may include more or fewer systems, and each system may include multiple components. In addition, each system and component of the vehicle system architecture 100 may be interconnected by wire or wirelessly.

[0064] The vehicle system architecture 100 includes a driving control system 101, which can be in a full or partial automatic driving mode. For example, the driving control system 101 can automatically control the vehicle to change lanes according to a lane change signal or lane change instruction provided by the auxiliary driving system 105 without human interaction.

[0065] The power system 102 may include components that provide power movement for the vehicle. For example, the power system 102 may include an engine, an energy source, a transmission, wheels, tires, etc. Among them, the engine may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine consisting of a gas-oil engine and an electric motor, and a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine converts the energy source into mechanical energy and provides it to the transmission. Examples of energy sources may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source may also provide energy for other systems of the vehicle. In addition, the transmission may include a gearbox, a differential, a drive shaft, a clutch, etc.

[0066] The sensor system 103 may include sensors for sensing the surrounding environment of the vehicle (such as sensors for sensing whether there are obstacles around) and pressure sensors for sensing whether there are passengers in the seats. For example, a positioning system (the positioning system may be a global positioning system (GPS) system, or a Beidou system or other positioning systems), a radar, a laser rangefinder, an inertial measurement unit (IMU), a camera, and a capacitive sensor. The positioning system may be used to locate the geographic location of the vehicle. The IMU is used to sense the position and orientation changes of the vehicle based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope. The radar may use radio signals to sense objects in the surrounding environment of the vehicle. The capacitive sensor is used to locate the metal wheel stopper on the parking space. In some embodiments, in addition to sensing objects, the radar may also be used to sense the speed and / or direction of travel of the object.

[0067] In order to detect environmental information, objects, etc. outside the vehicle, a camera or the like may be configured at an appropriate location outside the vehicle. For example, in order to obtain an environmental image of the side of the vehicle, the camera may be on a rearview mirror on the side of the vehicle. The camera may be a static or video camera.

[0068] The control system 104 may include a software system for implementing vehicle driving control, such as a system for analyzing the vehicle's surrounding environment, a system for pre-tensioning seat belts, a system for route planning, a system for avoiding obstacles, a visual system for image analysis, etc. The control system 104 may also include hardware systems such as a throttle, a steering wheel system, a seat belt system, an airbag system, and peripheral devices (such as a projection device, a display, etc.). In addition, the control system 104 may include components other than those shown and described in addition or in substitution. Alternatively, some of the components shown above may be reduced.

[0069] Furthermore, as described above, the control system 1204 can further implement a part of the above-mentioned projection prompt method, identifying the projection scene to which the collected environmental data around the vehicle belongs; determining the ground projection area that matches the projection scene; generating a control signal indicating the projection direction and projection angle according to the projection area; and controlling the projection module of the vehicle to project a prompt graphic that matches the projection scene in the projection area according to the control signal.

[0070] In addition, the control system 104 may also interact with external sensors, other autonomous driving devices, other computer systems, or users through the peripheral devices 106. The peripheral devices 106 may include wireless communication systems, onboard computers, microphones and / or speakers, cameras, and projectors.

[0071] After receiving the trigger, the assisted driving system 105 assists the power system 102 and the control system 104 to control the vehicle driving. In addition, the assisted driving system 105 interacts with the control system 104 to provide the control system 104 with a driving trigger, so that the control system 104 starts executing a program related to the projection prompt method based on the driving trigger, and through the program of the projection prompt method, executes: identifying the projection scene; determining the ground projection area; controlling the projection module to project a prompt graphic matching the projection scene in the projection area.

[0072] In some embodiments, the peripheral device 106 provides a means for a user of the control system 104 to interact with the user interface. For example, an onboard computer can provide information to a user of the vehicle. The user interface can also operate the onboard computer to receive input from the user. The onboard computer can be operated via a touch screen. In other cases, the peripheral device can provide a means for communicating with other devices located in the vehicle. For example, a microphone can receive audio (e.g., voice commands or other audio input) from a user of the control system 104. Similarly, a speaker can output audio to the user of the control system 104.

[0073] The wireless communication system can communicate with one or more devices wirelessly directly or via a communication network. For example, the wireless communication system can communicate with a network such as a cellular network, WiFi, and a wireless local area network (WLAN), or can communicate directly with a device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various autonomous driving communication systems, etc.

[0074] The power source 107 can provide power to various components of the vehicle. The power source 107 can be a rechargeable lithium-ion or lead-acid battery.

[0075] Some or all functions of implementing projection prompts are controlled by computer system 108. Computer system 108 may include at least one processor that executes instructions stored in a non-transitory computer-readable medium such as a memory. Computer system 108 provides execution code for implementing projection prompts for the control system.

[0076] The processor can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Those of ordinary skill in the art will appreciate that the processor, computer, or memory can actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, the memory can be a hard disk drive or other storage media that is located in a housing different from the computer. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Different from using a single processor to perform the steps described herein, some components such as steering components and deceleration components can each have their own processors that only perform calculations related to the functions specific to the components.

[0077] The user interface 109 is used to provide information to or receive information from a user of the vehicle. Optionally, the user interface 109 may include one or more input / output devices within the set of peripheral devices 106, such as a wireless communication system, an onboard computer, a microphone, and a speaker.

[0078] It should be understood that the above components are only examples. In actual applications, the components in the above modules or systems may be added or deleted according to actual needs. Figure 1 It should not be understood as limiting the embodiments of the present application.

[0079] Figure 2 is a schematic diagram of the main steps of a projection prompting method according to an embodiment of the present invention. Specifically, Figure 2 As shown, the projection prompt method mainly includes the following steps:

[0080] Step S201: Identify the projection scene: identify the projection scene to which the collected environmental data around the vehicle belongs.

[0081] The environmental data generally includes road data collected by the vehicle's sensors, such as obstacle locations, oncoming vehicle driving conditions, the vehicle's speed, road conditions, etc., and environmental images of the vehicle's environment collected by camera equipment. The environmental images may be images of the front, side, and / or rear of the vehicle.

[0082] Among them, the projection scene can be set according to the user. For example, the projection scene can be lane line recognition anomaly, oncoming vehicle deviation, vehicle deceleration, vehicle lane change, etc. In addition, the mapping relationship between the projection scene, the environmental data matching the projection scene, and the ground projection area matching the projection scene can also be set by the user. In the future, the projection scene and the ground projection area can be determined based on the mapping relationship, which helps to improve the flexibility of the projection scene and projection prompts, better meet user needs, and improve user experience.

[0083] Step S202: Determine the ground projection area: determine the ground projection area that matches the projection scene.

[0084] It can be understood that the ground projection area is limited by the adjustable angle and projection range of the projection module of the vehicle. Within the adjustable angle and projection range of the projection module, the user can define the corresponding ground projection area for the projection scene according to the actual situation. It is worth noting that the ground projection area is generally a relative area, such as the distance relative to the front of the vehicle, the distance relative to the central axis of the vehicle, the distance relative to the rear of the vehicle, the distance relative to the front of the same-direction vehicle on the adjacent lane, and the position relative to the oncoming vehicle.

[0085] Step S203: Generate a control signal: Generate a control signal indicating a projection direction and a projection angle according to the ground projection area.

[0086] The control signal indicating the projection direction and the projection angle can adjust the projection angle and the projection direction of the projection module of the vehicle to ensure that the prompt graphic projected subsequently can fall into the ground projection area.

[0087] The projection direction and the projection angle can be obtained by the direction of the line between the projection module of the vehicle and the ground projection area relative to the central axis of the vehicle and the angle between the line and the central axis of the vehicle. The direction relative to the central axis of the vehicle and the angle between the line and the central axis of the vehicle can be obtained by the existing technology and will not be described in detail here.

[0088] Step S204: Control projection: According to the control signal, control the projection module of the vehicle to project a prompt graphic matching the projection scene in the ground projection area.

[0089] The prompt graphic may be pre-configured or customized by the user.

[0090] against Figure 2 The provided technical solution determines a ground projection area that matches the projection scene by identifying the projection scene to which the collected environmental data around the vehicle belongs, and controls the projection module of the vehicle to project a prompt graphic that matches the projection scene in the projection area through a control signal indicating the projection direction and projection angle generated according to the projection area. By coordinating the projection area and the prompt graphic with the projection scene, differentiated prompt graphics can be projected in the projection area according to different projection scenes, so that the prompt graphic can be clearly observed by the driver of the vehicle or the drivers of other vehicles in the projection area, that is, a relatively obvious projection prompt can be automatically given according to the environment in which the vehicle is located, so as to improve the driving safety of the vehicle.

[0091] Exemplarily, for a projection scene with abnormal lane line recognition, in order to be able to identify the projection scene with abnormal lane line recognition, the specific implementation of the above-mentioned step S201 may include: identifying the lane lines located on both sides of the vehicle from the environmental image included in the environmental data; analyzing whether the clarity of the lane lines included in the environmental image is lower than a preset clarity threshold; when the analysis result indicates that the clarity of the lane lines is lower than the preset clarity threshold, determining that the environmental data belongs to a projection scene indicating abnormal lane line recognition.

[0092] Among them, the lane lines located on both sides of the vehicle can be identified from the environmental image by machine vision, or by other existing technical means, and the technical means for identifying the lane lines is not limited here.

[0093] In addition, the clarity of the lane line can be analyzed to determine the clarity of the lane line based on the grayscale of the identified lane line or the contrast between the lane line and the road.

[0094] It is understandable that, when the analysis result indicates that the clarity of the lane line is greater than or equal to the preset clarity threshold, it is determined that the environmental data does not belong to the projection scene indicating abnormal lane line recognition. In addition, when the lane line is not recognized from the environmental image, it is directly determined that the environmental data belongs to the projection scene indicating abnormal lane line recognition.

[0095] Furthermore, for the projection scene with lane line recognition abnormality, when the projection scene indicates lane line recognition abnormality, the lane line area on one side or both sides of the vehicle is determined as the ground projection area. Specifically, there are two technical means to determine the lane line area on one side or both sides of the vehicle as the ground projection area.

[0096] Among them, the first technical means to determine the lane line area on one side or both sides of the vehicle as the ground projection area is to use machine vision to identify the lane line position on one side or both sides of the vehicle from the environmental image included in the environmental data.

[0097] The second technical means of determining the lane line area on one side or both sides of the vehicle as the ground projection area: based on the central axis of the vehicle and the width of the vehicle, the lane line position on one side or both sides of the vehicle is located.

[0098] When the projection scene indicates lane line recognition abnormality, solid lane lines or dashed lane lines are projected on the lane line area on one side or both sides of the vehicle, and the display color of the solid lane lines or dashed lane lines is adjusted according to the environmental data.

[0099] For example, Figure 3 and Figure 5 As shown, it is identified that the environmental data of the vehicle V1 belongs to the projection scene of lane line recognition abnormality, and the clarity of the solid lane lines on both sides of the vehicle is lower than the preset clarity threshold, then the lane line area on both sides of the vehicle V1 is determined to be the ground projection area, and the solid lane line P2 is projected in the ground projection area. In addition, the display color of the solid lane line P2 can also be adjusted, such as red, yellow, green, blue, etc., to more prominently remind the driver or the auxiliary driving system.

[0100] For example, Figure 7 As shown, it is identified that the environmental data of the vehicle V1 belongs to the projection scene of abnormal lane line recognition, and the clarity of the solid lane line on one side of the vehicle is lower than the preset clarity threshold, then the lane line area on the side of the vehicle V1 is determined to be the ground projection area, and the dotted lane line P5 is projected in the ground projection area. In addition, the display color of the dotted lane line P5 can also be adjusted, such as red, yellow, green, blue, etc., to more prominently remind the driver or the auxiliary driving system.

[0101] For example, for a projection scene in which an oncoming vehicle has a risk of yaw across a lane line, in order to be able to identify the projection scene in which the oncoming vehicle has a risk of yaw across a lane line, the specific implementation of the above step S201 may include: extracting driving features related to the oncoming vehicle from environmental data, inputting the driving features related to the oncoming vehicle into a pre-trained model; using the pre-trained model to predict whether the oncoming vehicle has a risk of yaw across a lane line; and in the case that the oncoming vehicle has a risk of yaw, determining that the environmental data belongs to a projection scene indicating that the oncoming vehicle has yawed. The driving features related to the oncoming vehicle include multiple of the following features: the speed of the oncoming vehicle, the wheel deflection angle, the steering wheel movement, the head state of the driver of the oncoming vehicle, and the eyeball deflection field of the driver of the oncoming vehicle.

[0102] The pre-trained model can accurately and quickly identify whether the oncoming vehicle has the risk of crossing the lane line.

[0103] Furthermore, for projection scenes where there is a risk of the oncoming vehicle yawing across the lane line, the projection prompt method may further include: extracting historical driving features related to the oncoming vehicle from the historical environmental data of the vehicle; using the historical driving features to train the model or to correct the pre-trained model. By using the historical driving features related to the oncoming vehicle in the historical environmental data of the vehicle to train the model, the trained model or the corrected model can be more in line with the actual situation of the vehicle, so as to further improve the accuracy and reliability of identifying projection scenes indicating the yaw of the oncoming vehicle.

[0104] In addition, for the projection scene where the oncoming vehicle has the risk of yaw across the lane line, the specific implementation of the above step S202 may include: when the projection scene indicates that the oncoming vehicle is yawed, determining the road surface in front of the oncoming vehicle as the ground projection area. Further, the specific implementation of the above step S204 may include: when the projection scene indicates that the oncoming vehicle is yawed, projecting a warning pattern on the road surface in front of the oncoming vehicle, and adjusting the display color of the warning pattern according to the yaw condition of the oncoming vehicle; so as to give a clear and obvious warning pattern so that the driver of the oncoming vehicle or the auxiliary driving system can accurately obtain the warning pattern.

[0105] For example, Figure 4 and Figure 5As shown, if the oncoming vehicle V2 has the risk of deviating from the lane where the vehicle V1 is located, the area in front of the oncoming vehicle V2 is determined as the ground projection area, and the projection module of the vehicle V1 projects a warning graphic P1 for the risk of deviation in the ground projection area. In addition, the display color of the warning graphic P1 for the risk of deviation can also be adjusted, such as red, yellow, green, blue, etc., to remind the driver or the auxiliary driving system more prominently. It can be understood that the warning graphic for the risk of deviation can also be designed as other graphics, and the graphic style is not limited here. Warning words such as "Deviation" can also be projected.

[0106] In addition, in addition to the above-mentioned projection scenes of abnormal lane line recognition and the projection scenes of the oncoming vehicle having the risk of crossing the lane line, the vehicle lane change can also be used as a projection scene. For example, Figure 6 As shown, the vehicle V1 wants to change lanes to the adjacent lane. It monitors that there is another vehicle V3 behind the vehicle V1 in the adjacent lane, and then projects a prompt graphic P3 in front of the vehicle V3 to warn the vehicle to change lanes. In addition, the display color of the prompt graphic P3 to warn the vehicle to change lanes can also be adjusted, such as red, yellow, green, blue, etc., to more prominently remind the driver or the auxiliary driving system. It can be understood that the prompt graphic P3 to warn the vehicle to change lanes can also be designed as other graphics, and the graphic style is not limited here. Warning words such as "vehicle changing lanes" can also be projected.

[0107] In addition, the vehicle deceleration can also be used as a projection scene, for example, Figure 7 As shown, the vehicle V1 slows down, and it is monitored that there is another vehicle V4 behind the vehicle V1 in the lane where the vehicle V1 is located, then a prompt graphic P4 is projected in front of the vehicle V4 to warn the vehicle to slow down. In addition, the display color of the prompt graphic P4 to warn the vehicle to slow down can also be adjusted, such as red, yellow, green, blue, etc., to more conspicuously remind the driver or the auxiliary driving system. It can be understood that the prompt graphic P4 to warn the vehicle to slow down can also be designed as other graphics, and the graphic style is not limited here. Warning words such as "vehicle slow down" can also be projected.

[0108] It is worth noting that the above projection scenarios (lane line recognition abnormality, oncoming vehicle has the risk of crossing the lane line, vehicle lane change and vehicle deceleration) can be recognized simultaneously or one by one. Figure 5 As shown, the vehicle V1 simultaneously identifies that the lane line is abnormal and the oncoming vehicle V2 has the risk of crossing the lane line. Accordingly, the projection module simultaneously projects the lane line P2 and the warning graphic P1 of the risk of deviation. For another example, Figure 7As shown, the vehicle V1 simultaneously identifies the lane line recognition abnormality and the vehicle V1 deceleration. Accordingly, the projection module synchronously projects the lane line P5 and the prompt graphic P4 that warns the vehicle to decelerate. For another example, the vehicle V1 is only configured with a projection scene that identifies the lane line recognition abnormality. Accordingly, Figure 3 As shown, when the vehicle V1 identifies an abnormal lane line, the corresponding lane line P2 is projected. In addition, the vehicle can also be configured according to user needs to only project scenes where the oncoming vehicle has a risk of deviation, or a projection scene where the vehicle decelerates, or a projection scene where the vehicle changes lanes.

[0109] The following uses several projection scenes as examples to illustrate the main processes of the projection prompt method provided by the embodiments of the present invention.

[0110] For example, only the projection scene indicating lane line recognition abnormality is configured on the vehicle V1, such as Figure 8 As shown, for a projection scene configured only with an indication of lane line recognition anomaly, the projection prompt method may include the following steps:

[0111] Step S801: Identify lane lines: Identify lane lines on both sides of the vehicle from the environmental image included in the environmental data; if the lane lines are identified Y1, execute step S802; if the lane lines are not identified N1, execute step S803.

[0112] Step S802: Lane line clarity comparison: Analyze whether the clarity of the lane lines included in the environmental image is lower than the preset clarity threshold; if the analysis result indicates that the clarity of the lane lines is lower than the preset clarity threshold Y2, execute step S803; if the analysis result indicates that the clarity of the lane lines is equal to or higher than the preset clarity threshold Y2, execute step S804.

[0113] Step S803: Determine lane line recognition anomaly: Determine that the environmental data belongs to a projection scene indicating lane line recognition anomaly, and execute step S805.

[0114] Step S804: Determine that lane line recognition is normal and end the current process.

[0115] Step S805: Determine the ground projection area: determine the lane line area on one side or both sides of the vehicle as the ground projection area.

[0116] Step S806: Generate a control signal: Generate a control signal indicating a projection direction and a projection angle according to the ground projection area.

[0117] Step S807: Projecting lane lines: According to the control signal, the projection angle and transmission direction of the projection module are adjusted, and the projection module of the vehicle is controlled to project lane lines in the ground projection area.

[0118] For example, only the vehicle V1 is configured with a projection scene indicating that the oncoming vehicle has a risk of deviation, such as Fig. 9 As shown, for a projection scene configured only with an indication that an oncoming vehicle has a risk of yaw, the projection prompt method may include the following steps:

[0119] Step S901: Predicting the yaw risk of the oncoming vehicle: extracting the speed, wheel deflection angle, steering wheel movement, head state of the driver of the oncoming vehicle and eye deflection field of the driver of the oncoming vehicle from the environmental data, and inputting the speed, wheel deflection angle, steering wheel movement, head state of the driver of the oncoming vehicle and eye deflection field of the driver of the oncoming vehicle into a pre-trained model; using the pre-trained model to predict whether the oncoming vehicle has a yaw risk of crossing the lane line; if it is predicted that the oncoming vehicle has a yaw risk, Y3, executing step S902; if it is predicted that the oncoming vehicle does not have a yaw risk, N3, executing step S903.

[0120] Step S902: Determine the oncoming vehicle yaw: Determine whether the environmental data belongs to a projection scene indicating the oncoming vehicle yaw, and execute step S904.

[0121] Step S903: Determine that the oncoming vehicle is traveling normally and end the current process.

[0122] Step S904: Determine the ground projection area: determine the road surface in front of the oncoming vehicle as the ground projection area.

[0123] Step S905: Generate a control signal: Generate a control signal indicating a projection direction and a projection angle according to the ground projection area.

[0124] Step S906: Projecting a graphic that warns the oncoming vehicle of yaw: According to the control signal, the projection angle and transmission direction of the projection module are adjusted, and the projection module of the vehicle is controlled to project a graphic that warns the oncoming vehicle of yaw in the ground projection area in front of the oncoming vehicle.

[0125] For example, a projection scene indicating abnormal lane recognition and a projection scene indicating the risk of deviation of the oncoming vehicle are configured on the vehicle V1, such as Fig.10 As shown, for a projection scene configured only with an indication that an oncoming vehicle has a risk of yaw, the projection prompt method may include the following steps:

[0126] Step S1001: synchronously identify multiple projection scenes: identify lane lines from environmental data and predict the yaw risk of oncoming vehicles; when the lane line is not identified and it is predicted that the oncoming vehicle has a yaw risk or when the lane line is identified and the lane line clarity is lower than the preset clarity threshold and it is predicted that the oncoming vehicle has a yaw risk Y4, execute step S1002; when the lane line is not identified and it is predicted that the oncoming vehicle does not have a yaw risk or when the lane line is identified and the lane line clarity is lower than the preset clarity threshold and it is predicted that the oncoming vehicle does not have a yaw risk N4, execute step S1004; when the lane line is identified and the lane line clarity is not lower than the preset clarity threshold and it is predicted that the oncoming vehicle has a yaw risk Y5, execute step S1006; when the lane line is identified and the lane line clarity is not lower than the preset clarity threshold and it is predicted that the oncoming vehicle does not have a yaw risk N5, execute step S1008.

[0127] Step S1002: Determine the oncoming vehicle deviation and lane line recognition abnormality: Determine that the environmental data belongs to a projection scene indicating the oncoming vehicle deviation and a projection scene indicating the lane line recognition abnormality.

[0128] Step S1003: Projecting lane lines and graphics that warn oncoming vehicles of yaw: Determine the lane line area on one side or both sides of the vehicle and the road surface in front of the oncoming vehicle as the ground projection area; generate a control signal indicating the projection direction and projection angle according to the ground projection area; according to the control signal, adjust the projection angle and transmission direction of the projection module, and control the projection module of the vehicle to project lane lines on the lane line area on one side or both sides of the vehicle, and project a graphics that warns oncoming vehicles of yaw on the ground projection area in front of the oncoming vehicle, and end the current process.

[0129] Step S1004: Determine lane line recognition anomaly: Determine that the environmental data belongs to a projection scene indicating lane line recognition anomaly.

[0130] Step S1005: Projecting lane lines: Determine the lane line area on one side or both sides of the vehicle and the road surface in front of the oncoming vehicle as the ground projection area; generate a control signal indicating the projection direction and projection angle according to the ground projection area; according to the control signal, adjust the projection angle and transmission direction of the projection module, and control the projection module of the vehicle to project the lane line on the lane line area on one side or both sides of the vehicle, and end the current process.

[0131] Step S1006: Determine the oncoming vehicle yaw: Determine a projection scene in which the environmental data indicates abnormal lane line recognition.

[0132] Step S1007: Projecting a graphic to warn the oncoming vehicle of yaw: Determine the road surface in front of the oncoming vehicle as the ground projection area; generate a control signal indicating the projection direction and projection angle based on the ground projection area; adjust the projection angle and transmission direction of the projection module based on the control signal, and control the projection module of the vehicle to project a graphic to warn the oncoming vehicle of yaw in the ground projection area in front of the oncoming vehicle, and end the current process.

[0133] Step S1008: Determine that the oncoming vehicle is driving normally and end the current process.

[0134] In addition, the vehicle may be configured with a projection scene for deceleration and a projection scene for lane change. The recognition process and projection graphics for the projection scene for deceleration and the projection scene for lane change have been described in the above embodiments and will not be repeated here.

[0135] It is worth mentioning that for projecting lane lines and graphics to warn oncoming vehicles of vehicle deviation, the vehicle's projection module can be equipped with an adjustable light array in the front headlights. By controlling the angle and projection direction of the light array, the color of the projected light of the light array, etc., it is possible to project specific graphics of specific colors.

[0136] In addition, a light array with adjustable light emission can be set on the vehicle's rearview mirror and rear lights to project specific graphics of specific colors behind the vehicle through the rear lights or to project specific graphics of specific colors on adjacent lanes of the vehicle through the light array set on the vehicle's rearview mirror.

[0137] It should be noted that the traditional projection method may be able to project the lane lines on a straight road through the trapezoidal correction of the image, but the trapezoidal correction of the traditional projection method is a straight line projection, which may be able to project according to the lane lines when the vehicle is moving in a straight line. However, for the lane lines in the curved area, as the vehicle turns, the projection angle of the traditional projection method will continue to change, which will cause the traditional projection method to be unsuitable for highlighting the lane lines on the curved road. Compared with the traditional projection method, the embodiment of the present invention provides a process of projecting lane lines or specific graphics through the light array in the projection module on the vehicle. Since the angle of the light array is independently controlled, during the vehicle turning process, the angle of the array can be directly corrected to achieve the purpose of correcting the projection position of the lane line or the projection position of the specific graphic. Therefore, in addition to the straight road scene, the technical solution provided by the embodiment of the present invention is also applicable to the curved scene, and can calibrate the projection position of the lane lines or specific patterns in the curved scene.

[0138] Furthermore, an embodiment of the present invention also provides a projection prompting device. Fig.11As shown, the projection prompting device 1100 may include: an identification module 1101, a signal generating module 1102 and a control module 1103, wherein:

[0139] An identification module 1101 is used to identify the projection scene to which the collected environmental data around the vehicle belongs;

[0140] The signal generating module 1102 is used to determine the ground projection area matching the projection scene identified by the identifying module 1101; and generate a control signal indicating the projection direction and projection angle according to the projection area;

[0141] The control module 1103 is used to control the projection module 1201 of the vehicle to project a prompt graphic matching the projection scene in the projection area according to the control signal.

[0142] Furthermore, the recognition module 1101 is further used to identify lane lines located on both sides of the vehicle from the environmental image included in the environmental data; analyze whether the clarity of the lane lines included in the environmental image is lower than a preset clarity threshold; when the analysis result indicates that the clarity of the lane lines is lower than the preset clarity threshold, determine that the environmental data belongs to a projection scene indicating lane line recognition abnormality.

[0143] Furthermore, the identification module 1101 is further used to extract driving features related to the oncoming vehicle from the environmental data, and input the driving features related to the oncoming vehicle into a pre-trained model; use the pre-trained model to predict whether the oncoming vehicle has a risk of yaw across the lane line; when the oncoming vehicle has a risk of yaw, determine that the environmental data belongs to a projection scene indicating that the oncoming vehicle is yawing.

[0144] Furthermore, the driving characteristics related to the oncoming vehicle include multiple of the following characteristics: the vehicle speed of the oncoming vehicle, the wheel deflection angle, the steering wheel movement, the head state of the driver of the oncoming vehicle, and the eye deflection field of view of the driver of the oncoming vehicle.

[0145] Furthermore, the recognition module 1101 is further used to extract historical driving features related to the oncoming vehicle from the historical environmental data of the vehicle; and use the historical driving features to train a model or correct a pre-trained model.

[0146] Furthermore, the signal generating module 1102 is further used to determine that the road surface in front of the oncoming vehicle is the ground projection area when the projection scene indicates that the oncoming vehicle is yawed; and to determine that the lane line area on one side or both sides of the vehicle is the ground projection area when the projection scene indicates that the lane line recognition is abnormal.

[0147] Furthermore, the signal generating module 1102 is further used to use machine vision to identify the lane line position on one side or both sides of the vehicle from the environmental image included in the environmental data; or, based on the central axis of the vehicle and the width of the vehicle, locate the lane line position on one side or both sides of the vehicle.

[0148] Furthermore, the control module 1103 is further used to project a warning pattern on the road surface in front of the oncoming vehicle when the projection scene indicates that the oncoming vehicle has yawed, and adjust the display color of the warning pattern according to the yaw condition of the oncoming vehicle; when the projection scene indicates that the lane line recognition is abnormal, project a solid lane line or a dotted lane line on the lane line area on one side or both sides of the vehicle, and adjust the display color of the solid lane line or the dotted lane line according to the environmental data.

[0149] Furthermore, an embodiment of the present invention also provides a projection prompting system. Fig.12 As shown, the projection prompting system 1200 may include: a projection module 1201 and the above-mentioned projection prompting device 1100 .

[0150] The projection module 1201 may be disposed on the front side, rear side or rearview mirror of the vehicle.

[0151] Furthermore, an embodiment of the present invention also provides an electronic device. The electronic device may include:

[0152] one or more processors;

[0153] a storage device for storing one or more programs,

[0154] When the one or more programs are executed by the one or more processors, the one or more processors implement the projection prompt method provided in the above embodiment.

[0155] Furthermore, an embodiment of the present invention also provides a computer-readable medium, on which a computer program for implementing a visualization-assisted lane-changing method is stored.

[0156] When the computer program is executed by the vehicle-mounted processor, the projection prompt method provided in the above embodiment is implemented.

[0157] Reference below Fig.13 , which shows a schematic diagram of the structure of a computer system 1300 suitable for implementing the projection prompting method according to an embodiment of the present invention. Fig.13 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0158] like Fig.13As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the system 1300 are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0159] The following components are connected to the I / O interface 1305: an input section 1306 including an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0160] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the central processing unit (CPU) 1301, the above-mentioned functions defined in the system of the present invention are executed.

[0161] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0162] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes the above-mentioned identification module, signal generation module and control module. The names of these modules or units do not constitute a limitation on the modules or units themselves in some cases. For example, the identification module may also be described as "a module or unit for identifying the projection scene to which the collected environmental data around the vehicle belongs".

[0164] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: identifying the projection scene to which the collected environmental data around the vehicle belongs; determining the ground projection area that matches the projection scene; generating a control signal indicating the projection direction and projection angle according to the projection area; and controlling the projection module of the vehicle to project a prompt graphic that matches the projection scene in the projection area according to the control signal.

[0165] According to the technical solution of the embodiment of the present invention, by identifying the projection scene to which the collected environmental data around the vehicle belongs, a ground projection area matching the projection scene is determined, and by a control signal indicating the projection direction and projection angle generated according to the projection area, the projection module of the vehicle is controlled to project a prompt graphic matching the projection scene in the projection area. By coordinating the projection area and the prompt graphic with the projection scene, differentiated prompt graphics can be projected in the projection area according to different projection scenes, so that the prompt graphic can be clearly observed by the driver of the vehicle or the drivers of other vehicles in the projection area, that is, a relatively obvious projection prompt can be automatically given according to the environment in which the vehicle is located, so as to improve the driving safety of the vehicle.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A projection prompting method, characterized in that: include: Identify the projection scene to which the collected environmental data around the vehicle belongs; Determining a ground projection area that matches the projection scene; generating a control signal indicating a projection direction and a projection angle according to the ground projection area; According to the control signal, the projection module of the vehicle is controlled to project a prompt graphic matching the projection scene in the ground projection area.

2. The projection prompting method according to claim 1, characterized in that: The projection scene satisfied by the environmental data around the vehicle collected by the identification includes: Identifying lane lines located on both sides of the vehicle from an environment image included in the environment data; Analyzing whether the clarity of the lane line included in the environment image is lower than a preset clarity threshold; When the analysis result indicates that the clarity of the lane line is lower than the preset clarity threshold, it is determined that the environmental data belongs to a projection scene indicating abnormal lane line recognition.

3. The projection prompting method according to claim 1 or 2, characterized in that: The identification of the projection scene satisfied by the collected environmental data around the vehicle also includes: extracting driving features related to the oncoming vehicle from the environmental data, and inputting the driving features related to the oncoming vehicle into a pre-trained model; Predicting, using the pre-trained model, whether the oncoming vehicle has a risk of yaw that crosses a lane line; In a case where the oncoming vehicle has the risk of deviation, it is determined that the environmental data belongs to a projection scene indicating that the oncoming vehicle is deviating.

4. The projection prompting method according to claim 3, characterized in that: The driving characteristics related to the oncoming vehicle include multiple of the following characteristics: The speed of the oncoming vehicle, the wheel steering angle, the steering wheel movement, the head state of the driver of the oncoming vehicle, and the eye deviation field of view of the driver of the oncoming vehicle.

5. The projection prompting method according to claim 3, characterized in that: Also includes: Extracting historical driving features related to the oncoming vehicle from the historical environmental data of the vehicle; The historical driving characteristics are used to train a model or to modify a pre-trained model.

6. The projection prompting method according to claim 1, characterized in that: The determining of a ground projection area matching the projection scene comprises: In a case where the projection scene indicates that the oncoming vehicle is yawed, determining that the road surface in front of the oncoming vehicle is the ground projection area; When the projection scene indicates that the lane line recognition is abnormal, the lane line area on one side or both sides of the vehicle is determined as the ground projection area.

7. The projection prompting method according to claim 6, characterized in that: The determining the lane line area on one side or both sides of the vehicle as the ground projection area includes: Using machine vision to identify the lane line position on one side or both sides of the vehicle from the environmental image included in the environmental data; or, Based on the central axis of the vehicle and the width of the vehicle, the lane line positions on one side or both sides of the vehicle are located.

8. The projection prompting method according to claim 6, characterized in that: The projection module controlling the vehicle projects a prompt graphic matching the projection scene in the ground projection area, including: When the projection scene indicates that the oncoming vehicle is yawed, a warning pattern is projected on the road surface in front of the oncoming vehicle, and the display color of the warning pattern is adjusted according to the yaw condition of the oncoming vehicle; When the projection scene indicates lane line recognition abnormality, solid lane lines or dashed lane lines are projected on the lane line area on one side or both sides of the vehicle, and the display color of the solid lane lines or the dashed lane lines is adjusted according to the environmental data.

9. A projection prompting device, characterized in that: include: Identification module, signal generation module and control module, wherein: The recognition module is used to recognize the projection scene to which the collected environmental data around the vehicle belongs; The signal generating module is used to determine a ground projection area that matches the projection scene identified by the identifying module; and generate a control signal indicating a projection direction and a projection angle according to the ground projection area; The control module is used to control the projection module of the vehicle to project a prompt graphic matching the projection scene in the ground projection area according to the control signal.

10. A vehicle, characterized in that: It comprises a projection module and the projection prompting device as claimed in claim 9.

Images