Assisted driving method, device, electronic device and storage medium
By determining the road type and driver status, the assisted driving system automatically adjusts the assisted driving mode, solving the problem of poor user experience of assisted driving systems in the prior art under complex road environments, and improving vehicle safety and user satisfaction.
Patent Information
- Application Number
- CN202210264870.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-17
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-03-17
AI Technical Summary
The existing assisted driving system is difficult to perform reasonable and effective assisted driving under the conditions of complex road types, resulting in poor user experience.
By determining the road type based on the current position of the vehicle, combining the lane line information collected by the visual perception system and the driver image collected by the internal image acquisition device, the assisted driving mode, including a preset reminder mode and a takeover mode, is determined to perform assisted driving.
It improves the safety of vehicle driving, meets the driver's needs for assisted driving functions, avoids unnecessary frequent reminders, and improves the user experience.
Smart Images

Figure CN114506332B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle technology, and in particular to an assisted driving method, device, electronic device and storage medium. Background Art
[0002] The assisted driving system uses a perception system installed on the vehicle (such as radar, visual perception system and positioning system) to obtain environmental data around the vehicle during driving, process and analyze it, and then assist the driver in controlling the vehicle and improve vehicle driving safety.
[0003] At present, after the assisted driving system is turned on, the assisted driving system usually gives prompts based on the environmental data around the vehicle obtained by the perception system, but the road types are complex, making it difficult to perform reasonable and effective assisted driving for specific road types. Summary of the invention
[0004] When the existing assisted driving system is in operation, it usually provides periodic prompts based on the environmental data collected by the environmental perception system. It is difficult to provide reasonable and effective assisted driving for specific road types, resulting in a poor user experience.
[0005] In order to solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide a driving assistance method, device, electronic device and storage medium.
[0006] According to a first aspect of an embodiment of the present disclosure, there is provided a driving assistance method, comprising:
[0007] Determining the type of road on which the vehicle is currently traveling based on the current position of the vehicle;
[0008] Determining lane line information of a lane in which the vehicle is currently located based on a lane line image collected by a visual perception system of the vehicle;
[0009] Determining a current driving state of the driver of the vehicle based on an image of the driver captured by an image capture device inside the vehicle;
[0010] Determining an assisted driving mode based on the road type, the lane line information and the driving state, wherein the assisted driving mode includes a preset reminder mode and / or a takeover mode;
[0011] Assisted driving is performed based on the assisted driving mode.
[0012] According to a second aspect of an embodiment of the present disclosure, there is provided a driving assistance device, comprising:
[0013] A road type determination module, used to determine the road type of the current driving section of the vehicle based on the current position of the vehicle;
[0014] A lane line information determination module, used to determine the lane line information of the lane where the vehicle is currently located based on the lane line image collected by the visual perception system of the vehicle;
[0015] A driving state determination module, configured to determine a current driving state of the driver of the vehicle based on an image of the driver acquired by an image acquisition device inside the vehicle;
[0016] an assisted driving module determination module, configured to determine an assisted driving mode based on the road type, the lane line information and the driving state, wherein the assisted driving mode includes a preset reminder mode and a takeover mode;
[0017] The assisted driving module is used to perform assisted driving based on the assisted driving mode.
[0018] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the assisted driving method described in the first aspect above.
[0019] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:
[0020] processor;
[0021] a memory for storing instructions executable by the processor;
[0022] The processor is used to read the executable instructions from the memory and execute the instructions to implement the assisted driving method described in the first aspect above.
[0023] Based on the assisted driving method, device, electronic device and storage medium provided by the above embodiments of the present disclosure, the road type of the current driving section of the vehicle is determined based on the current position of the vehicle, and the lane line information of the lane where the vehicle is currently located is determined based on the lane line image collected by the visual perception system of the vehicle, and the current driving state of the driver is determined based on the driver image inside the vehicle, and then based on the road type, lane line information and driving state, the assisted driving mode that meets the current driving needs of the vehicle can be reasonably determined, and assisted driving can be performed according to the assisted driving mode. This method can not only improve the safety of vehicle driving and meet the driver's needs for assisted driving functions, but also avoid frequent unnecessary reminders to the driver, which is conducive to improving user experience.
[0024] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.
[0026] Figure 1 is a flowchart of an assisted driving method in one embodiment of the present disclosure;
[0027] Figure 2 is a flow chart of step S4 in one embodiment of the present disclosure;
[0028] Figure 3 is a flow chart of step S5 in one embodiment of the present disclosure;
[0029] Figure 4 is a flow chart of step S1 in one embodiment of the present disclosure;
[0030] Figure 5 is a flow chart of step S3 in one embodiment of the present disclosure;
[0031] Figure 6 is a flow chart of step S3 in another embodiment of the present disclosure;
[0032] Figure 7 is a structural block diagram of a driving assistance device in one embodiment of the present disclosure;
[0033] Figure 8 is a structural block diagram of an auxiliary driving module 500 in one embodiment of the present disclosure;
[0034] Fig. 9 is a structural block diagram of a road type determination module 100 in one embodiment of the present disclosure;
[0035] Fig.10 is a structural block diagram of a driving state determination module 300 in one embodiment of the present disclosure;
[0036] Fig.11 is a structural block diagram of a driving state determination module 300 in another embodiment of the present disclosure;
[0037] Fig.12 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0038] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.
[0039] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
[0040] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.
[0041] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
[0042] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.
[0043] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.
[0044] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.
[0045] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0046] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0047] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0048] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.
[0049] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.
[0050] Exemplary Systems
[0051] Vehicles with assisted driving functions are provided with an environmental perception system and an in-vehicle image acquisition device. The environmental perception system may include a radar perception system, a visual perception system, and a satellite positioning system. The in-vehicle image acquisition device may include a camera provided in the vehicle, which acquires the driver's image through the camera. The vehicle control system is respectively connected to the environmental perception system and the in-vehicle image acquisition device, and the vehicle position and lane line information acquired by the environmental perception system are combined with the driver's image acquired by the in-vehicle image acquisition device for comprehensive analysis, so that the assisted driving mode adapted to the current road environment can be reasonably determined.
[0052] Exemplary Methods
[0053] Figure 1 is a flow chart of an assisted driving method in one embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the following steps are included:
[0054] S1: Based on the current position of the vehicle, determine the road type of the current driving section of the vehicle.
[0055] The current position of the vehicle is obtained through the positioning system. The positioning system may include a vehicle-mounted positioning system, such as a vehicle-mounted global positioning system (GPS) or a vehicle-mounted Beidou system, and the current position of the vehicle can be directly obtained through the vehicle-mounted positioning system. The positioning system may also include a positioning system located on a mobile terminal in the vehicle, such as a GPS of a user's mobile phone, and the current position of the mobile terminal can be obtained through the positioning system located on the mobile terminal in the vehicle, and the current position of the mobile terminal is used as the current position of the vehicle.
[0056] By matching the road positions of each road pre-stored in the electronic map with the current position of the vehicle, the current driving section of the vehicle can be determined. For example, the current driving section of the vehicle can be determined based on navigation software.
[0057] When the road type of the current driving section is pre-stored in the electronic map, the road type of the current driving section can be obtained through the electronic map; when the road type of the current driving section is not stored in the electronic map, the road type of the current driving section can be obtained from a server that has pre-stored road types of the current driving section through the network.
[0058] S2: Based on the lane line image collected by the vehicle's visual perception system, determine the lane line information of the lane the vehicle is currently in.
[0059] The first image block where the first lane line of the current lane (taking a left-hand drive vehicle as an example, the first side may be the lane line on the left side of the cab) is located is extracted from the lane line image. By analyzing the first image block, the lane line parameters of the first lane line of the current lane can be obtained, and the confidence of the first lane line of the current lane of the vehicle can also be obtained.
[0060] A second image block where the second side lane line of the current lane (taking a left-hand drive vehicle as an example, the first side may be the lane line on the right side of the cab) is located is extracted from the lane line image. By analyzing the second image block, lane line parameters of the second side lane line of the current lane can be obtained, and the confidence level of the second side lane line of the current lane of the vehicle can also be obtained.
[0061] S3: Determine the current driving state of the driver of the vehicle based on the driver image captured by the image capture device inside the vehicle.
[0062] The driver's current driving state can be identified by performing image recognition on the driver image using a pre-trained driving state recognition model. The driving state recognition model is obtained by model training based on sample driver images and driving state labels of the sample driver images. Based on the driver's driving state, it can be determined whether the driver is in a fatigued state, that is, the driving state can include a fatigued driving state and a non-fatigued driving state.
[0063] S4: Determine an assisted driving mode based on the road type, lane line information and driving status, wherein the assisted driving mode includes a preset reminder mode and / or a takeover mode.
[0064] When the current vehicle condition is determined to be a safe condition based on the road type, lane line information and driving status, the assisted driving mode can be determined as a preset reminder mode; when the current vehicle condition is determined to be a dangerous condition based on the road type, lane line information and driving status, the assisted driving mode can be determined as a takeover mode.
[0065] The preset reminder mode may include a periodic reminder mode, and may also include a mode for reminding for special road conditions, wherein the special road conditions may include road congestion and bad weather conditions, and the like.
[0066] In the event of an emergency or dangerous situation determined based on the road type, lane line information and driving status (such as an impending traffic accident such as a collision), the system will directly switch to takeover mode. In the event of a non-emergency or dangerous situation determined based on the road type, lane line information and driving status (such as driving beyond the safe speed threshold in bad weather), the driver can be reminded and the system will switch to takeover mode after the driver confirms.
[0067] S5: Assisted driving based on assisted driving mode.
[0068] It should be noted that this embodiment does not limit the order of execution of step S1, step S2 and step S3. You can first select one step from step S1, step S2 and step S3 to execute, then select one step from the remaining two steps to execute, and finally execute only the remaining one step; you can also first select one step from step S1, step S2 and step S3 to execute, and then execute the remaining two steps at the same time; you can also first execute two steps from step S1, step S2 and step S3 at the same time, and then execute the remaining one step; you can also execute step S1, step S2 and step S3 at the same time.
[0069] In this embodiment, the type of road on which the vehicle is currently traveling is determined based on the vehicle's current position, and the lane line information of the lane in which the vehicle is currently located is determined based on the lane line image collected by the vehicle's visual perception system, and the driver's current driving status is determined based on the driver image inside the vehicle. Based on the road type, lane line information and driving status, an assisted driving mode that meets the current driving needs of the vehicle can be reasonably determined. Assisted driving according to the assisted driving mode can not only improve the safety of vehicle driving and meet the driver's needs for assisted driving functions, but also avoid frequent unnecessary reminders to the driver, thereby improving user experience.
[0070] Figure 2 FIG. 4 is a flow chart of step S4 in one embodiment of the present disclosure. Figure 2 As shown, step S4 includes:
[0071] S4-1: If the road type is a highway type or an elevated bridge type, the lane line information meets the preset lane line conditions, and the driving state is a fatigue driving state, the assisted driving mode is determined to be the takeover mode.
[0072] Since the vehicle is usually traveling at a stable and high speed on a highway or viaduct, it is easy to be dangerous when the driver is in a fatigued driving state. At this time, if the lane lines on both sides of the lane where the vehicle is currently located meet the preset lane line conditions, the auxiliary driving mode is determined to be the takeover mode. The takeover control of the vehicle through the takeover mode can improve the driving safety of the vehicle. Among them, the preset lane line condition can, for example, detect whether the value of the lane line detection length of the lane lines on both sides of the lane where the vehicle is currently traveling and the value of the current vehicle speed meet the first preset condition. Assuming that the lane line detection length of the lane lines on both sides of the lane where the vehicle is currently traveling is L, and the current vehicle speed is V, when L>KV, it can be considered that the first preset condition is met, where K is a real number greater than 0. In addition, the preset lane line condition can also detect whether the confidence of the lane lines on both sides of the lane where the vehicle is currently traveling meets the second preset condition. Assuming that the confidence of the lane lines on both sides of the lane where the vehicle is currently traveling is P, when P is greater than Pt, it can be considered that the second preset condition is met, where Pt is a preset value between (0, 1).
[0073] S4-2: If the driving state is a fatigue driving state and the road type is not a highway type or an overpass type, or if the driving state is a fatigue driving state and the lane line information does not meet the preset lane line conditions, the assisted driving mode is determined to be the preset reminder mode.
[0074] Since the speed of vehicles is usually unstable and slow when traveling on non-highways and non-viaducts, the driver has more reaction time when an emergency occurs compared to when the vehicle is traveling on highways and viaducts. Therefore, a preset reminder mode can be used to remind the driver when the driver is fatigued.
[0075] Since the lane line information does not meet the preset lane line conditions, the assisted driving system may find it difficult to perform lane centering control based on the vehicle and the lane lines on the left and right sides of the current lane. Therefore, a preset reminder mode can be used to remind the driver.
[0076] It should be noted that this embodiment does not limit the order of execution of step S4-1 and step S4-2. Step S4-1 may be executed first, and then step S4-2; step S4-2 may be executed first, and then step S4-1; or step S4-1 and step S4-2 may be executed simultaneously.
[0077] In this embodiment, an assisted driving mode that meets the current vehicle conditions can be reasonably determined based on the road type, lane line information of the vehicle's current lane, and driving status, so that assisted driving can be performed based on the determined assisted driving mode, providing a good user experience.
[0078] Figure 3 FIG. 5 is a flow chart of step S5 in one embodiment of the present disclosure. Figure 3 As shown, step S5 includes:
[0079] S5-1: If it is determined that the assisted driving mode is the takeover mode, lane centering control is performed based on the vehicle's position in the current driving lane and the centerline position of the current driving lane, the lateral control takeover torque limit value is adjusted, and the hands-off alarm suppression logic is turned off.
[0080] In the takeover mode, in order to ensure the driving safety of the vehicle, the vehicle can be controlled by at least one control method.
[0081] The first control method performs lane centering control based on the vehicle's position in the current lane and the centerline position of the current lane to avoid sudden lane changes that may cause the vehicle to collide with vehicles in other lanes, and to avoid the vehicle colliding with the isolation belt or curb on the side of the road, thereby improving vehicle driving safety.
[0082] The second control method, by adjusting the lateral control takeover torque limit value, can reduce the lateral change in the vehicle's driving direction caused by the driver's misoperation of the steering wheel under fatigue driving conditions, thereby improving vehicle driving safety.
[0083] The third control method is that since the driver turns off the alarm by shaking the steering wheel due to the hands-off alarm suppression logic, by turning off the hands-off alarm suppression logic, the driver can be continuously alarmed, for example, by waking up the driver through continuous sound and light alarms, which makes it easier for the driver to control the vehicle in time and improve vehicle driving safety.
[0084] S5-2: Control the vehicle deceleration by continuous braking.
[0085] On the basis of controlling the vehicle through the above three control methods, the vehicle deceleration can be controlled by continuous braking to further improve the vehicle driving safety.
[0086] In this embodiment, in the takeover mode, the vehicle's driving safety can be greatly improved by controlling lane centering, adjusting the lateral control takeover torque limit value, turning off the hands-off alarm suppression logic, and controlling vehicle deceleration by continuous braking.
[0087] Figure 4 FIG. 1 is a flow chart of step S1 in one embodiment of the present disclosure. Figure 4 As shown, step S1 includes:
[0088] S1-1: Obtain the current position of the vehicle, the average speed and average acceleration of the vehicle within a preset first time window.
[0089] The current position of the vehicle is obtained through the positioning device.
[0090] In an example of the present disclosure, a preset time period can be pushed forward from the current time as the start time of the preset first time window, the current time can be used as the end time of the preset first time window, and the continuous time between the start time and the end time can be used as the preset first time window. A , the time when the current time is pushed forward by the preset time period is T B , then the starting time of the first time window is preset to be T B , the end time of the first time window is preset to be T A , the first time window is preset as [T B ,T A ], the length of the preset time period is T A -T B The average speed and average acceleration of the vehicle in the preset first time window are collected through the on-board central control system.
[0091] S1-2: Based on the current position of the vehicle, determine the road type set to which the vehicle is currently traveling.
[0092] When a vehicle is traveling on a road section whose projections overlap in the vertical direction, such as a viaduct with a road below the vehicle, or a vehicle is traveling on a road below an viaduct above, or a vehicle is traveling on an overpass, the current road section traveled by the vehicle corresponds to at least one road type, and a set of road types to which the current road section traveled by the vehicle belongs is generated based on the at least one road type.
[0093] S1-3: If the road type set to which the vehicle is currently traveling includes a highway type or an elevated bridge type, the average speed is greater than a preset speed threshold and the average acceleration is less than a preset acceleration threshold, the road type is determined to be a highway type or an elevated bridge type.
[0094] When a vehicle is traveling on a highway or an overpass, the driving speed is usually stable and high; when a vehicle is traveling on a road under an overpass, the driving speed is usually unstable and low due to the presence of traffic lights on the road.
[0095] Based on this, when the road type set to which the vehicle is traveling includes a highway type or an viaduct type, and the average speed within the preset first time window is greater than the preset vehicle speed threshold and the average acceleration is less than the preset acceleration threshold, it can be determined that the vehicle is currently traveling on a highway or an viaduct.
[0096] In this embodiment, the type of the road on which the vehicle is currently traveling can be accurately determined by combining the vehicle positioning with the average speed and average acceleration of the vehicle within a preset first time window.
[0097] In one embodiment of the present disclosure, step S2 may include: determining the lane line confidence and lane line detection length of the lane the vehicle is currently in based on the lane line image. Accordingly, in step S4-1 and step S4-2, the preset lane line conditions include: the value of the lane line detection length is greater than the value of the product of the vehicle speed and the preset vehicle speed coefficient, and the lane line confidence is greater than the preset lane line confidence threshold.
[0098] Since the lane line image only includes the partial imaging of the lane lines on both sides of the current lane, the detected length of the lane line in the actual lane can be obtained based on the image length of the lane line in the lane line image. The lane line detection module can be trained based on the lane line sample image to train a lane line detection model. The lane line detection module can detect the lane line confidence and lane line detection length of the lane where the vehicle is currently located.
[0099] For example, the preset speed coefficient product may be 1.5, and the preset lane line confidence threshold may be 90%. In this example, if the length of the lane lines on both sides of the lane at the target time is greater than the speed at the target time*1.5, and the lane line confidence at the target time is greater than 90%, it can be determined that the lane line is clear and the lane line is long, and the centering control margin is more sufficient, thereby satisfying the lane centering control condition.
[0100] In this embodiment, the lane line image is recognized to determine the lane line confidence and lane line detection length of the lane the vehicle is currently in. Based on the lane line confidence and lane line detection length of the lane the vehicle is currently in, it can be effectively judged whether the lane line at the current moment meets the lane centering control conditions.
[0101] In one embodiment of the present disclosure, step S2 may further include: determining the number of lane line curvature mutations of the lane currently located by the vehicle within a preset second time window based on the lane line image collected by the visual perception system. Accordingly, in step S4-1 and step S4-2, the preset lane line condition also includes: the number of lane line curvature mutations is less than a preset lane line curvature mutation threshold.
[0102] In one example of the present disclosure, the current time is T C , the time when the current time is pushed forward by the preset time period is T D , then the starting time of the second time window is preset to be T D , the end time of the second time window is preset as T C , the second time window is preset as [T D ,T C ], the length of the preset time period is T C -T D .
[0103] When there are repeated or intersecting lane lines on the road surface, it will cause a sudden change in curvature. By recognizing the lane line image, the number of sudden changes in lane curvature in the vehicle's current lane within the preset second time window can be obtained. Sudden changes in curvature will cause more drastic lane centering control, posing a safety risk.
[0104] In this embodiment, the lane line image is identified to determine the number of lane line curvature mutations in the lane where the vehicle is currently located within the preset second time window. Lane centering control is performed only when the number of lane line curvature mutations in the lane where the vehicle is currently located within the preset second time window is less than a preset lane line curvature mutation threshold, which can improve the safety of lane centering control.
[0105] Figure 5 FIG. 1 is a flow chart of step S3 in one embodiment of the present disclosure. Figure 5 As shown, step S3 includes:
[0106] S3-A-1: Obtain the lateral displacement of the vehicle within a preset third time window.
[0107] In one example of the present disclosure, a preset time period can be pushed forward from the current time as the start time of the preset third time window, the current time is used as the end time of the preset second time window, and the continuous time between the start time and the end time is used as the preset second time window. For example, the current time is T E , the time when the current time is pushed forward by the preset time period is T F , then the starting time of the third time window is preset to be T F , the end time of the second time window is preset as T E , the third time window is preset as [T F ,T E ], the length of the preset time period is T E -T F The wheel speed can be obtained through the sensor, and the wheel speed can be converted to the vehicle speed. The steering wheel torque can be obtained through the vehicle central control system. The lateral displacement of the vehicle within the preset third time window is determined based on the lateral vehicle speed and the steering wheel torque.
[0108] S3-A-2: Based on the driver image captured by the image acquisition device, determine whether the driver is currently in a fatigue state.
[0109] By training a fatigue state recognition model based on sample images containing the driver, and processing the real-time acquired images containing the current driver based on the trained fatigue state recognition model, it can be determined whether the driver is currently in a fatigue state.
[0110] S3-A-3: If the judgment result is that the driver is currently in a fatigue state and the lateral displacement is not within the preset first lateral displacement range, the driving state is determined to be a fatigue driving state. The preset first lateral displacement range is determined according to the length of the preset third time window. For example, when the lateral displacement of the vehicle exceeds 5 meters within 3 seconds, it can be determined that the driver has misoperated the steering wheel.
[0111] In this embodiment, since the fatigue state recognition model mainly recognizes images containing the driver's eye area, it determines whether the driver is in a fatigue state based on the driver's eye opening degree. However, some drivers' eye features are not obvious. For example, even if the eyes are in a normal open state, the eye opening degree is still not large, which is easy to be misidentified as a fatigue state by the fatigue state recognition model. Therefore, on the basis of recognizing the driving state of the driver's image, combined with the lateral displacement of the vehicle within the preset third time window, the driving state can be accurately determined.
[0112] Figure 6FIG. 1 is a flow chart of step S3 in another embodiment of the present disclosure. Figure 6 As shown, step S3 includes:
[0113] S3-B-1: Obtaining the working state of the lane centering control function and the lateral displacement of the vehicle within a preset fourth time window. The working state of the lane centering control function includes the lane centering control function being in an activated state and the lane centering control function being in an inactivated state.
[0114] S3-B-2: Based on the driver image captured by the image acquisition device, determine whether the driver is currently in a fatigue state.
[0115] S3-B-3: If it is determined that the driver is currently in a fatigued state, the lane centering control function is in an inactive state, and the lateral displacement is not within the preset second lateral displacement range, the driving state is determined to be a fatigued driving state. The preset second lateral displacement range is determined according to the length of the preset fourth time window. For example, when the lateral displacement of the vehicle exceeds 5 meters within 3 seconds, it can be considered that the driver has misoperated the steering wheel.
[0116] In this embodiment, since the fatigue state recognition model mainly recognizes images containing the driver's eye area, it determines whether the driver is in a fatigue state based on the driver's eye opening degree. However, some drivers' eye features are not obvious. For example, even if the eyes are in a normal open state, the eye opening degree is still not large, which is easy to be misidentified as a fatigue state by the fatigue state recognition model. Therefore, on the basis of recognizing the driving state of the driver's image, combined with the lateral displacement of the vehicle within the preset third time window, the driving state can be accurately determined.
[0117] Any assisted driving method provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any assisted driving method provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any assisted driving method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.
[0118] Exemplary Devices
[0119] Figure 7 : is a structural block diagram of a driving assistance device in one embodiment of the present disclosure. Figure 7 As shown, the auxiliary driving device includes:
[0120] A road type determination module 100, for determining the road type of the current driving section of the vehicle based on the current position of the vehicle;
[0121] A lane line information determination module 200, configured to determine the lane line information of the lane in which the vehicle is currently located based on the lane line image collected by the visual perception system of the vehicle;
[0122] A driving state determination module 300, configured to determine a current driving state of the driver of the vehicle based on an image of the driver captured by an image capture device inside the vehicle;
[0123] An assisted driving mode determination module 400, configured to determine an assisted driving mode based on the road type, the lane line information and the driving state, wherein the assisted driving mode includes a preset reminder mode and a takeover mode;
[0124] The assisted driving module 500 is used to perform assisted driving based on the assisted driving mode.
[0125] In one embodiment of the present disclosure, the assisted driving mode determination module 400 is used to determine that the assisted driving mode is the takeover mode if the road type is a highway type or an overpass type, the lane line information satisfies the preset lane line condition, and the driving state is a fatigue driving state; the assisted driving module determination module 400 is also used to determine that the assisted driving mode is the preset reminder mode if the driving state is a fatigue driving state and the road type does not belong to the highway type and the overpass type, or the driving state is a fatigue driving state and the lane line information does not meet the preset lane line condition.
[0126] Figure 8 FIG. 5 is a structural block diagram of an auxiliary driving module 500 in one embodiment of the present disclosure. Figure 8 As shown, the auxiliary driving module 500 includes:
[0127] A first auxiliary driving unit 510 is configured to, if it is determined that the auxiliary driving mode is the takeover mode, perform lane centering control based on the position of the vehicle in the current driving lane and the centerline position of the current driving lane, adjust the lateral control takeover torque limit value, and turn off the hands-off alarm suppression logic;
[0128] The second auxiliary driving unit 520 is used to control the vehicle to decelerate by continuous braking.
[0129] Fig. 9 FIG. 1 is a structural block diagram of a road type determination module 100 in one embodiment of the present disclosure. Fig. 9 As shown, the road type determination module 100 includes:
[0130] An acquisition unit 110 is used to acquire the current position of the vehicle, the average speed and the average acceleration of the vehicle within a preset first time window;
[0131] A road type set determining unit 120, configured to determine, based on the current position of the vehicle, a road type set to which the vehicle is currently traveling;
[0132] The road type determination unit 130 is used to determine that the road type is a highway type or a viaduct type if the road type set to which the vehicle is currently traveling includes a highway type or an viaduct type, and the average speed is greater than a preset vehicle speed threshold and the average acceleration is less than a preset acceleration threshold.
[0133] In one embodiment of the present disclosure, the assisted driving mode determination module 400 is used to determine the lane line confidence and lane line detection length of the lane in which the vehicle is currently located based on the lane line image; wherein the preset lane line condition includes: the value of the lane line detection length is greater than the value of the product of the vehicle speed and the preset vehicle speed coefficient, and the lane line confidence is greater than the preset lane line confidence threshold.
[0134] In one embodiment of the present disclosure, the assisted driving mode determination module 400 is also used to determine the number of lane line curvature mutations in the lane where the vehicle is currently located within a preset second time window based on the lane line image collected by the visual perception system; wherein the preset lane line condition also includes: the number of lane line curvature mutations is less than a preset lane line curvature mutation threshold.
[0135] Fig.10 FIG. 3 is a structural block diagram of a driving state determination module 300 in one embodiment of the present disclosure. Fig.10 As shown, the driving state determination module 300 includes:
[0136] A lateral displacement determination unit 310, configured to obtain a lateral displacement of the vehicle within a preset third time window;
[0137] A first judging unit 320, configured to judge whether the driver is currently in a fatigue state based on the driver image captured by the image capturing device;
[0138] The first driving state determining unit 330 is configured to determine that the driving state is a fatigue driving state if it is determined that the driver is currently in a fatigue state and the lateral displacement is not within a first preset lateral displacement range.
[0139] Fig.11 FIG. 3 is a structural block diagram of a driving state determination module 300 in another embodiment of the present disclosure. Fig.10 As shown, the driving state determination module 300 includes:
[0140] An acquisition unit 340 is used to acquire a working state of a lane centering control function and a lateral displacement of the vehicle within a preset fourth time window;
[0141] A second judgment unit 350 is used to judge whether the driver is currently in a fatigue state based on the driver image collected by the image acquisition device;
[0142] The second driving state determination unit 360 is used to determine that the driving state is a fatigue driving state if it is determined that the driver is currently in a fatigue state, the working state of the lane centering control function is in an inactive state, and the lateral displacement is not within a second preset lateral displacement range.
[0143] It should be noted that the specific implementation of the assisted driving device in the embodiment of the present disclosure is similar to the specific implementation of the assisted driving method in the embodiment of the present disclosure. Please refer to the assisted driving method part for details. In order to reduce redundancy, it will not be repeated here.
[0144] Exemplary Electronic Devices
[0145] Below, reference Fig.12 To describe the electronic device according to the embodiment of the present disclosure. Fig.12 As shown, the electronic device includes one or more processors 10 and a memory 20 .
[0146] The processor 10 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
[0147] The memory 20 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the assisted driving method of each embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0148] In one example, the electronic device 10 may further include: an input device 30 and an output device 40, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 30 may be, for example, a keyboard, a mouse, etc. The output device 40 may include, for example, a display, a speaker, a printer, and a communication network and a remote output device connected thereto, etc.
[0149] Of course, to simplify, Fig.12 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.
[0150] Exemplary computer-readable storage media
[0151] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0152] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0153] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0154] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
[0155] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
[0156] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0157] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0158] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A driving assistance method, include: Determining the type of road on which the vehicle is currently traveling based on the current position of the vehicle; Determining lane line information of a lane in which the vehicle is currently located based on a lane line image collected by a visual perception system of the vehicle; Determining a current driving state of the driver of the vehicle based on an image of the driver captured by an image capture device inside the vehicle; Determining an assisted driving mode based on the road type, the lane line information and the driving state, wherein the assisted driving mode includes a preset reminder mode and / or a takeover mode; Performing assisted driving based on the assisted driving mode; Among them, the assisted driving mode is determined based on the road type, the lane line information and the driving state, including: if the road type is a highway type or an overpass type, the lane line information satisfies the preset lane line condition, and the driving state is a fatigue driving state, the assisted driving mode is determined to be the takeover mode, and the preset lane line condition includes: the value of the lane line detection length of the lane lines on both sides of the vehicle's current driving lane and the value of the current vehicle speed meet the first preset condition, and the confidence of the lane lines on both sides of the vehicle's current driving lane meets the second preset condition.
2. The assisted driving method according to claim 1, in, The determining of the assisted driving mode based on the road type, the lane line information and the driving state further includes: If the driving state is a fatigue driving state and the road type does not belong to a highway type and an viaduct type, or the driving state is a fatigue driving state and the lane line information does not meet the preset lane line condition, the assisted driving mode is determined to be the preset reminder mode.
3. The assisted driving method according to claim 2, in, The performing assisted driving based on the assisted driving mode includes: If it is determined that the assisted driving mode is the takeover mode, lane centering control is performed based on the position of the vehicle in the current driving lane and the centerline position of the current driving lane, the lateral control takeover torque limit value is adjusted, and the hands-off alarm suppression logic is turned off; The vehicle is controlled to decelerate in a continuous braking manner.
4. The assisted driving method according to claim 1 or 2, in, The determining, based on the current position of the vehicle, the road type of the current driving section of the vehicle comprises: Obtaining the current position of the vehicle, the average speed and average acceleration of the vehicle within a preset first time window; Based on the current position of the vehicle, determining a set of road types to which the current driving section of the vehicle belongs; If the road type set to which the vehicle is currently traveling includes a highway type or an elevated bridge type, the average speed is greater than a preset vehicle speed threshold and the average acceleration is less than a preset acceleration threshold, the road type is determined to be a highway type or an elevated bridge type.
5. The driving assistance method according to claim 2, in, The determining, based on the lane line image collected by the visual perception system of the vehicle, lane line information of the lane where the vehicle is currently located includes: Based on the lane line image, determining a lane line confidence and a lane line detection length of a lane where the vehicle is currently located; Among them, the preset lane line conditions include: the value of the lane line detection length is greater than the value of the product of the vehicle speed and the preset vehicle speed coefficient, and the lane line confidence is greater than the preset lane line confidence threshold.
6. The assisted driving method according to claim 5, in, The determining the lane line information of the lane where the vehicle is currently located based on the lane line image collected by the visual perception system of the vehicle further includes: Based on the lane line image collected by the visual perception system, determining the number of sudden changes in lane line curvature of the lane where the vehicle is currently located within a preset second time window; Among them, the preset lane line condition also includes: the number of lane line curvature mutations is less than a preset lane line curvature mutation threshold.
7. The assisted driving method according to claim 1 or 2, in, The determining the current driving state of the driver of the vehicle based on the driver image acquired by the image acquisition device inside the vehicle includes: Acquiring a lateral displacement of the vehicle within a preset third time window; Based on the driver image acquired by the image acquisition device, determining whether the driver is currently in a fatigue state; If it is determined that the driver is currently in a fatigue state and the lateral displacement is not within a first preset lateral displacement range, the driving state is determined to be a fatigue driving state.
8. The assisted driving method according to claim 1 or 2, in, The determining the current driving state of the driver of the vehicle based on the driver image acquired by the image acquisition device inside the vehicle includes: Acquiring a working state of a lane centering control function and a lateral displacement of the vehicle within a preset fourth time window; Based on the driver image acquired by the image acquisition device, determining whether the driver is currently in a fatigue state; If it is determined that the driver is currently in a fatigue state, the working state of the lane centering control function is in an inactivated state, and the lateral displacement is not within a second preset lateral displacement range, it is determined that the driving state is a fatigue driving state.
9. A driving assistance device, include: A road type determination module, used to determine the road type of the current driving section of the vehicle based on the current position of the vehicle; A lane line information determination module, used to determine the lane line information of the lane where the vehicle is currently located based on the lane line image collected by the visual perception system of the vehicle; A driving state determination module, configured to determine a current driving state of the driver of the vehicle based on an image of the driver acquired by an image acquisition device inside the vehicle; an assisted driving mode determination module, configured to determine an assisted driving mode based on the road type, the lane line information and the driving state, wherein the assisted driving mode includes a preset reminder mode and a takeover mode; An assisted driving module, used for performing assisted driving based on the assisted driving mode; Among them, the assisted driving module is used to determine that the assisted driving mode is the takeover mode if the road type is a highway type or an overpass type, the lane line information satisfies the preset lane line conditions, and the driving state is a fatigue driving state, and the preset lane line conditions include: the value of the lane line detection length of the lane lines on both sides of the vehicle's current lane and the value of the current vehicle speed satisfy the first preset condition, and the confidence of the lane lines on both sides of the vehicle's current lane satisfies the second preset condition.
10. A computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the assisted driving method described in any one of claims 1 to 8.
11. An electronic device, the electronic device include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the assisted driving method described in any one of claims 1-8 above.
