A lane departure warning control method, device, equipment and storage medium

Abstract

This application discloses a lane departure warning control method, device, equipment, and storage medium. The method includes: when a target commercial vehicle is detected to be deviating from its lane, acquiring first information related to vehicle movement, and determining a lane departure index, a fatigue index, and an environmental index based on the first information; calculating a comprehensive index affecting the lane departure based on weighting factors assigned to each index; selecting a warning strategy from a matching warning level based on the comprehensive index; and controlling the vibration of a corresponding area of ​​the target commercial vehicle's steering wheel to alert the driver of the lane departure. In this way, by matching a comprehensive index determined by multiple indices to a warning level consistent with the current lane departure situation, and controlling the target commercial vehicle's steering wheel to vibrate at different frequencies in different zones according to the warning strategy within the warning level, the driver can promptly, intuitively, and accurately identify lane departure. This diversified warning method enhances the driving experience.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, specifically to the fields of assisted driving and vehicle warning, and particularly to a lane departure warning control method, device, equipment and storage medium. Background Technology

[0002] Traditional lane departure warnings rely on visual (e.g., instrument panel flashing) or auditory (e.g., beeping) alerts, which place certain demands on the driver's driving skills and the driving environment. Drivers may not be able to notice lane departure warnings in time if they are focused on the road ahead and cannot see the instrument panel flashing, or if the environment inside or outside the vehicle is noisy.

[0003] Although existing lane departure detection systems can be identified by cameras and alerted via unidirectional vibration, the accuracy is poor, the warning methods are limited and cannot be differentiated based on the degree of deviation, resulting in a poor driving experience. Summary of the Invention

[0004] This application provides a lane departure warning method, device, equipment, and storage medium to solve the problems of poor recognition accuracy, single warning prompt method, inability to provide differentiated prompts based on the degree of deviation, and poor driving experience.

[0005] The technical solution is as follows: Firstly, a lane departure warning and control method is provided, including: When lane departure of a target commercial vehicle is detected, first information related to vehicle driving in the current driving scenario is obtained. The first information includes at least: road information, vehicle driving information, driver vital signs information and image information, and environmental information. The lane departure index is determined based on the road information and the vehicle driving information. Based on the driver's vital signs and image information, the driver's fatigue index at the time of this lane departure is determined; The environmental index at the time of this lane departure is determined based on the environmental information. Based on the deviation index, the fatigue index, the environmental index, and the weighting factors assigned to each index, calculate the comprehensive index affecting this lane departure. Based on the comprehensive index, a warning level matching the current lane departure is selected, and based on the warning strategy in the selected warning level, the corresponding area of ​​the steering wheel of the target commercial vehicle is controlled to vibrate to alert the driver of lane departure.

[0006] In one possible implementation, the lane departure index is determined based on the road information and the vehicle driving information, specifically including: Based on the road information and the vehicle driving information, the lateral displacement during a set time period in which lane departure occurs is determined, and the lateral departure speed is calculated based on the set time period and the lateral displacement. The absolute value of the ratio of the lateral deviation speed to the set baseline deviation speed is used as the deviation index for this lane departure.

[0007] In one possible implementation, the driver's fatigue index at the time of the lane departure is determined based on the driver's vital signs information and image information, specifically including: The driver's fatigue index is determined based on the driver's physical signs information; and the driver's state index is determined based on the driver's image information. If any index exceeds the set threshold, the fatigue index or state index with the largest value among the indices that exceed the set threshold shall be used as the driver's fatigue index during this lane departure. Otherwise, the fatigue index and the state index are weighted and averaged, and the average result is used as the driver's fatigue index during this lane departure.

[0008] In one possible implementation, the driver's vital signs information is heart rate information; then, determining the driver's fatigue index based on the driver's vital signs information specifically includes: The driver's heart rate information is obtained by the pulse wave sensor integrated in the steering wheel, and a first fatigue index and a second fatigue index are extracted for the driver. The first fatigue index is the standard deviation of the heart rate RR interval, which is used to characterize the overall fatigue level. The second fatigue index is the root mean square of the continuous heart rate RR difference, which is used to characterize the acute fatigue level. The driver's fatigue index is calculated based on the following formula: in, It is the number one fatigue index. It is the second fatigue index. , It is the weight of the first fatigue index. It is the weight of the second fatigue index.

[0009] In one possible implementation, determining the environmental index at the time of the lane departure based on the environmental information specifically includes: Based on the environmental information obtained from millimeter-wave radar and / or cameras, the visibility, crosswind intensity, and traffic flow in adjacent lanes in the current environment are extracted. Calculate the environmental index for this lane departure using the following formula: in, It's visibility. It is the crosswind intensity. It is the traffic flow in the adjacent lane; It is the furthest distance at which visibility can be monitored. It is the critical wind speed that can maintain vehicle safety. It is the critical traffic volume threshold for avoiding collision risks.

[0010] In one possible implementation, after acquiring the first information related to the target commercial vehicle's driving in the current driving scenario, and before determining the lane departure index based on the road information and the vehicle driving information, the method further includes: Based on the road information and vehicle driving information in the first information, the vehicle operating condition of the target commercial vehicle is determined; According to the vehicle operating condition, a matching weight item is searched from a preset weight database. The preset weight database stores a variety of vehicle operating conditions, each of which is mapped to a weight item. Each weight item includes a weight factor assigned to the deviation index, a weight factor assigned to the fatigue index, and a weight factor assigned to the environmental index.

[0011] In one possible implementation, the deviation index also carries the deviation direction; then, based on the warning strategy in the selected warning level, the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle is controlled, specifically including: If the selected warning level is primary, then the steering wheel of the target commercial vehicle will be controlled to vibrate at a low frequency on the side corresponding to the deviation direction, and the instrument panel will be controlled to display a warning. If the selected warning level is medium, then control the steering wheel of the target commercial vehicle to perform alternating low-frequency vibration in two zones, control the instrument to display a warning, and control the application of torque opposite to the deviation direction. If the selected warning level is high, the steering wheel of the target commercial vehicle will vibrate at high frequency, the instrument panel will display a warning, and the vehicle will switch to lane keeping mode.

[0012] Secondly, a lane departure warning control device is provided, comprising: The information acquisition module is used to acquire first information related to the vehicle's driving in the current driving scenario when a target commercial vehicle is detected to have deviated from its lane. The first information includes at least: road information, vehicle driving information, driver's vital signs information and image information, and environmental information. The lane departure index determination module is used to determine the lane departure index based on the road information and the vehicle driving information. The fatigue index determination module is used to determine the driver's fatigue index at the time of the lane departure based on the driver's physical signs information and image information. An environmental index determination module is used to determine the environmental index at the time of this lane departure based on the environmental information. The comprehensive index determination module is used to calculate the comprehensive index affecting the current lane departure based on the deviation index, the fatigue index, the environmental index, and the weighting factors assigned to each index. The warning module is used to select a warning level matching the current lane departure based on the comprehensive index, and to control the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle based on the warning strategy in the selected warning level, so as to alert the driver of lane departure.

[0013] Thirdly, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0014] Fourthly, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the aspects described above and any possible implementation thereof.

[0015] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0016] In a sixth aspect, an autonomous vehicle is provided, including the electronic devices described above.

[0017] The beneficial effects of the technical solution provided in this application include at least the following: As can be seen from the above technical solution, when a lane departure is detected in a target commercial vehicle, the embodiments of this application acquire first information related to vehicle driving, and determine the lane departure index, fatigue index, and environmental index based on the first information. Then, according to the weighting factors assigned to each index, a comprehensive index affecting the lane departure is calculated. Subsequently, a warning strategy in the matching warning level is selected based on the comprehensive index, and the corresponding area of ​​the steering wheel of the target commercial vehicle is controlled to vibrate to alert the driver of the lane departure. In this way, the magnitude of the comprehensive index determined by multiple indices affecting lane departure can be matched with a warning level that corresponds to the current lane departure situation and degree. Furthermore, the steering wheel of the target commercial vehicle is controlled to vibrate at different frequencies in different zones according to the warning strategy in the warning level, allowing the driver to intuitively and accurately identify the occurrence of lane departure. Moreover, the warning alert method is diversified, and differentiated alerts can be given according to the degree of lane departure, thereby enabling the driver to be aware of the lane departure situation in a timely manner and improving the driving experience.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the steps of a lane departure warning control method provided in an embodiment of this application.

[0021] Figures 2a-2c These are schematic diagrams illustrating vibration alerts in different areas of the steering wheel, provided in another embodiment of this application.

[0022] Figure 3 This is a structural block diagram of a lane departure warning control device provided in another embodiment of this application.

[0023] Figure 4 This is a block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] Obviously, the described embodiments are only some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0026] It should be noted that the terminal devices involved in the embodiments of this application may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; the display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0027] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0028] Given the poor accuracy, limited warning methods, and inability to differentiate warnings based on the degree of deviation in existing lane departure warning systems, resulting in a subpar driving experience, this application proposes a novel lane departure warning and control scheme. The main inventive concept involves acquiring first information related to vehicle movement when lane departure is detected in a target commercial vehicle. Based on this first information, the deviation index, fatigue index, and environmental index for this lane departure are determined. Then, a comprehensive index affecting this lane departure is calculated based on weighting factors assigned to each index. Subsequently, a warning strategy within a matching warning level is selected based on this comprehensive index, controlling the vibration of the corresponding area of ​​the target commercial vehicle's steering wheel to alert the driver of the lane departure. In this way, the magnitude of the comprehensive index determined by multiple indices affecting lane departure can be matched to a warning level consistent with the current lane departure situation and degree. Furthermore, the steering wheel of the target commercial vehicle vibrates at different frequencies in different zones according to the warning strategy within the warning level, allowing the driver to intuitively and accurately identify the occurrence of lane departure. Moreover, the diverse warning methods, which differentiate warnings based on the degree of lane departure, enable the driver to be promptly informed of the lane departure situation, thus improving the driving experience.

[0029] The lane departure warning control scheme involved in this application will be described in detail below through specific embodiments.

[0030] Reference Figure 1 The diagram shown illustrates the steps of a lane departure warning control method according to an embodiment of this application. It should be understood that the executing entity of this lane departure warning control method can be a lane departure warning control device, which can be an electronic device such as a computer, tablet computer, smart wearable device, or smartphone with data calculation, processing, and storage functions, or a software module integrated into a similar electronic device.

[0031] like Figure 1 As shown, the lane departure warning control method may include the following steps: Step 102: When lane departure of the target commercial vehicle is detected, first information related to vehicle driving in the current driving scenario is obtained. The first information includes at least: road information, vehicle driving information, driver vital signs information and image information, and environmental information.

[0032] Lane departure detection by a target commercial vehicle can be achieved in various ways, which is not the key improvement of this application. The main focus of this application is on how to efficiently, promptly, and specifically warn the driver of lane departure when the target commercial vehicle detects it has departed its lane.

[0033] When the target commercial vehicle detects lane departure, it immediately triggers sensors, either locally or externally, to acquire initial information about the vehicle in the current driving scenario. This initial information includes road information, vehicle driving information, driver vital signs and image information, and environmental information. Road information may include: road name, road width, number of lanes, speed limit, intersections, etc.; this information can be obtained from maps in navigation software and local image acquisition devices (cameras or radar, etc.). Vehicle driving information may include: vehicle speed, acceleration, obstacles, etc.; this information can be obtained from local cameras or millimeter-scale radar sensors. Driver vital signs information may include: driver's heart rate, body temperature, pulse, blood oxygen, etc.; this information can be obtained from locally integrated medical sensors and wearable devices worn by the driver. Driver image information may be images or videos captured by a local camera of the driver operating the target commercial vehicle, such as facial images. Environmental information may include weather conditions and traffic flow at the current driving scenario location, such as wind speed and visibility, reflecting environmental risk conditions; this information can be obtained from weather software and cameras.

[0034] Step 104: Determine the lane departure index based on the road information and the vehicle driving information.

[0035] Optionally, in this application, the deviation index of the current lane departure is determined based on the road information and the vehicle driving information. Specifically, the lateral displacement during a set time period in which the lane departure occurred can be determined based on the driving speed in the road information and the vehicle driving information, and the lateral deviation speed can be calculated based on the set time period and the lateral displacement. The absolute value of the ratio of the lateral deviation speed to the set benchmark deviation speed is used as the deviation index of the current lane departure.

[0036] Specifically, it can be expressed by the following formula: in, The lateral deviation speed, This is a lateral displacement. To set a time period, To set a baseline deviation speed. When the lateral deviation speed is greater than or equal to 0.3 m / s, 90% of drivers will need external intervention to avoid deviation; here, a baseline deviation speed of 10 m / s can be set; this baseline deviation speed setting can be flexibly adjusted according to actual needs, and is only an example here without limitation.

[0037] In this application, in addition to determining the deviation index according to the above method, the deviation speed when lane departure occurs can be calculated based on road information and vehicle driving information, and the lateral deviation speed can be decomposed from the speed. Then, the absolute value of the ratio of the lateral deviation speed to the set benchmark deviation speed is used as the deviation index of this lane departure.

[0038] Step 106: Based on the driver's physical condition information and image information, determine the driver's fatigue index during this lane departure.

[0039] Optionally, when determining the driver's fatigue index at the time of lane departure based on the driver's physical characteristics and image information, the fatigue index can be determined based on the driver's physical characteristics; and the state index can be determined based on the driver's image information. If either index exceeds a set threshold, the fatigue index or state index with the largest value among the indices exceeding the set threshold is taken as the driver's fatigue index at the time of lane departure; otherwise, the fatigue index and the state index are weighted and averaged, and the average result is taken as the driver's fatigue index at the time of lane departure.

[0040] Furthermore, the driver's vital signs information is heart rate information; therefore, when determining the driver's fatigue index based on the driver's vital signs information, specifically, the driver's heart rate information can be obtained through the pulse wave sensor integrated into the steering wheel, and the driver's first fatigue index and second fatigue index can be extracted; wherein, the first fatigue index is the standard deviation of the heart rate RR interval, used to characterize the overall fatigue level; the second fatigue index is the root mean square of the continuous heart rate RR difference, used to characterize the acute fatigue level; the driver's fatigue index is calculated based on the following formula: in, It is the number one fatigue index. It is the second fatigue index. , It is the weight of the first fatigue index. It is the weight of the second fatigue index.

[0041] In practice, a photoplethysmography (PPG) sensor can be integrated into the steering wheel, for example, by embedding PPG sensors at the 3 o'clock and 9 o'clock positions on the steering wheel to monitor heart rate variability in real time.

[0042] In this application, the driver's state index is determined based on the driver's image information. Specifically, the driver's image information during the driving process can be captured by an image acquisition device such as a camera on the vehicle. Then, the driver's physical state is analyzed from the driver's facial expressions or body movements to obtain the driver's state index.

[0043] It should be understood that in this application, a threshold value can be pre-set for the fatigue index, i.e., the fatigue index threshold, and a threshold value can be pre-set for the state index, i.e., the state index threshold. When the calculated fatigue index exceeds the fatigue index threshold, or the calculated state index exceeds the state index threshold, then the fatigue index or state index exceeding the threshold can be used as the driver's fatigue index during the lane departure; when both the calculated fatigue index and the calculated state index exceed the state index threshold, then the larger of the fatigue index and the state index can be used as the driver's fatigue index during the lane departure.

[0044] In fact, in addition to the above methods for determining the fatigue index, the driver's fatigue index at the time of lane departure can also be determined solely based on the driver's fatigue index or solely based on the driver's state index.

[0045] Step 108: Determine the environmental index at the time of this lane departure based on the environmental information.

[0046] Optionally, the environmental index at the time of this lane departure is determined based on the environmental information. Specifically, this can be based on the environmental information obtained from millimeter-wave radar and / or cameras, extracting the visibility, crosswind intensity, and traffic flow in the adjacent lane in the current environment; the environmental index at the time of this lane departure is calculated using the following formula: in, It's visibility. It is the crosswind intensity. It is the traffic flow in the adjacent lane; It is the furthest distance at which visibility can be monitored. It is the critical wind speed that can maintain vehicle safety. This is the critical traffic flow level to avoid collision risks. Preferably, The value can be 1000. It can take the value 100. It can take the value 10.

[0047] In practice, one or more environmental images can be acquired from image acquisition devices such as millimeter-wave radar and cameras. Then, information such as visibility and traffic flow in the current environment can be identified and analyzed from the environmental images. At the same time, the crosswind intensity in the current environment can also be measured using millimeter-wave radar.

[0048] Step 110: Calculate the comprehensive index affecting this lane departure based on the deviation index, the fatigue index, the environmental index, and the weighting factors assigned to each index.

[0049] In this application, different weighting factors can be pre-assigned to different indices for each type of target commercial vehicle, based on the vehicle's type. For example, for type A target commercial vehicle, the weighting factor for the deviation index could be 0.2, the weighting factor for the fatigue index could be 0.3, and the weighting factor for the environmental index could be 0.5; for type B target commercial vehicle, the weighting factors for the deviation index and fatigue index could be 0.3, and the weighting factor for the environmental index could be 0.4. Alternatively, the same weighting factors can be assigned to different types of target commercial vehicles, meaning that the three indices for each type of target commercial vehicle can have the same weighting factors. For example, for both type A and type B target commercial vehicles, the weighting factor for the deviation index could be 0.4, the weighting factor for the fatigue index could be 0.2, and the weighting factor for the environmental index could be 0.4.

[0050] The weighting factors assigned to each index in advance can be determined based on historical experience data. For example, based on the historical lane departure data of vehicles, the influence of different indices on lane departure can be analyzed to determine the weighting factors.

[0051] Optionally, after obtaining the first information related to vehicle driving in the current driving scenario of the target commercial vehicle, and before determining the deviation index of this lane departure based on the road information and the vehicle driving information, the vehicle operating condition of the target commercial vehicle can also be determined based on the road information and the vehicle driving information in the first information; according to the vehicle operating condition, a matching weight item is searched from a preset weight database, wherein the preset weight database stores multiple vehicle operating conditions, each vehicle operating condition is mapped to a weight item, and each weight item includes a weight factor assigned to the deviation index, a weight factor assigned to the fatigue index, and a weight factor assigned to the environmental index.

[0052] In other words, the weighting factors assigned to each index can be adjusted not only based on the type of the target commercial vehicle (which is a fixed weighting factor), but also in real-time based on the vehicle's operating conditions during operation. This allows for the allocation of weighting factors to the target commercial vehicle according to different circumstances.

[0053] Step 112: Select a warning level that matches the current lane departure based on the comprehensive index, and control the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle based on the warning strategy in the selected warning level to alert the driver of lane departure.

[0054] In this application, multiple warning levels can be preset, each warning level corresponds to a different range of comprehensive index values, and each warning level is configured with a corresponding warning strategy.

[0055] Optionally, the deviation index also includes the deviation direction; therefore, when controlling the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle based on the warning strategy in the selected warning level, if the selected warning level is primary, then the area on the side of the steering wheel of the target commercial vehicle corresponding to the deviation direction is controlled to vibrate at low frequency, and the instrument panel is controlled to display a warning. (Refer to...) Figure 2a As shown, if the deviation direction in the deviation index is leftward, then the left side of the steering wheel of the target commercial vehicle will vibrate at a low frequency (the red mark in the figure indicates low-frequency vibration, which is consistently applied to the left side). If the selected warning level is medium, then the steering wheel of the target commercial vehicle will vibrate alternately in two zones at a low frequency, the instrument panel will display a warning, and a torque opposite to the direction of deviation will be applied. (Refer to...) Figure 2bAs shown, if the deviation direction in the deviation index is leftward, then the steering wheel of the target commercial vehicle will first vibrate at a low frequency in the left area, then at a low frequency in the right area, and so on, alternating between the left and right sides (red marks in the diagram indicate low-frequency vibrations, with alternating low-frequency vibrations in the left and right areas). Specifically, vibration can begin in the area corresponding to the side of the deviation direction to indicate the opposite deviation, followed by repeated alternating low-frequency vibrations on both sides to warn the driver that the current deviation is relatively high. If the selected warning level is high, the entire steering wheel of the target commercial vehicle will vibrate at a high frequency, the instrument panel will display a warning, and the vehicle will switch to lane keeping assist mode. (Refer to...) Figure 2c As shown, the deviation direction in the deviation index is leftward deviation. At this time, the steering wheel of the target commercial vehicle will vibrate at a high frequency around the entire circle (the green mark in the figure indicates high frequency vibration, and it will continue to vibrate at a high frequency around the entire circle). Since the deviation is already very high, there is no need to remind the user of the deviation. Instead, the driver is warned directly through a high warning method that the degree of deviation is very high and that timely intervention by the autonomous driving system is required. This avoids the danger caused by the human's untimely reaction and improves driving safety and driving experience.

[0056] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0057] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0058] Figure 3 This application provides a structural block diagram of a lane departure warning control device according to one embodiment. Figure 3As shown. The lane departure warning control device 300 of this embodiment may include an information acquisition module 301, a deviation index determination module 302, a fatigue index determination module 303, an environmental index determination module 304, a comprehensive index determination module 305, and a warning module 306. The information acquisition module 301 is used to acquire first information related to vehicle driving in the current driving scenario when a target commercial vehicle is detected to have deviated from its lane. The first information includes at least: road information, vehicle driving information, driver physical condition information and image information, and environmental information. The deviation index determination module 302 is used to determine the deviation index of the current lane departure based on the road information and the vehicle driving information. The fatigue index determination module 303 is used to determine the driver's fatigue index at the time of the current lane departure based on the driver's physical condition information and image information. The environmental index determination module 304 is used to determine the environmental index at the time of the current lane departure based on the environmental information. The comprehensive index determination module 305 is used to calculate a comprehensive index affecting the current lane departure based on the deviation index, the fatigue index, the environmental index, and weighting factors assigned to each index. The warning module 306 is used to select a warning level matching the current lane departure based on the comprehensive index, and to control the corresponding area of ​​the steering wheel of the target commercial vehicle to vibrate based on the warning strategy in the selected warning level, so as to alert the driver to lane departure.

[0059] It should be noted that some or all of the lane departure warning control device in this embodiment may be an application located on a local terminal, or it may be a plugin or software development kit (SDK) or other functional unit set in an application located on a local terminal, or it may be a processing engine located on a network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side autonomous driving platform, etc. This embodiment does not impose any particular limitations on this.

[0060] It is understood that the application can be a native program installed on the local terminal, or it can be a web application of a browser on the local terminal. This embodiment does not limit this.

[0061] Optionally, in one possible implementation of this embodiment, when the deviation index determination module 302 determines the deviation index of the current lane deviation based on the road information and the vehicle driving information, it is specifically used to determine the lateral displacement of the set time period in which the lane deviation occurred based on the driving speed in the road information and the vehicle driving information, and calculate the lateral deviation speed according to the set time period and the lateral displacement; and use the absolute value of the ratio of the lateral deviation speed to the set benchmark deviation speed as the deviation index of the current lane deviation.

[0062] Optionally, in one possible implementation of this embodiment, when the fatigue index determination module 303 determines the driver's fatigue index at the time of lane departure based on the driver's physical signs information and image information, it is specifically used to determine the driver's fatigue index based on the driver's physical signs information; and to determine the driver's state index based on the driver's image information; if any index exceeds a set threshold, the fatigue index or state index with the largest value among the indices exceeding the set threshold is taken as the driver's fatigue index at the time of lane departure; otherwise, the fatigue index and the state index are weighted and averaged, and the average result is taken as the driver's fatigue index at the time of lane departure.

[0063] Optionally, in one possible implementation of this embodiment, the driver's vital signs information is heart rate information; then, when the fatigue index determination module 303 determines the driver's fatigue index based on the driver's vital signs information, it specifically uses the driver's heart rate information obtained through the pulse wave sensor integrated in the steering wheel to extract the driver's first fatigue index and second fatigue index; wherein, the first fatigue index is the standard deviation of the heart rate RR interval, used to characterize the overall fatigue level; the second fatigue index is the root mean square of the continuous heart rate RR difference, used to characterize the acute fatigue level; the driver's fatigue index is calculated based on the following formula: in, It is the number one fatigue index. It is the second fatigue index. , It is the weight of the first fatigue index. The weight of the second fatigue index is 70, which is the lower limit threshold of SDNN in the resting state of healthy adults as determined in this application, and the 40 is the normal value threshold of parasympathetic nerve activity as determined in this application.

[0064] Optionally, in one possible implementation of this embodiment, when determining the environmental index at the time of lane departure based on the environmental information, the environmental index determination module 304 is specifically used to extract the visibility, crosswind intensity, and traffic flow in the adjacent lane from the environmental information obtained from millimeter-wave radar and / or camera; and calculate the environmental index at the time of lane departure using the following formula: in, It's visibility. It is the crosswind intensity. It is the traffic flow in the adjacent lane; It is the furthest distance at which visibility can be monitored. It is the critical wind speed that can maintain vehicle safety. It is the critical traffic volume threshold for avoiding collision risks.

[0065] Optionally, in one possible implementation of this embodiment, the lane departure warning control device further includes a weight allocation module. After the information acquisition module 301 acquires the first information related to the target commercial vehicle's driving in the current driving scenario, and before the deviation index determination module 302 determines the deviation index of this lane departure based on the road information and the vehicle driving information, the weight allocation module is used to determine the vehicle operating condition of the target commercial vehicle based on the road information and vehicle driving information in the first information; and to search for a matching weight item from a preset weight database according to the vehicle operating condition. The preset weight database stores multiple vehicle operating conditions, each vehicle operating condition is mapped to a weight item, and each weight item includes a weight factor allocated to the deviation index, a weight factor allocated to the fatigue index, and a weight factor allocated to the environmental index.

[0066] Optionally, in one possible implementation of this embodiment, the deviation index further includes the deviation direction; then, when the warning module 306 controls the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle based on the warning strategy in the selected warning level, specifically, if the selected warning level is primary, it controls the side area of ​​the steering wheel of the target commercial vehicle corresponding to the deviation direction to vibrate at low frequency, and controls the instrument to display a prompt; if the selected warning level is intermediate, it controls the two sides of the steering wheel of the target commercial vehicle to perform alternating low-frequency vibration in two zones, and controls the instrument to display a prompt, and controls the application of torque opposite to the deviation direction; if the selected warning level is advanced, it controls the entire ring of the steering wheel of the target commercial vehicle to vibrate at high frequency, and controls the instrument to display a prompt, and switches to the lane keeping automatic mode.

[0067] In this embodiment, when a target commercial vehicle is detected to be deviating from its lane, first information related to the vehicle's movement is acquired. Based on this first information, the deviation index, fatigue index, and environmental index for this lane deviation are determined. Then, according to the weighting factors assigned to each index, a comprehensive index affecting this lane deviation is calculated. Subsequently, based on this comprehensive index, a warning strategy from the matching warning level is selected, and the corresponding area of ​​the target commercial vehicle's steering wheel is controlled to vibrate to alert the driver of the lane deviation. In this way, the magnitude of the comprehensive index determined by multiple indices affecting lane deviation can be matched with a warning level that corresponds to the current lane deviation situation and degree. Furthermore, the steering wheel of the target commercial vehicle is controlled to vibrate at different frequencies in different zones according to the warning strategy within the warning level, allowing the driver to intuitively and accurately identify the occurrence of lane deviation. Moreover, the warning alert methods are diversified, allowing for differentiated alerts based on the degree of lane deviation, thereby enabling the driver to promptly understand the lane deviation situation and improving the driving experience.

[0068] One embodiment of this application provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the lane departure warning control method described above.

[0069] One embodiment of this application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the lane departure warning control method as described above.

[0070] One embodiment of this application provides an autonomous driving vehicle, including the electronic devices described above. Specifically, the autonomous driving vehicle can be a Level 2 or higher vehicle.

[0071] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0072] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0073] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0074] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the lane departure warning control method. For example, in some embodiments, the lane departure warning control method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the lane departure warning control method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the lane departure warning control method by any other suitable means (e.g., by means of firmware).

[0076] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0077] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0078] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0080] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0081] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0082] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0083] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A lane departure warning and control method, characterized in that, include: When lane departure of a target commercial vehicle is detected, first information related to vehicle driving in the current driving scenario is obtained. The first information includes at least: road information, vehicle driving information, driver vital signs information and image information, and environmental information. Based on the road information and vehicle driving information in the first information, the vehicle operating condition of the target commercial vehicle is determined; According to the vehicle operating condition, a matching weight item is searched from a preset weight database. The preset weight database stores a variety of vehicle operating conditions, each of which is mapped to a weight item. Each weight item includes a weight factor assigned to the deviation index, a weight factor assigned to the fatigue index, and a weight factor assigned to the environmental index. The lane departure index is determined based on the road information and the vehicle driving information. Based on the driver's vital signs and image information, the driver's fatigue index at the time of lane departure is determined, specifically including: determining the driver's fatigue index based on the driver's vital signs; and determining the driver's state index based on the driver's image information. If either index exceeds a set threshold, the fatigue index or state index with the largest value exceeding the set threshold is taken as the driver's fatigue index at the time of lane departure; otherwise, the fatigue index and the state index are weighted and averaged, and the average result is taken as the driver's fatigue index at the time of lane departure. The driver's vital signs are heart rate information. The driver's fatigue index is then determined based on the driver's vital signs, specifically including: extracting the driver's first fatigue index and second fatigue index from the driver's heart rate information obtained through a pulse wave sensor integrated into the steering wheel; wherein the first fatigue index is the standard deviation of the heart rate RR interval, used to characterize the overall fatigue level; the second fatigue index is the root mean square of the continuous heart rate RR difference, used to characterize the acute fatigue level; the driver's fatigue index is calculated based on the following formula: in, It is the number one fatigue index. It is the second fatigue index. , It is the weight of the first fatigue index. It is the weight of the second fatigue index; The environmental index at the time of this lane departure is determined based on the environmental information. Based on the deviation index, the fatigue index, the environmental index, and the weighting factors assigned to each index, calculate the comprehensive index affecting this lane departure. Based on the comprehensive index, a warning level matching the current lane departure is selected, and based on the warning strategy in the selected warning level, the corresponding area of ​​the steering wheel of the target commercial vehicle is controlled to vibrate to alert the driver of lane departure.

2. The method as described in claim 1, characterized in that, The lane departure index is determined based on the road information and the vehicle driving information, specifically including: Based on the road information and the vehicle driving information, the lateral displacement during a set time period in which lane departure occurs is determined, and the lateral departure speed is calculated based on the set time period and the lateral displacement. The absolute value of the ratio of the lateral deviation speed to the set baseline deviation speed is used as the deviation index for this lane departure.

3. The method as described in claim 1, characterized in that, Based on the aforementioned environmental information, the environmental index at the time of this lane departure is determined, specifically including: Based on the environmental information obtained from millimeter-wave radar and / or cameras, the visibility, crosswind intensity, and traffic flow in adjacent lanes in the current environment are extracted. Calculate the environmental index for this lane departure using the following formula: in, It's visibility. It is the crosswind intensity. It is the traffic flow in the adjacent lane; It is the farthest distance at which visibility can be monitored. It is the critical wind speed that can maintain vehicle safety. It is the critical traffic volume threshold for avoiding collision risks.

4. The method as described in claim 1, characterized in that, The deviation index also carries the deviation direction; therefore, based on the warning strategy in the selected warning level, the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle is controlled, specifically including: If the selected warning level is primary, then the steering wheel of the target commercial vehicle will be controlled to vibrate at a low frequency on the side corresponding to the deviation direction, and the instrument panel will be controlled to display a warning. If the selected warning level is medium, then control the steering wheel of the target commercial vehicle to perform alternating low-frequency vibration in two zones, control the instrument to display a warning, and control the application of torque opposite to the deviation direction. If the selected warning level is high, the steering wheel of the target commercial vehicle will vibrate at high frequency, the instrument panel will display a warning, and the vehicle will switch to lane keeping mode.

5. A lane departure warning control device, characterized in that, include: The information acquisition module is used to acquire first information related to the vehicle's driving in the current driving scenario when a target commercial vehicle is detected to have deviated from its lane. The first information includes at least: road information, vehicle driving information, driver's vital signs information and image information, and environmental information. The weight allocation module is used to determine the vehicle operating condition of the target commercial vehicle based on the road information and vehicle driving information in the first information; and to search for a matching weight item from a preset weight database according to the vehicle operating condition. The preset weight database stores multiple vehicle operating conditions, each vehicle operating condition is mapped to a weight item, and each weight item includes a weight factor assigned to the deviation index, a weight factor assigned to the fatigue index, and a weight factor assigned to the environmental index. The lane departure index determination module is used to determine the lane departure index based on the road information and the vehicle driving information. The fatigue index determination module is used to determine the driver's fatigue index at the time of lane departure based on the driver's vital signs information and image information. Specifically, the fatigue index determination module is used to: determine the driver's fatigue index based on the driver's vital signs information; and determine the driver's state index based on the driver's image information. If either index exceeds a set threshold, the largest fatigue index or state index among those exceeding the set threshold is taken as the driver's fatigue index at the time of lane departure; otherwise, the fatigue index and the state index are weighted and averaged, and the average result is taken as the driver's fatigue index at the time of lane departure. The driver's vital signs information is heart rate information. Specifically, when determining the driver's fatigue index based on the driver's vital signs information, the fatigue index determination module uses the driver's heart rate information obtained through the pulse wave sensor integrated in the steering wheel to extract the driver's first fatigue index and second fatigue index. The first fatigue index is the standard deviation of the heart rate RR interval, used to characterize the overall fatigue level; the second fatigue index is the root mean square of the continuous heart rate RR difference, used to characterize the acute fatigue level. The driver's fatigue index is calculated based on the following formula: in, It is the number one fatigue index. It is the second fatigue index. , It is the weight of the first fatigue index. It is the weight of the second fatigue index; An environmental index determination module is used to determine the environmental index at the time of this lane departure based on the environmental information. The comprehensive index determination module is used to calculate the comprehensive index affecting the current lane departure based on the deviation index, the fatigue index, the environmental index, and the weighting factors assigned to each index. The warning module is used to select a warning level matching the current lane departure based on the comprehensive index, and to control the vibration of the corresponding area of ​​the steering wheel of the target commercial vehicle based on the warning strategy in the selected warning level, so as to alert the driver of lane departure.

6. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-4.

7. An autonomous vehicle, including the electronic device as described in claim 6.

Images