Human-machine collaborative obstacle avoidance method and system based on fatigue detection

Abstract

The invention provides a man-machine cooperation obstacle avoidance method and system based on fatigue detection, and the method comprises the steps: training a fatigue detection model; performing fatigue detection on the target driver by using the fatigue detection model to obtain a fatigue detection result; based on the fatigue detection result, determining a control right distribution result of the driver model and the automatic driving system; and performing man-machine cooperative obstacle avoidance according to a control right distribution result. The invention discloses a man-machine cooperative obstacle avoidance method and system based on fatigue detection. The method comprises the following steps: performing fatigue detection on a target driver by using a trained fatigue detection model to obtain a fatigue detection result; based on the fatigue detection result, the control right distribution result of the double-driving double-control man-machine co-driving structure is determined, man-machine cooperative obstacle avoidance is performed according to the control right distribution result, scatter points formed by the yaw velocity and the lateral velocity in the automobile obstacle avoidance process are more concentrated, and the smoothness and safety of the track are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a human-machine collaborative obstacle avoidance method and system based on fatigue detection. Background Art

[0002] Fatigue detection refers to the technology that determines whether the driver is in a state of fatigue by analyzing the driver's physiological and behavioral characteristics (such as blinking frequency, yawning frequency, head posture, etc.). This technology can monitor the driver's status in real time and issue an alarm when signs of fatigue are detected to prevent possible traffic accidents.

[0003] Human-machine collaborative obstacle avoidance refers to an obstacle avoidance strategy that combines human drivers and machine intelligence (autonomous driving system) in an autonomous driving system. This strategy aims to improve the efficiency and safety of obstacle avoidance, especially in complex or uncertain environments.

[0004] Existing autonomous driving systems generally avoid obstacles based on the perception results of sensors. Since obstacles may appear suddenly (for example, pedestrians suddenly rushing out), it is easy for the vehicle to make emergency obstacle avoidance, skid or lose control when driving at high speed, affecting the stability and safety of the vehicle.

[0005] In view of this, there is an urgent need for a human-machine collaborative obstacle avoidance method and system based on fatigue detection to at least solve the above-mentioned deficiencies. Summary of the invention

[0006] One of the purposes of the present invention is to provide a human-machine collaborative obstacle avoidance method and system based on fatigue detection, which uses a trained fatigue detection model to perform fatigue detection on a target driver to obtain a fatigue detection result; based on the fatigue detection result, the control right allocation result of a dual-driving and dual-control human-machine co-driving structure is determined, and human-machine collaborative obstacle avoidance is performed according to the control right allocation result. During the obstacle avoidance process of the car, the scattered points composed of the yaw angular velocity and the lateral velocity are more concentrated, thereby improving the smoothness and safety of the trajectory.

[0007] An embodiment of the present invention provides a human-machine collaborative obstacle avoidance method based on fatigue detection, comprising:

[0008] Step 1: Train the fatigue detection model;

[0009] Step 2: Use the fatigue detection model to perform fatigue detection on the target driver and obtain fatigue detection results;

[0010] Step 3: Based on the fatigue detection results, determine the driver model and the control rights allocation result of the automatic driving system;

[0011] Step 4: Perform human-machine collaborative obstacle avoidance based on the control rights allocation results.

[0012] Preferably, training the fatigue detection model includes:

[0013] Obtaining the driver's driving process information;

[0014] The fatigue detection model is trained using the BP neural network based on the driver's driving process information.

[0015] Preferably, determining the control rights allocation result of the driver model and the automatic driving system based on the fatigue detection result includes:

[0016] Characterize the fatigue detection results and obtain the fatigue factor;

[0017] Obtain space collision risk;

[0018] According to the spatial collision risk and fatigue factor, the automatic driving control weight coefficient is obtained through fuzzy logic reasoning.

[0019] An embodiment of the present invention provides a human-machine collaborative obstacle avoidance method based on fatigue detection, further comprising:

[0020] The driver model is established based on the optimal preview lateral acceleration.

[0021] An embodiment of the present invention provides a human-machine collaborative obstacle avoidance method based on fatigue detection, further comprising:

[0022] The autonomous driving system uses a linear quadratic regulator to control the vehicle to follow the planned trajectory.

[0023] Preferably, using the fatigue detection model to perform fatigue detection on the target driver to obtain the fatigue detection result includes:

[0024] Obtaining reference monitoring data of the target driver;

[0025] According to the reference monitoring data, a preceding triggering behavior set is obtained;

[0026] Determine the triggering frequency of the preceding triggering behavior of the same behavior type according to the preceding triggering behavior set;

[0027] According to the trigger frequency, the monitoring weight of the preceding triggering behavior of the corresponding behavior type is determined;

[0028] Obtain real-time monitoring data of the target driver;

[0029] Obtain target preceding triggering behavior and target monitoring weight according to real-time monitoring data;

[0030] According to the target monitoring weight and the preset computing resources, the strategy library is adjusted, and the fatigue detection model is used to perform fatigue detection on the target driver.

[0031] Preferably, obtaining reference monitoring data of the target driver includes:

[0032] Acquire the first historical monitoring data of the target driver;

[0033] Obtain the first driver attribute of the target driver;

[0034] Based on fatigue monitoring big data, target monitoring data is obtained according to the first driver's attributes;

[0035] The first historical monitoring data and the target monitoring data are used together as reference monitoring data;

[0036] Wherein, based on the fatigue monitoring big data, according to the first driver's attributes, the target monitoring data is obtained, including:

[0037] Parse fatigue monitoring big data to obtain the second driver attribute set of the monitoring personnel set;

[0038] Support vectorization is performed on the first driver attribute and the second driver attribute in the second driver attribute set respectively, so as to obtain a first support vector and a second support vector set correspondingly;

[0039] Classifying the second support vector set based on a support vector machine to obtain a plurality of third support vector sets;

[0040] Calculate the vector angle between the first support vector and each set of vectors in the third support vector set, and then calculate the mean of the vector angles corresponding to the third support vector set;

[0041] Determine the monitoring personnel corresponding to the third support vector set with the smallest vector angle mean and use it as the target monitoring personnel;

[0042] The second historical monitoring data of the target monitoring person is used as the target monitoring data.

[0043] Preferably, obtaining real-time monitoring data of the target driver includes:

[0044] Acquire first monitoring data of a first camera preset at a first position of a center console of a vehicle driven by a target driver;

[0045] Acquire second monitoring data of a second camera preset at a second position of a center console of a vehicle driven by the target driver;

[0046] Get the audio data from the vehicle microphone;

[0047] Get vehicle navigation information;

[0048] The first monitoring data, the second monitoring data, the radio data and the vehicle navigation information are used together as real-time monitoring data;

[0049] The first position is within a preset range of the intersection of the A-pillar and the center console close to the target driver;

[0050] Among them, the first camera keeps working when the target driver starts driving the vehicle;

[0051] The second position is in the area directly in front of the target driver's seat;

[0052] Among them, after the second camera recognizes the target's preceding triggering behavior, it is lifted from inside the center console based on a preset lifting device and starts working.

[0053] Preferably, obtaining the spatial collision risk includes:

[0054] Obtain visual information and radar scanning information of the target driver's vehicle;

[0055] Obtain the acquisition camera parameters of visual information;

[0056] Acquire first layout information of the acquisition camera relative to the vehicle body;

[0057] Reconstructing a three-dimensional model of the visual range according to the acquired camera parameters, the first layout information and the visual information;

[0058] Annotate the vehicle model and the planned route correspondingly in the three-dimensional model to obtain an annotated model;

[0059] Acquiring second layout information of the vehicle radar relative to the vehicle body;

[0060] Get radar parameters of vehicle radar;

[0061] Reconstruct the blind area model in the annotation model according to the radar parameters, the second layout information and the radar scanning information to obtain the target model;

[0062] Obtaining the vehicle speed of the vehicle driven by the target driver;

[0063] According to the vehicle speed, the obstacle model range circle in the target model is determined;

[0064] Calculate the relative distance between the obstacle model and the vehicle model in the obstacle model range circle;

[0065] Calculate the angle sum of the movement directions of the obstacle model and the vehicle model in the obstacle model range circle;

[0066] The relative distance sum and the angle sum are normalized, and the normalized results are summed to obtain the spatial collision risk.

[0067] An embodiment of the present invention provides a human-machine collaborative obstacle avoidance system based on fatigue detection, comprising:

[0068] Fatigue detection model training subsystem, used to train fatigue detection model;

[0069] A fatigue detection result acquisition subsystem is used to perform fatigue detection on the target driver using a fatigue detection model and obtain fatigue detection results;

[0070] A control rights allocation subsystem, used to determine the control rights allocation result of the driver model and the automatic driving system based on the fatigue detection result;

[0071] The human-machine collaborative obstacle avoidance subsystem is used to perform human-machine collaborative obstacle avoidance according to the control right allocation result. The beneficial effects of the present invention are:

[0072] The present invention uses a trained fatigue detection model to perform fatigue detection on a target driver to obtain a fatigue detection result; based on the fatigue detection result, a control right allocation result of a dual-driver and dual-control human-machine co-driving structure is determined, and human-machine collaborative obstacle avoidance is performed according to the control right allocation result. During the obstacle avoidance process of the car, the scattered points composed of the yaw angular velocity and the lateral velocity are more concentrated, thereby improving the smoothness and safety of the trajectory.

[0073] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0074] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0076] Figure 1 Schematic diagram of a human-machine collaborative obstacle avoidance method based on fatigue detection in an embodiment of the present invention;

[0077] Figure 2 Schematic diagram of a human-machine collaborative obstacle avoidance system based on fatigue detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0079] The embodiment of the present invention provides a human-machine collaborative obstacle avoidance method based on fatigue detection, such as Figure 1 As shown, including:

[0080] Step 1: Train a fatigue detection model; the fatigue detection model is an AI model that identifies whether the driver is in a fatigue state and evaluates the degree of fatigue by analyzing the driver's physiological signals (such as facial expressions, eye closure frequency, head movement, etc.) and behavioral characteristics (such as driving habits, reaction time, etc.);

[0081] Step 2: Perform fatigue detection on the target driver using the fatigue detection model to obtain fatigue detection results; wherein, when performing fatigue detection on the target driver using the fatigue detection model, the collected data (such as physiological data and behavioral data) of the target driver collected in real time is input into the fatigue detection model to obtain fatigue detection results;

[0082] Step 3: Based on the fatigue detection results, determine the control rights allocation results of the driver model and the automatic driving system; wherein the driver model and the automatic driving system are a dual-driving and dual-control human-machine co-driving structure, wherein the automatic driving system uses a linear quadratic regulator to control the vehicle to track the planned trajectory, the driver model is established based on the optimal preview lateral acceleration, based on fuzzy logic reasoning, according to the fatigue detection results and the spatial collision risk (SRC), the human-machine collaborative obstacle avoidance strategy is designed, and the control rights allocation result is the control rights of the automatic driving system;

[0083] Step 4: Perform human-machine collaborative obstacle avoidance according to the control right allocation result. When performing human-machine collaborative obstacle avoidance according to the control right allocation result, the angle output by the driver model and the angle output by the automatic driving system are used as the final control signal after the control right allocation strategy is applied.

[0084] The working principle and beneficial effects of the above technical solution are:

[0085] The present invention uses a trained fatigue detection model to perform fatigue detection on a target driver to obtain a fatigue detection result; based on the fatigue detection result, a control right allocation result of a dual-driver and dual-control human-machine co-driving structure is determined, and human-machine collaborative obstacle avoidance is performed according to the control right allocation result. During the obstacle avoidance process of the car, the scattered points composed of the yaw angular velocity and the lateral velocity are more concentrated, thereby improving the smoothness and safety of the trajectory.

[0086] In one embodiment, training a fatigue detection model includes:

[0087] Acquire the driver's driving process information; wherein the driver's driving process information is: driving process data marked with fatigue driving and non-fatigue driving (such as: facial image, driving process behavior);

[0088] The fatigue detection model is trained using the BP neural network based on the driver's driving process information.

[0089] The working principle and beneficial effects of the above technical solution are:

[0090] The present invention introduces marked driver driving process information, uses BP neural network to learn data features of fatigue driving process data and non-fatigue driving process data, obtains the learned fatigue detection model, and automatically identifies the driver's fatigue state based on the collected data, which is more intelligent.

[0091] In one embodiment, based on the fatigue detection result, determining the driver model and the control right allocation result of the automatic driving system includes:

[0092] Characterize the fatigue detection result to obtain a fatigue factor; wherein the fatigue factor is: an indicator for quantifying the fatigue detection result, such as: a fatigue detection result characterization vector;

[0093] Obtaining a spatial collision risk; wherein the spatial collision risk is: an indicator for evaluating the potential collision risk between the vehicle and surrounding obstacles, which is determined by the relative distance between the vehicle and the obstacle and the angle (heading angle) between the vehicle's moving direction and the obstacle;

[0094] According to the spatial collision risk and fatigue factor, the automatic driving control weight coefficient is obtained through fuzzy logic reasoning.

[0095] The working principle and beneficial effects of the above technical solution are:

[0096] The present invention introduces the spatial collision risk and fatigue factor, and infers the automatic driving control weight coefficient according to fuzzy logic for subsequent calculation of the front wheel turning angle of cooperative obstacle avoidance, so that obstacle avoidance is more accurate.

[0097] In one embodiment, fatigue detection is performed on a target driver using a fatigue detection model to obtain a fatigue detection result, including:

[0098] Obtaining reference monitoring data of the target driver; wherein the reference monitoring data is: fatigue driving data obtained by fatigue monitoring of the target driver using a fatigue detection model in the past, including: physiological signals and driving data;

[0099] According to the reference monitoring data, a preceding triggering behavior set is obtained; wherein the preceding triggering behavior set is: a driving behavior set, an expression set, and a semantic set before generating a fatigue detection result;

[0100] According to the preceding triggering behavior set, the triggering frequency of the preceding triggering behavior of the same behavior type is determined; wherein the behavior type is the type of behavior, such as driving time, expression, and speech; the triggering frequency is the frequency of determining driving fatigue after the preceding triggering behavior of the corresponding behavior type exists;

[0101] According to the trigger frequency, the monitoring weight of the preceding trigger behavior of the corresponding behavior type is determined; wherein the monitoring weight and the trigger frequency are numerically equal;

[0102] Acquire real-time monitoring data of the target driver; wherein the real-time monitoring data includes: driving data, voice data and facial image data of the target driver collected in real time;

[0103] Obtain the target preceding trigger behavior and target monitoring weight according to the real-time monitoring data; wherein the target preceding trigger behavior is specifically: the real-time trigger behavior that matches the preceding trigger behavior in the real-time monitoring data; the target monitoring weight is: the monitoring weight corresponding to the preceding trigger behavior that matches the preceding trigger behavior;

[0104] According to the target monitoring weight and the preset computing power resource adjustment strategy library, the fatigue detection model is used to detect the target driver's fatigue. Among them, the preset computing power resource adjustment strategy library includes: one-to-one corresponding monitoring weights and computing power adjustment strategies. The larger the monitoring weight, the higher the system computing power allocation.

[0105] The working principle and beneficial effects of the above technical solution are:

[0106] Different drivers have different driving habits, and their behaviors that may cause fatigue driving are also different. For example, long-distance drivers are prone to fatigue driving when driving for a long time at night. Another example is that driver A tends to blink more frequently and shake his head when he is sleepy, driver B tends to nod when he is sleepy, and driver C tends to forget to turn on the turn signal when he is sleepy. Therefore, reference monitoring data is obtained to determine the set of previous triggering behaviors that have been used to determine fatigue driving of the target driver in the past, and to determine the triggering frequency of previous triggering behaviors of the same behavior type. The higher the triggering frequency, the more likely it is that fatigue driving behavior will occur after the corresponding previous triggering behavior. More system computing power needs to be allocated to the fatigue detection model, which improves the timeliness of fatigue detection.

[0107] In one embodiment, obtaining reference monitoring data of a target driver includes:

[0108] Acquire first historical monitoring data of the target driver; wherein the first historical monitoring data is: fatigue driving data of the target driver obtained by fatigue monitoring using a fatigue detection model in the past;

[0109] Acquire a first driver attribute of the target driver; wherein the first driver attribute is: driving-related information of the target driver, such as: driving habits, age, etc.;

[0110] Based on fatigue monitoring big data, target monitoring data is obtained according to the first driver's attributes;

[0111] The first historical monitoring data and the target monitoring data are used together as reference monitoring data;

[0112] Wherein, based on the fatigue monitoring big data, according to the first driver's attributes, the target monitoring data is obtained, including:

[0113] Parse the fatigue monitoring big data to obtain a second driver attribute set of the monitoring personnel set; wherein the fatigue monitoring big data is obtained through the vehicle computer system sharing; the monitoring personnel set is: a set of drivers monitored by the fatigue monitoring big data; the second driver attribute set includes multiple second driver attributes, which are the driver attributes of the monitoring personnel;

[0114] The first driver attribute and the second driver attribute in the second driver attribute set are respectively support vectorized to obtain a first support vector and a second support vector set; wherein the support vectorization is: converting the driver attribute into a numerical vector form that can be processed by the support vector machine algorithm, such as: an attribute information feature vector, and the attribute information features include: driving habits, age;

[0115] Classifying the second support vector set based on a support vector machine to obtain a plurality of third support vector sets;

[0116] Calculate the vector angle of each set vector in the first support vector and the third support vector set, and then calculate the mean of the vector angles corresponding to the third support vector set; wherein, the larger the vector angle, the greater the difference between the first driver attribute corresponding to the first support vector and the second driver attribute corresponding to the third support vector, and the mean of the vector angle represents the similarity between the driver attributes of the target driver and a class of second drivers represented by the third support vector;

[0117] Determine the monitoring personnel corresponding to the third support vector set with the smallest vector angle mean and use it as the target monitoring personnel;

[0118] The second historical monitoring data of the target monitoring person is used as the target monitoring data, wherein the second historical monitoring data is: the fatigue driving data of the target monitoring person obtained by fatigue monitoring using the fatigue detection model in the past.

[0119] The working principle and beneficial effects of the above technical solution are:

[0120] The present invention adopts two methods to obtain reference monitoring data of the target driver. The first method is to directly obtain the fatigue driving data of the target driver obtained by fatigue monitoring using a fatigue detection model in history; the second method is to introduce big data technology to obtain fatigue monitoring big data, and support vectorize the first driver attribute and the second driver attribute obtained by analyzing the fatigue monitoring big data respectively to obtain a first support vector and a second support vector set. Generally, people with the same type of driver attributes have a greater probability of having similar preceding triggering behaviors. Therefore, the second support vector set is classified based on the support vector machine to obtain multiple third support vector sets; each third support vector set The vector set represents a type of second driver (for example, a long-distance train driver). The vector angle mean of the first support vector and the set vector in the third support vector set is calculated, and the vector angle mean is used to represent the similarity between the target driver and the driver of the corresponding type in the third support vector set. The smaller the vector angle mean, the higher the similarity of the corresponding driver attributes. The fatigue driving data of the target monitoring person corresponding to the third support vector set with the smallest vector angle mean is obtained by using the fatigue detection model for fatigue monitoring in the past as the target monitoring data, which improves the suitability of the target monitoring data acquisition and makes the extraction of subsequent preceding triggering behaviors more accurate.

[0121] In one embodiment, obtaining real-time monitoring data of a target driver includes:

[0122] Acquire first monitoring data of a first camera preset at a first position of a center console of a vehicle driven by a target driver;

[0123] Acquire second monitoring data of a second camera preset at a second position of a center console of a vehicle driven by the target driver;

[0124] Acquire the sound data collected by the vehicle microphone; wherein the sound data is the collected voice data of the target driver;

[0125] Obtaining vehicle navigation information; wherein the vehicle navigation information is: real-time navigation of the vehicle driven by the target driver;

[0126] The first monitoring data, the second monitoring data, the radio data and the vehicle navigation information are used together as real-time monitoring data;

[0127] The first position is within a preset range of the intersection of the A-pillar and the center console close to the target driver; the preset range is manually preset, for example: within a circle center area with a radius of 30 cm and the intersection of the A-pillar and the center console close to the target driver as the center, and the first position is also required to be limited to the center console, and the first camera at the first position can capture the main driver's seat;

[0128] Among them, the first camera keeps working when the target driver starts driving the vehicle;

[0129] The second position is in the area directly in front of the target driver's seat; the area directly in front is, for example, the center console area directly above the instrument panel;

[0130] Among them, after the second camera recognizes the target's preceding triggering behavior, it is lifted up from the center console based on the preset lifting device and starts working. The second camera is in a dormant state when the target's preceding triggering behavior is not recognized, hidden inside the center console, and does not obstruct the target driver's sight. After recognizing the target's preceding triggering behavior, it is lifted up again to capture more behavioral details and facial details of the target driver from the front.

[0131] The working principle and beneficial effects of the above technical solution are:

[0132] The present invention sets two cameras at the center console of the vehicle driven by the target driver to obtain monitoring data (first monitoring data and second monitoring data), thereby improving the multi-directionality of monitoring data acquisition; in addition, the introduction of the sound reception data of the vehicle microphone and the vehicle navigation information improves the comprehensiveness of real-time monitoring data acquisition.

[0133] In one embodiment, obtaining the spatial collision risk includes:

[0134] Obtaining visual information and radar scanning information of the vehicle driven by the target driver; wherein the visual information is obtained by photographing with a camera preset on the body of the vehicle driven by the target driver, and the radar scanning information is obtained by scanning with a radar probe preset on the body of the vehicle driven by the target driver;

[0135] Obtaining the acquisition camera parameters of the visual information; wherein the acquisition camera parameters are: the parameters of the camera preset on the body of the vehicle driven by the target driver, such as: aperture range, field of view angle range;

[0136] Acquire first layout information of the acquisition camera relative to the vehicle body; wherein the first layout information is: position distribution information of the acquisition camera relative to the vehicle driven by the target driver;

[0137] Reconstructing a three-dimensional model of the visual range according to the acquired camera parameters, the first layout information and the visual information; wherein, when reconstructing the three-dimensional model of the visual range, the captured images are spliced ​​according to the first layout information and the visual information, and while splicing, the distortion of the splicing process is determined according to the acquired camera parameters and the first layout information, and a reconstruction strategy for eliminating distortion is introduced into the three-dimensional model of the reconstructed visual range to eliminate lens distortion to obtain the three-dimensional model;

[0138] The vehicle model and the planned route are marked in the three-dimensional model to obtain a marked model; wherein the vehicle model is a three-dimensional modeling result of the target driver driving the vehicle, and the planned route is obtained based on the vehicle navigation, and the planned route is marked in the three-dimensional model according to the relative position relationship between the planned route and the actual space;

[0139] Acquire second layout information of the vehicle radar relative to the vehicle body; wherein the second layout information is: the layout of the vehicle radar relative to the vehicle body;

[0140] Obtain radar parameters of the vehicle radar; wherein the radar parameters include: operating frequency, detection range, and angular resolution of the radar;

[0141] Reconstruct the blind area model in the annotation model according to the radar parameters, the second layout information and the radar scanning information to obtain the target model; wherein, when reconstructing the blind area model in the annotation model according to the radar parameters, the second layout information and the radar scanning information, establish a three-dimensional point cloud model around the vehicle body according to the fusion of the radar scanning information and the second layout information, then correct the radar scanning information according to the second layout information and the radar parameters, supplement the blind area of ​​the three-dimensional point cloud model based on the corrected radar scanning information to obtain the target model;

[0142] Obtain the vehicle speed of the target driver driving the vehicle; wherein the vehicle speed is, for example: 60km / h;

[0143] According to the vehicle speed, determine the obstacle model range circle in the target model; wherein, the radius of the obstacle model range circle is determined according to the vehicle speed. The faster the vehicle speed, the longer the radius of the obstacle model range circle in the target model. The specific proportional relationship between the vehicle speed and the radius of the obstacle model range circle is preset by the staff. In addition, the obstacle model range circle takes the midpoint of the vehicle body model as the center.

[0144] Calculate the relative distance sum between the obstacle model and the vehicle model in the obstacle model range circle; wherein the relative distance sum is: the sum of the relative distances between each obstacle model and the vehicle model in the obstacle model range circle;

[0145] Calculate the angle sum of the movement directions of the obstacle model and the vehicle model in the obstacle model range circle; wherein the angle sum is: the angle between the movement direction of the obstacle model and the movement direction of the vehicle model;

[0146] The relative distance sum and the angle sum are normalized, and the normalized results are summed to obtain the spatial collision risk. During the normalization process, the larger the relative distance sum, the smaller the spatial collision risk. In particular, when the relative distance sum is 0, the spatial collision risk is 0. The larger the angle sum, the greater the spatial collision risk. The specific normalization relationship is preset manually.

[0147] The working principle and beneficial effects of the above technical solution are:

[0148] The present invention obtains visual information and radar scanning information of the target driver driving the vehicle, and constructs a three-dimensional model around the body of the vehicle driven by the target driver based on the visual information and radar scanning information. However, since the number of cameras and radars around the target driver driving the vehicle is limited and fixed, the accuracy of three-dimensional reconstruction based on any single data is relatively poor. Therefore, an initial three-dimensional model is first established based on the first layout information and visual information. Since the initial three-dimensional model is based on the splicing and fusion of videos shot by cameras in different directions, a fusion algorithm at the splicing point can be introduced according to the first layout information to reduce the video fusion error. In addition, the vehicle model and the future route of the navigation in the navigation information are also marked in the three-dimensional model based on the relative position relationship to obtain a marked model. The second layout information of the vehicle radar is obtained, and a three-dimensional point cloud model is reconstructed according to the second layout information and the radar scanning information. The model range of the three-dimensional point cloud model is not as good as the marked model, but it covers the blind area of ​​the camera (for example: the four corner areas of the vehicle driven by the target driver). Therefore, after the radar scanning data is corrected according to the radar parameters and the second layout information, the missing blind area in the marked model is completed to obtain the target model;

[0149] The target model is a three-dimensional model of the vehicle body detectable range constructed in real time; the faster the target driver drives the vehicle, the more obstacles may collide with it; therefore, the obstacle model range circle in the target model is determined according to the vehicle speed, and the higher the vehicle speed, the larger the corresponding obstacle model range circle; the relative distance sum of the obstacle model and the vehicle model in the obstacle model range circle is calculated, and the larger the relative distance sum is, the farther the target driver drives the vehicle from the obstacle, and the lower the spatial collision risk; the angle of the obstacle model and the vehicle model in the obstacle model range circle is calculated, and the larger the angle is, the greater the possibility of the target driver driving the vehicle and the obstacle colliding head-on, and the higher the spatial collision risk; the relative distance sum and the angle sum are normalized to obtain the spatial collision risk, thereby improving the calculation accuracy of the spatial collision risk.

[0150] The embodiment of the present invention provides a human-machine collaborative obstacle avoidance system based on fatigue detection, such as Figure 2 As shown, including:

[0151] A fatigue detection model training subsystem 1 is used to train a fatigue detection model;

[0152] A fatigue detection result acquisition subsystem 2 is used to perform fatigue detection on a target driver using a fatigue detection model and obtain a fatigue detection result;

[0153] A control right allocation subsystem 3, used for determining a control right allocation result of a driver model and an automatic driving system based on a fatigue detection result;

[0154] The human-machine collaborative obstacle avoidance subsystem 4 is used to perform human-machine collaborative obstacle avoidance according to the control right allocation result.

[0155] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A human-machine collaborative obstacle avoidance method based on fatigue detection, characterized in that: include: Training fatigue detection models; Use the fatigue detection model to perform fatigue detection on the target driver and obtain fatigue detection results; Based on the fatigue detection results, determine the driver model and the control rights allocation results of the autonomous driving system; According to the control rights allocation results, human-machine collaborative obstacle avoidance is carried out.

2. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 1, characterized in that: Training a fatigue detection model, including: Obtaining the driver's driving process information; The fatigue detection model is trained using the BP neural network based on the driver's driving process information.

3. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 1, characterized in that: Based on the fatigue detection results, the driver model and the control rights allocation results of the autonomous driving system are determined, including: Characterize the fatigue detection results and obtain the fatigue factor; Obtain space collision risk; According to the spatial collision risk and fatigue factor, the automatic driving control weight coefficient is obtained through fuzzy logic reasoning.

4. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 1, characterized in that: Also includes: The driver model is established based on the optimal preview lateral acceleration.

5. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 1, characterized in that: Also includes: The autonomous driving system uses a linear quadratic regulator to control the vehicle to follow the planned trajectory.

6. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 1, characterized in that: Use the fatigue detection model to perform fatigue detection on the target driver and obtain fatigue detection results, including: Obtaining reference monitoring data of the target driver; According to the reference monitoring data, a preceding triggering behavior set is obtained; Determine the triggering frequency of the preceding triggering behavior of the same behavior type according to the preceding triggering behavior set; According to the trigger frequency, the monitoring weight of the preceding triggering behavior of the corresponding behavior type is determined; Obtain real-time monitoring data of the target driver; Obtain target preceding triggering behavior and target monitoring weight according to real-time monitoring data; According to the target monitoring weight and the preset computing resources, the strategy library is adjusted, and the fatigue detection model is used to perform fatigue detection on the target driver.

7. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 6, characterized in that: Obtain reference monitoring data of the target driver, including: Acquire the first historical monitoring data of the target driver; Obtain the first driver attribute of the target driver; Based on fatigue monitoring big data, target monitoring data is obtained according to the first driver's attributes; The first historical monitoring data and the target monitoring data are used together as reference monitoring data; Wherein, based on the fatigue monitoring big data, according to the first driver's attributes, the target monitoring data is obtained, including: Parse fatigue monitoring big data to obtain the second driver attribute set of the monitoring personnel set; Support vectorization is performed on the first driver attribute and the second driver attribute in the second driver attribute set respectively, so as to obtain a first support vector and a second support vector set correspondingly; Classifying the second support vector set based on a support vector machine to obtain a plurality of third support vector sets; Calculate the vector angle between the first support vector and each set of vectors in the third support vector set, and then calculate the mean of the vector angles corresponding to the third support vector set; Determine the monitoring personnel corresponding to the third support vector set with the smallest vector angle mean and use it as the target monitoring personnel; The second historical monitoring data of the target monitoring person is used as the target monitoring data.

8. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 6, characterized in that: Obtain real-time monitoring data of the target driver, including: Acquire first monitoring data of a first camera preset at a first position of a center console of a vehicle driven by a target driver; Acquire second monitoring data of a second camera preset at a second position of a center console of a vehicle driven by the target driver; Get the audio data from the vehicle microphone; Get vehicle navigation information; The first monitoring data, the second monitoring data, the radio data and the vehicle navigation information are used together as real-time monitoring data; The first position is within a preset range of the intersection of the A-pillar and the center console close to the target driver; Among them, the first camera keeps working when the target driver starts driving the vehicle; The second position is in the area directly in front of the target driver's seat; Among them, after the second camera recognizes the target's preceding triggering behavior, it is lifted from inside the center console based on a preset lifting device and starts working.

9. The human-machine collaborative obstacle avoidance method based on fatigue detection as claimed in claim 3, characterized in that: Get the space collision risk, including: Obtain visual information and radar scanning information of the target driver's vehicle; Obtaining the camera parameters for collecting visual information; Acquire first layout information of the acquisition camera relative to the vehicle body; Reconstructing a three-dimensional model of the visual range according to the acquired camera parameters, the first layout information and the visual information; Annotate the vehicle model and the planned route correspondingly in the three-dimensional model to obtain an annotated model; Acquiring second layout information of the vehicle radar relative to the vehicle body; Get radar parameters of vehicle radar; Reconstruct the blind area model in the annotation model according to the radar parameters, the second layout information and the radar scanning information to obtain the target model; Obtaining the vehicle speed of the vehicle driven by the target driver; According to the vehicle speed, the obstacle model range circle in the target model is determined; Calculate the relative distance between the obstacle model and the vehicle model in the obstacle model range circle; Calculate the angle sum of the movement directions of the obstacle model and the vehicle model in the obstacle model range circle; The relative distance sum and the angle sum are normalized, and the normalized results are summed to obtain the spatial collision risk.

10. A human-machine collaborative obstacle avoidance system based on fatigue detection, characterized in that: include: Fatigue detection model training subsystem, used to train fatigue detection model; A fatigue detection result acquisition subsystem is used to perform fatigue detection on the target driver using a fatigue detection model and obtain fatigue detection results; A control rights allocation subsystem, used to determine the control rights allocation result of the driver model and the automatic driving system based on the fatigue detection result; The human-machine collaborative obstacle avoidance subsystem is used to perform human-machine collaborative obstacle avoidance based on the control rights allocation results.

Images