An assisted driving system and method based on visual recognition

Abstract

The application belongs to the technical field of auxiliary driving, and particularly relates to an auxiliary driving system and method based on visual recognition, which comprises obtaining first element information and second element information in the driving process of a vehicle; a model is established for road elements and road edge elements in the second element information respectively, the road edge elements affecting the recognition of the road elements are recognized, and the recognized road edge elements are used for planning a movement path of the first elements between the second elements; the model established according to the road elements and the road edge elements is used for segmentation, so that the road elements are segmented into at least two first sub-elements which are related to each other, and the road edge elements are respectively segmented into independent second sub-elements. The application can provide driving safety guarantee, detection and perception ability of mixed road lane lines and lane boundaries in different interference environments, provide judgment and guidance of driving safety state for drivers, reduce the driving strength of the drivers, and improve driving safety and traffic efficiency.

Description

Technical Field

[0001] This invention belongs to the field of driver assistance technology, specifically relating to a driver assistance system and method based on vision recognition. Background Technology

[0002] Advanced Driver Assistance Systems (ADAS) based on vision recognition are systems that use machine vision technology to perceive and understand the vehicle's driving environment, thereby assisting the driver in making decisions and operations. The system acquires image data of the surrounding environment through visual sensors such as onboard cameras, and uses computer vision, machine learning, and other algorithms to process and analyze this data in order to detect and identify information such as roads, traffic signs, pedestrians, and obstacles.

[0003] To improve the safety of motor vehicle driving and effectively avoid traffic accidents, vehicle driver assistance systems have become a research hotspot in the field of intelligent transportation.

[0004] Problems with existing technology:

[0005] However, current vehicle driver assistance systems cannot reliably provide driving safety in situations such as low light, blurred or damaged lane lines, unclear road conditions, complex vehicle environment obstacles, and various factors that interfere with driver operation. Summary of the Invention

[0006] The purpose of this invention is to provide a vision-based driver assistance system and method that can provide driving safety assurance, detect and perceive lane lines and lane boundaries on mixed roads under different interference environments, provide drivers with judgment and guidance on driving safety status, reduce the driver's driving intensity, and improve driving safety and traffic efficiency.

[0007] The specific technical solution adopted by this invention is as follows:

[0008] A vision-based driver assistance method includes the following steps:

[0009] Acquire first-factor information and second-factor information during vehicle operation;

[0010] By establishing models for the road elements and road edge elements in the second element information, the road edge elements that affect the identification of road elements are identified, which are then used to plan the movement path of the first element between the second element.

[0011] The model is segmented based on road features and road edge features, such that each road feature is divided into at least two interrelated first sub-features, and each road edge feature is divided into an independent second sub-feature.

[0012] The top view of the first sub-element and the second sub-element is obtained by inverse projection transformation, and a three-dimensional visualization model of the first and second sub-element is constructed by mapping it to the driver assistance system.

[0013] Based on the obtained 3D models of the first and second visual elements, a position model of the vehicle itself relative to the first element in the first element is obtained.

[0014] Based on the second element, driving routes are planned and avoidance measures are taken to identify hazardous elements.

[0015] Also includes:

[0016] The driver's overall depth image in the cab is input through a depth camera, and an eye-tracking camera is set up to obtain the range of environmental information recorded by the eye based on the eye position. The amount of environmental information obtained by the driver is obtained, and the amount of environmental information covers the information of the first element and the second element.

[0017] When the amount of information is greater than the sum of the environmental information in the first element and the second element, it is determined that the driver is in focused driving mode.

[0018] If the amount of information is less than the set of environmental information in the first element and the second element, the driver is determined to be in non-focused driving mode, and then the system will determine whether the driver has acquired the dangerous elements contained in the set of environmental information in the first element and the second element.

[0019] If yes, then it is considered safe;

[0020] If not, it is deemed dangerous, and an assisted driving system will be activated.

[0021] The first element information includes vehicle outline information, vehicle speed information, weather conditions, vehicle braking distance and performance information;

[0022] The first sub-element information includes the safe braking distance after braking at the current driving speed within the sensor recognition range of the vehicle. The sensor recognition range outside the safe braking distance is divided into multiple first sub-elements.

[0023] The hazardous elements include hazardous elements that are stationary relative to the first element.

[0024] And dangerous elements that move relative to the first element.

[0025] A lane departure avoidance collision method includes the following steps:

[0026] Obtain the lane profile of the first sub-feature in the first feature;

[0027] Uncertain road regions are identified using the fuzzy entropy method, and road regions are extracted using an improved region growing method.

[0028] The final lane line is generated by fitting the main control points at the edge of the lane line, and the first element is used to judge and warn of vehicles deviating from the lane in the mixed structured and unstructured road.

[0029] A three-dimensional visualization detection method includes the following steps:

[0030] First and second elements are obtained, and the threshold of dangerous elements in the first and second elements is obtained through image enhancement algorithm. The binarized depth data is then used to segment the dangerous elements.

[0031] The segmented multiple hazardous elements are mapped into the first element to generate a complete hazardous element region;

[0032] The mapping of hazardous elements and hazardous element regions is obtained through 3D visualization, and the 3D hazardous element regions are then fed back to the hazard early warning system.

[0033] A driver behavior recognition method includes the following steps:

[0034] Obtain driver posture features;

[0035] A one-dimensional Hough voting depth space ellipse detection method based on inverse mapping space growth is used to locate the steering wheel;

[0036] An improved Zhang-Suen algorithm combining hybrid adaptive DBSCAN is used to extract behavioral features from driver depth images;

[0037] FAST feature extraction and localization using multi-mapping connected domain clustering analysis is used to generate driver key points and optimize driver state recognition.

[0038] It also includes the following steps:

[0039] The size of the driver's eye-mapping area and the head rotation direction data at the generated driver's joints are obtained through training.

[0040] The system compares the driver's field of vision with the hazardous elements and the intersection of these hazardous areas to determine whether the driver is in a safe driving state.

[0041] A vision-based driver assistance system includes:

[0042] The detection module includes at least one set of two detection modules, at least one depth camera installed inside the driver's cab, and at least one eye-tracking camera;

[0043] The driver assistance system includes a lane keeping assist system, an adaptive cruise control system, an intelligent parking system, and a blind spot monitoring system.

[0044] Danger warning system.

[0045] The technical effects achieved by this invention are as follows:

[0046] This invention acquires and identifies various factors affecting vehicle driving to plan the vehicle's driving path and achieve assisted driving. By using inverse projection transformation to obtain top views of each sub-element in the current first and second elements, the invention can locate each factor affecting vehicle driving safety. Through the vehicle's assisted driving system, the accurate identification of each factor enables precise planning of the driving route, thereby improving the safety of assisted driving.

[0047] This invention monitors and identifies the driver's state in the cab using a depth camera, and judges the identified driver state to identify and warn of abnormal driver states, or to assist driving by intervening in driving.

[0048] This invention compares the driver's field of vision with the first and second elements to identify areas containing hazardous elements, and compares the intersection between the driver's field of vision and the hazardous element areas. It then compares the current field of vision with the hazardous elements and hazardous element areas, calculates the intersection, and determines whether the intersection covers all hazardous elements or hazardous element areas. This is used to determine whether the driver is in a safe driving state. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the vision recognition-based assisted driving method of the present invention. Figure 1 ;

[0050] Figure 2 This is a flowchart illustrating the vision recognition-based assisted driving method of the present invention. Figure 2 ;

[0051] Figure 3 This is a flowchart illustrating the lane departure avoidance collision method of the present invention;

[0052] Figure 4 This is a flowchart illustrating the three-dimensional visualization detection method of this invention;

[0053] Figure 5 This is a flowchart illustrating the driver behavior recognition method of the present invention;

[0054] Figure 6 This is a schematic diagram of the driver recognition area in this invention. Detailed Implementation

[0055] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0056] like Figure 1 As shown, a vision-based driver assistance method includes the following steps:

[0057] S1. Obtain the first element information and the second element information during the vehicle's driving process;

[0058] S2. By establishing models for the road elements and road edge elements in the second element information, the road edge elements that affect the identification of road elements are identified, which are then used to plan the movement path of the first element between the second element.

[0059] S3. Based on the model established by the road elements and road edge elements, segment the road elements so that each road element is divided into at least two interrelated first sub-elements, and each road edge element is divided into an independent second sub-element.

[0060] S4. Obtain the top view of the first sub-element and the second sub-element through inverse projection transformation, and construct a three-dimensional visualization model of the first and second sub-element by mapping it to the assisted driving system.

[0061] S5. Based on the obtained 3D models of the first and second visual elements, obtain the vehicle's own position model relative to the first element in the first and second elements.

[0062] S6. Based on the elements identified as dangerous in the second element, plan the driving route and avoid them.

[0063] According to the above steps, in step S1, for example, by acquiring information on various elements during vehicle driving, and further by acquiring and identifying various elements affecting vehicle driving through the model established in S2, it is possible to plan the vehicle driving path to achieve assisted driving.

[0064] In step S3, by segmenting the various elements that affect vehicle driving and independently identifying each segmented element, the identification efficiency can be effectively improved. Furthermore, by identifying the interrelated first sub-elements and independent second sub-elements and further cooperating with the model established in step S2 for path planning, the safety level of the driving path of assisted driving can be improved.

[0065] In steps S4-S6, the inverse projection transformation is the reverse application of the projection transformation. The specific application method is existing technology, so it will not be elaborated on here. The inverse projection transformation obtains the top view of each sub-element in the current first element and second element, which is used to locate each element that affects the driving safety of the vehicle. Through the vehicle's assisted driving system, the accurate identification of each element can realize the planning of a precise driving route, further improving the safety of assisted driving.

[0066] See attached document Figure 2 A vision-based driver assistance method further includes the following steps:

[0067] S7. Input the overall depth image of the driver in the cab through a depth camera, and set up an eye-tracking camera to obtain the range of environmental information recorded by the eyes based on the eye position.

[0068] S8. Obtain the amount of environmental information obtained by the driver, covering the information of the first and second elements.

[0069] S9. When the amount of information is greater than the sum of the environmental information in the first element and the second element, it is determined that the driver is in focused driving mode.

[0070] S10. When the amount of information is less than the set of environmental information in the first element and the second element, the driver is determined to be in non-focused driving mode, and then the determination of whether the driver has obtained the dangerous elements contained in the set of environmental information in the first element and the second element is initiated.

[0071] S11. If yes, then it is determined to be safe;

[0072] S12. If not, it is deemed dangerous, and the driver assistance system is activated.

[0073] Based on the above steps, in order to further improve the level of assisted driving, a depth camera is used to monitor and identify the driver's status in the cab, and to judge the identified driver status. This is used to identify and warn the driver in an abnormal state, or to assist driving by intervening in driving.

[0074] Furthermore, in step S7, the depth camera's recognition of the driver's state includes, but is not limited to, driving postures such as nodding, blinking, and body movements, and at least one eye-tracking camera is further set up to identify whether the driver is focused on driving during the driving process;

[0075] In steps S8-S10, the driver's observation status of the first and second elements during driving is obtained through eye tracking in step S7. The range of the driver's observed elements is obtained by observing the direction of the driver's eye and pupil changes. The driver's observation of each element is obtained through identification and judgment of the elements in steps S1-S7. The level of detail of the driver's observation of each element, or whether the observation of dangerous elements is comprehensive, is obtained.

[0076] Among them, the danger elements include the second element being stationary relative to the first element, and the danger elements being in motion relative to the first element. The criteria for determining the danger elements are: any object that poses a danger to a vehicle in motion is a danger element, including obstacles such as vehicles, road signs, signs, and roadblocks that are stationary relative to the first element, and objects that are in motion relative to the first element, such as vehicles, people, animals, and rolling stones.

[0077] Among the aforementioned hazards, road signs or markers used to indicate safety are also considered hazards relative to the driver's judgment if the driver does not pay attention to them; these hazards are those that will be encountered in the future.

[0078] In addition, for the aforementioned hazardous elements, such as road signs and markers, if they are tilted or toppled, they are considered direct hazardous elements.

[0079] In the above steps S1-S12, the first element information includes the vehicle's own outline information, vehicle speed information, weather conditions, vehicle braking distance and performance information.

[0080] Furthermore, the first sub-element information includes the safe braking distance after braking at the current driving speed within the vehicle's sensor recognition range, and the sensor recognition range outside the safe braking distance is divided into multiple first sub-elements.

[0081] Please refer to Figure 3 A lane departure avoidance collision method includes the following steps:

[0082] S101. Obtain the lane outline of the first sub-element in the first element;

[0083] S102. Based on the fuzzy entropy method, determine the uncertain road regions, and use an improved region growing method to extract the road regions.

[0084] S103. The final lane line is generated by fitting the main control points of the lane line edge;

[0085] S104. Determine and issue a warning regarding vehicles deviating from lanes in a mixed structured and unstructured road system, as identified in the first element.

[0086] According to the above steps, in step S101, for the first sub-element of the first element, the lane contour is identified and acquired. The acquisition method is to identify and record it through radar or camera installed on the vehicle.

[0087] Furthermore, in step S102, uncertain road areas are identified using the fuzzy entropy method. The fuzzy entropy of the road area is calculated to determine whether it is an uncertain road area. A higher fuzzy entropy value indicates that there are many uncertain factors in the area, such as shadows, puddles, stains, pebbles, etc.

[0088] Furthermore, an improved region growing method is used to extract road regions, specifically including:

[0089] For a defined but not uncertain road area, an improved growth method is used to extract the road area, avoiding the misclassification of non-road areas as road areas, so as to realize the determination of road area and driving path.

[0090] Steps S103 and S104 generate the final lane line from the master control points fitted to the edge of the lane line, which is used for lane departure warning of vehicles in mixed structured and unstructured roads.

[0091] By quickly and accurately identifying lane lines, lane markings, vehicles, pedestrians, etc., the safety of drivers and passengers can be ensured.

[0092] Please refer to Figure 4 As shown, a three-dimensional visualization detection method includes the following steps:

[0093] S201. Obtain the first element and the second element, and obtain the threshold of the dangerous element in the first element and the second element through an image enhancement algorithm;

[0094] S202. Perform binarization processing on the binocular depth data to segment hazardous elements;

[0095] S203. Map the segmented multiple hazard elements into the first element to generate a complete hazard element region;

[0096] S204. Obtain the mapping of hazardous elements and hazardous element areas through three-dimensional visualization, and then feed it back to the hazard warning system.

[0097] Based on the above steps, since both static and moving hazardous elements may experience a certain degree of state shift or change after the vehicle travels near them, it is necessary to set boundary pre-settings for hazardous elements. In addition, this can also prevent calculation deviations caused by external factors during the calculation process of the assisted driving system, thereby improving the safety of the route planning process.

[0098] As in step S201, the image enhancement algorithm can first improve the image accuracy. On the other hand, for certain elements such as rocks, utility poles, trees, and roadblocks, fuzzy thresholds can be used to achieve fuzzy calculations. For elements with high risk, image enhancement calculations are performed. This method is used to improve the application of computing power in the system operation process, improve the targeting of the operation, and improve the accuracy of the operation on dangerous elements, thereby further improving driving safety.

[0099] Furthermore, setting fuzzy thresholds can increase the ability of vehicles to anticipate dangerous factors and take braking measures in advance during driving.

[0100] Optionally, the image enhancement algorithm uses a contour wave transform technique, which is suitable for images with low contrast and high noise, such as infrared images. By defining a parameterized contrast in the NSCT domain of the contour wave transform technique and enhancing the high-frequency coefficients with a nonlinear gain function, image details can be highlighted and contrast improved.

[0101] In addition, unlike the blur threshold in this embodiment, setting a threshold by estimating the noise level can suppress noise and improve image quality.

[0102] In step S202, by recognizing the same element through at least two cameras and by giving an obstacle determination threshold, the binarized depth data is processed to segment the obstacle from the background, thereby improving the accuracy and robustness of obstacle detection.

[0103] Optionally, the iterative Normalized Cut segmentation method can be used to fuse fragmented and irregular obstacles and generate a complete obstacle region. The iterative Normalized Cut segmentation method can effectively cluster similar obstacle regions and improve the completeness of obstacle detection.

[0104] By identifying each element in step S202 and further mapping each element to the first element, a complete hazardous area is generated.

[0105] Further, the distribution of dangerous areas is determined by the relative regional positions of the hazardous elements mapped in step S203 with the first element. Then, the three-dimensional visualization of the hazardous element area is obtained by mapping the hazardous elements and the hazardous element area through three-dimensional visualization, and further fed back to the early warning system.

[0106] Furthermore, the warning system also includes image display devices, which can prioritize alerting the driver or assisting the driving system by displaying three-dimensional visualized hazardous areas.

[0107] Because of the over-reliance on driver assistance systems, these systems may not be able to ensure safe passage in certain scenarios. In such cases, driver intervention is required, necessitating frequent vehicle operation by the driver to handle special situations.

[0108] Reference Figure 5 As shown, a driver behavior recognition method includes the following steps:

[0109] S301. Obtain the driver's posture features and input the overall depth image of the driver in the cab through a depth camera;

[0110] S302. A one-dimensional Hough voting depth space ellipse detection method based on inverse mapping space growth is used to locate the steering wheel and further extract the driver target.

[0111] S303. An improved Zhang-Suen algorithm combining hybrid adaptive DBSCAN is used to extract behavioral features from driver depth images;

[0112] S304. FAST feature extraction and localization using multi-mapping connected domain clustering analysis generates driver key points, optimizes driver state recognition, and enables real-time identification and warning of dangerous driver behaviors, thereby comprehensively improving driving safety.

[0113] In step S301, a depth camera is used to acquire an overall depth image of the driver inside the cab. This image can provide three-dimensional information about the distance of objects in the scene from the camera, which is helpful for subsequent processing and analysis.

[0114] In step S302, the improved one-dimensional Hough voting depth space ellipse detection method based on inverse mapping space growth is used to locate the steering wheel. Through the inverse mapping space growth technique, the position of the steering wheel in the image can be determined more accurately. One-dimensional Hough voting is a commonly used line detection method, but it is used here for ellipse detection to adapt to the shape features of the steering wheel.

[0115] After locating the steering wheel, the driver is further extracted from the depth image as the target area, and techniques including but not limited to image segmentation and target tracking are used to ensure accurate identification of the driver's position and posture.

[0116] In step S303, the improved Zhang-Suen algorithm of hybrid adaptive DBSCAN is used. This algorithm combines the advantages of DBSCAN (Density-Based Spatial Clustering of Applications with Noise) and the Zhang-Suen algorithm to extract behavioral features from driver depth images. DBSCAN is a density-based spatial clustering algorithm, while the Zhang-Suen algorithm is often used for image segmentation. This hybrid method can more accurately identify and extract driver behavioral features.

[0117] In step S304, FAST (Features from Accelerated Segment Test) is a fast feature point detection algorithm. In this embodiment, it is used to extract key points from the driver image and determine the correlation between these key points through multi-mapping connected domain clustering analysis, which helps to more accurately locate the driver's joints, such as the head, shoulders, elbows, etc.

[0118] Based on the above steps, the features and key information extracted through these steps can be used to more accurately identify the driver's state. For example, it can be used to determine whether the driver is in a normal driving state or whether there are signs of distraction or fatigue.

[0119] The system can monitor changes in driver behavior in real time and issue warnings when it detects potentially dangerous behaviors. This helps to remind drivers to pay attention to safety in a timely manner and avoid traffic accidents. By integrating the above technologies and algorithms, the system can significantly improve driving safety. It can not only reduce the risk of accidents caused by driver negligence or incorrect operation, but also improve the driver's perception of the road environment, thereby making more reasonable driving decisions.

[0120] Please refer to Figure 6 As shown, it also includes the following steps:

[0121] S401. Obtain the size of the driver's eye-mapping area and the head rotation direction data based on the generated driver's joint points through training;

[0122] S402. Based on the driver's field of vision, the comparison of hazardous elements and the intersection of hazardous element areas is used to determine whether the driver is in a safe driving state.

[0123] In step S401, refer to Figure 6First, by measuring the driver's eye-mapping area 'a' when the driver is looking straight ahead without turning their head, and the area 'b' in the figure is the visual field that the driver can recognize when the driver is turning their head, based on the generated head rotation direction data of the driver's key points, and combined with the eye-mapping area 'a' and the area 'b' in the figure when the driver is turning their head, the area 'c' that the driver can recognize when turning their head is obtained.

[0124] In step S402, the driver's field of vision is first compared with the first element and the second element to find the area containing the dangerous element, and the intersection between the driver's field of vision and the dangerous element area is compared. Before the judgment, the driver's current field of vision is first determined to be one of a, b, and c. The current field of vision is compared with the dangerous element and the dangerous element area to calculate the intersection. It is then determined whether the intersection covers all dangerous elements or dangerous element areas to determine whether the driver is in a safe driving state.

[0125] A vision-based driver assistance system includes:

[0126] The detection module includes at least one set of two detection modules, at least one depth camera installed inside the driver's cab, and at least one eye-tracking camera;

[0127] Driver assistance systems include lane keeping assist, adaptive cruise control, intelligent parking, and blind spot monitoring.

[0128] Hazard warning systems are used to alert drivers to potential unidentified hazards.

[0129] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A vision-based driver assistance method, characterized in that, Includes the following steps: Acquire first-factor information and second-factor information during vehicle operation; By establishing models for the road elements and road edge elements in the second element information, the road edge elements that affect the identification of road elements are identified, which are then used to plan the movement path of the first element between the second element. The model is segmented based on road features and road edge features, such that each road feature is divided into at least two interrelated first sub-features, and each road edge feature is divided into an independent second sub-feature. The top view of the first sub-element and the second sub-element is obtained by inverse projection transformation, and a three-dimensional visualization model of the first and second sub-element is constructed by mapping it to the driver assistance system. Based on the obtained 3D models of the first and second visual elements, a position model of the vehicle itself relative to the first element in the first element is obtained. Based on the second element, driving routes are planned and avoidance measures are taken to identify hazardous elements.

2. The visual recognition-based assisted driving method according to claim 1, characterized in that, Also includes: The driver's overall depth image in the cab is input through a depth camera, and an eye-tracking camera is set up to obtain the range of environmental information recorded by the eye based on the eye position. The amount of environmental information obtained by the driver is obtained, and the amount of environmental information covers the information of the first element and the second element. When the amount of information is greater than the sum of the environmental information in the first element and the second element, it is determined that the driver is in focused driving mode. If the amount of information is less than the set of environmental information in the first element and the second element, the driver is determined to be in non-focused driving mode, and then the system will determine whether the driver has acquired the dangerous elements contained in the set of environmental information in the first element and the second element. If yes, then it is considered safe; If not, it is deemed dangerous, and an assisted driving system will be activated.

3. A vision-based assisted driving method according to any one of claims 1 or 2, characterized in that: The first element information includes vehicle outline information, vehicle speed information, weather conditions, vehicle braking distance and performance information; The first sub-element information includes the safe braking distance after braking at the current driving speed within the sensor recognition range of the vehicle. The sensor recognition range outside the safe braking distance is divided into multiple first sub-elements.

4. The visual recognition-based assisted driving method according to claim 2, characterized in that: The hazardous elements include hazardous elements that are stationary relative to the first element. And dangerous elements that move relative to the first element.

5. The visual recognition-based assisted driving method according to claim 1, characterized in that, The method includes a lane departure avoidance collision method, which comprises the following steps: Obtain the lane profile of the first sub-feature in the first feature; Uncertain road regions are identified using the fuzzy entropy method, and road regions are extracted using an improved region growing method. The final lane line is generated by fitting the main control points at the edge of the lane line, and the first element is used to judge and warn of vehicles deviating from the lane in the mixed structured and unstructured road.

6. The visual recognition-based assisted driving method according to claim 1, characterized in that, This includes a three-dimensional visualization detection method, which comprises the following steps: First and second elements are obtained, and the threshold of dangerous elements in the first and second elements is obtained through image enhancement algorithm. The binarized depth data is then used to segment the dangerous elements. The segmented multiple hazardous elements are mapped into the first element to generate a complete hazardous element region; The mapping of hazardous elements and hazardous element regions is obtained through 3D visualization, and the 3D hazardous element regions are then fed back to the hazard early warning system.

7. The visual recognition-based assisted driving method according to claim 1, characterized in that, The method includes a driver behavior recognition method, which comprises the following steps: Obtain driver posture features; A one-dimensional Hough voting depth space ellipse detection method based on inverse mapping space growth is used to locate the steering wheel; An improved Zhang-Suen algorithm combining hybrid adaptive DBSCAN is used to extract behavioral features from driver depth images; FAST feature extraction and localization using multi-mapping connected domain clustering analysis is used to generate driver key points and optimize driver state recognition.

8. The visual recognition-based assisted driving method according to claim 7, characterized in that, The driver behavior recognition method further includes the following steps: The size of the driver's eye-mapping area and the head rotation direction data at the generated driver's joints are obtained through training. The system compares the driver's field of vision with the hazardous elements and the intersection of these hazardous areas to determine whether the driver is in a safe driving state.

9. A vision-based driver assistance system, characterized in that, include: The detection module includes at least one set of two detection modules, at least one depth camera installed inside the driver's cab, and at least one eye-tracking camera; The driver assistance system includes a lane keeping assist system, an adaptive cruise control system, an intelligent parking system, and a blind spot monitoring system. Danger warning system; The driver assistance system is used to implement the method as described in any one of claims 1-8.

Images