Auxiliary driving system and method based on visual identification

Abstract

The invention belongs to the technical field of auxiliary driving, and particularly relates to an auxiliary driving system and method based on visual identification, and the method comprises the steps: obtaining first element information and second element information in a vehicle driving process; respectively establishing models for the road elements and the road edge elements in the second element information, and identifying the road edge elements influencing the road element identification so as to plan a motion path of the first element between the second elements; segmenting according to a model established by the road elements and the road edge elements, so that the road elements are segmented into at least two first sub-elements which are associated with each other, and the road edge elements respectively establish independent second sub-elements. According to the invention, driving safety guarantee and detection perception capability of mixed road lane lines and lane boundaries in different interference environments can be provided, judgment and guidance of a driving safety state are provided for a driver, driving intensity of the driver is reduced, and driving safety and traffic efficiency are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of assisted driving, and in particular relates to an assisted driving system and method based on visual recognition. Background Art

[0002] An assisted driving system (ADAS) based on visual recognition is a system that uses machine vision technology to perceive and understand the vehicle's driving environment, thereby assisting the driver in making decisions and operations. The system obtains image data of the surrounding environment through visual sensors such as on-board cameras, and uses computer vision, machine learning and other algorithms to process and analyze this data to achieve detection and recognition of information such as roads, traffic signs, pedestrians, and obstacles.

[0003] In order to improve the driving safety of motor vehicles and effectively avoid traffic accidents, vehicle driving assistance systems have become a research hotspot in the current intelligent transportation field.

[0004] Problems with existing technologies: However, the current vehicle driving assistance system cannot reliably provide driving safety protection in low light, when lane lines are blurred and damaged, when road shape is unclear, when obstacles in the vehicle environment are complex, and when multiple factors interfere with the driver's operation. Summary of the invention

[0005] The purpose of the present invention is to provide an assisted driving system and method based on visual recognition, which can provide driving safety protection, the detection and perception capabilities of mixed road lane lines and lane boundaries under different interference environments, provide the driver with judgment and guidance on the driving safety status, reduce the driver's driving intensity, and improve driving safety and traffic efficiency.

[0006] The technical solution adopted by the present invention is as follows: An assisted driving method based on visual recognition comprises the following steps: Acquire the first element information and the second element information during the vehicle driving process; By respectively establishing models for the road elements and road edge elements in the second element information, the road edge elements that affect the identification of the road elements are identified, and used to plan the movement path of the first element between the second elements; Segmentation is performed according to the model established for the road element and the road edge element, so that the road element is segmented into at least two mutually related first sub-elements, and the road edge elements are respectively established into independent second sub-elements; A top view of the first sub-element and the second sub-element is obtained through inverse projection transformation, and a three-dimensional visualization model of the first element and the second element is constructed by mapping them to the auxiliary driving system; Based on the obtained three-dimensional models of the visualized first element and the second element, obtain a vehicle position model of the first element relative to the first element and the second element; The driving route is planned and avoided based on the dangerous factors determined among the second factors.

[0007] Also includes: An overall depth image of the driver in the cab is inputted through a depth camera, and an eye tracking camera is set to obtain the range of environmental information recorded by the eyeball based on the eyeball position, and obtain the amount of environmental information obtained by the driver, wherein the amount of environmental information covers the amount of information of the first element and the second element; When the amount of information is greater than the sum of the amounts of environmental information in the first factor and the second factor, it is determined that the driver is in a focused driving mode; When the amount of information is less than the set of environmental confidence amounts in the first factor and the second factor, it is determined that the driver is in a non-focused driving mode, and then it is started to determine whether the driver has obtained the dangerous factors included in the set of environmental information amounts in the first factor and the second factor; If yes, it is considered safe; If not, it is judged as dangerous and the auxiliary driving system is used.

[0008] The first element information includes the vehicle's own profile information, vehicle speed information, weather conditions, vehicle braking distance and performance information; The first sub-element information includes a safe braking distance after braking at the current speed of the vehicle within a sensor recognition range set for the vehicle, and the sensor recognition range outside the safe braking distance is divided into a plurality of first sub-elements.

[0009] The risk factor comprises a risk factor in which the second factor is stationary relative to the first factor, and the dangerous element of relative motion with respect to the first element.

[0010] A lane departure collision avoidance method comprises the following steps: Get the lane profile of the first sub-element in the first element; The uncertain road area is judged based on the fuzzy entropy method, and the road area is extracted using the improved region growing method. The final lane line is generated by fitting the main control points of the lane line edge, and a judgment and warning is made on the vehicle deviation from the lane in the structured and unstructured mixed road in the first element.

[0011] A three-dimensional visualization detection method comprises the following steps: Obtaining the first element and the second element, obtaining the threshold of the dangerous element in the first element and the second element through an image enhancement algorithm, and performing binarization processing on the binocular depth data to segment the dangerous element; Mapping the segmented plurality of hazardous elements into the first element to generate a complete hazardous element region; The three-dimensional hazardous element area is obtained by mapping hazardous elements and hazardous element areas through three-dimensional visualization, and is fed back to the hazardous warning system.

[0012] A driver behavior recognition method comprises the following steps: Acquire driver posture characteristics; One-dimensional Hough voting deep space ellipse detection method based on inverse mapping space growth to locate the steering wheel; The improved Zhang-Suen algorithm combined with hybrid adaptive DBSCAN is used to extract the behavioral features of the driver in the depth image; The FAST feature extraction and positioning of multi-map connected domain clustering analysis is applied to generate driver joint points and optimize the driver's state recognition.

[0013] The following steps are also included: The size of the driver's eyeball mapping area and the head rotation direction data in the generated driver's joint points are obtained through training; The intersection relationship between the dangerous factors and the dangerous factor areas is compared according to the driver's field of vision capture area to determine whether the driver is in a safe driving state.

[0014] An assisted driving system based on visual recognition, comprising: A detection module, the detection module comprising at least one set of two detection modules, at least one depth camera mounted inside the cab, and at least one eye tracking camera; Assisted driving system, including lane keeping assist system, adaptive cruise control system, smart parking system, blind spot monitoring system; Hazard warning system.

[0015] The technical effects achieved by the present invention are: The present invention acquires and identifies various factors that affect vehicle driving to achieve vehicle driving path planning to achieve assisted driving. A top view of each sub-element in the current first element and second element is acquired through inverse projection transformation to locate each factor that affects vehicle driving safety. Through the assisted driving system carried by the vehicle, accurate identification of each factor can achieve accurate driving route planning and improve the safety of assisted driving.

[0016] The present invention monitors the driver's state in the cab through a depth camera, identifies the driver's state, and judges the identified driver's state, so as to identify and warn the driver in an abnormal state, or assist driving by intervening in driving.

[0017] The present invention compares the driver's field of vision capture area with the dangerous element area between the first element and the second element, and compares the intersection between the driver's field of vision capture area and the dangerous element area, uses the current field of vision area to compare with the dangerous element and the dangerous element area, calculates the intersection, and determines whether the intersection covers all dangerous elements or dangerous element areas, which is used to determine whether the driver is in a safe driving state. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the process of the assisted driving method based on visual recognition in the present invention. Figure 1 ; Figure 2 This is a schematic diagram of the process of the assisted driving method based on visual recognition in the present invention. Figure 2 ; Figure 3 is a flow chart of the lane departure collision avoidance method of the present invention; Figure 4 It is a schematic diagram of the process of the three-dimensional visualization detection method in the present invention; Figure 5 It is a flow chart of the driver behavior identification method in the present invention; Figure 6 It is a structural schematic diagram of the driver identification area in the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0020] like Figure 1 As shown, a method for assisting driving based on visual recognition includes the following steps: S1. Acquire first element information and second element information during vehicle driving; S2, by respectively establishing models for the road elements and road edge elements in the second element information, identifying the road edge elements that affect the identification of the road elements, and using them to plan the movement path of the first element between the second elements; S3, segmenting the road element and the road edge element according to the model established, so that the road element is segmented into at least two mutually related first sub-elements, and the road edge element is respectively established into independent second sub-elements; S4. Obtaining a top view of the first sub-element and the second sub-element through inverse projection transformation, and constructing a three-dimensional visualization model of the first element and the second element by mapping them to the auxiliary driving system; S5. Based on the obtained three-dimensional models of the visualized first element and the second element, obtain a vehicle position model relative to the first element of the first element and the second element; S6. Plan a driving route and avoid dangerous factors based on the second factors.

[0021] According to the above steps, in step S1, for example, by acquiring information of various elements during the vehicle's driving process, and further acquiring and identifying various elements that affect the vehicle's driving through the model established in S2, the vehicle's driving path can be planned to achieve assisted driving.

[0022] In step S3, by segmenting the acquired elements that affect vehicle driving and independently identifying and processing the segmented elements, the efficiency of identification can be effectively improved. Furthermore, by identifying and processing the interrelated first sub-elements and the independent second sub-elements, and further cooperating with the model established in step S2 for path planning, the safety of the assisted driving path can be improved.

[0023] In steps S4-S6, the inverse projection transformation is achieved by the reverse application of the projection transformation. The specific application method is a prior art, so it will not be described in detail here. The top view of the current first element and each sub-element in the second element is obtained through the inverse projection transformation, which is used to realize the positioning of each element that affects the driving safety of the vehicle. Through the assisted driving system carried by the vehicle, the precise identification of each element can realize the planning of accurate driving routes, thereby further improving the safety of assisted driving.

[0024] See attached Figure 2 , an assisted driving method based on visual recognition, further comprising the following steps: S7, input the overall depth image of the driver in the cab through the depth camera, and set the eye tracking camera to obtain the range of environmental information recorded by the eye based on the eye position, S8, obtaining the amount of environmental information obtained by the driver, where the amount of environmental information covers the amount of information of the first element and the second element; S9, when the amount of information is greater than the sum of the amount of environmental information in the first factor and the second factor, it is determined that the driver is in a focused driving mode; S10, when the amount of information is less than the set of environmental confidence amounts in the first factor and the second factor, it is determined that the driver is in a non-focused driving mode, and then it is started to determine whether the driver has obtained the dangerous factors included in the set of environmental information amounts in the first factor and the second factor; S11: Yes, it is determined to be safe; S12: No, it is judged as dangerous and the auxiliary driving system is used.

[0025] According to the above steps, in order to further improve the level of assisted driving, the driver's status in the cab is monitored and identified through a depth camera, and the identified driver's status is judged to identify and warn the driver in an abnormal state, or to assist driving by intervening in driving.

[0026] Furthermore, in step S7, the depth camera recognizes the driver's state including but not limited to: driving postures such as nodding, blinking, and body movements, and further sets at least one eye tracking camera to identify whether the driver is in a focused driving state during driving; In steps S8-S10, the driver's observation status of the first element and the second element during driving is obtained by tracking the eyeballs in step S7, the range of the driver's observation elements is obtained by observing the direction of the driver's eyeballs and pupil changes, and the driver's observation level of each element is obtained by identifying and judging the elements in steps S1-S7, or whether the driver's observation of dangerous elements is comprehensive.

[0027] Among them, the dangerous factors include the dangerous factors that the second factor is stationary relative to the first factor, and the dangerous factors that move relatively with the first factor. The dangerous factor judgment criteria are: objects that may cause danger to the vehicle during driving are dangerous factors, including obstacles such as vehicles, road signs, signs, roadblocks, etc. that are stationary relative to the first factor, and objects that move relatively with the first factor, such as vehicles, people, animals, rolling stones, etc.

[0028] Among the above-mentioned dangerous factors, road signs or symbols used to remind safety are also dangerous factors relative to the driver's judgment when the driver is not paying attention. This dangerous factor is something that may be encountered in the future.

[0029] In addition, for the above-mentioned dangerous elements, such as road signs and symbols, if they are tilted or collapsed, they will be judged as direct dangerous elements.

[0030] In the above steps S1-S12, the first element information includes the vehicle's own profile information, vehicle speed information, weather conditions, vehicle braking distance and performance information.

[0031] Furthermore, the first sub-element information includes a safe braking distance after braking at the current speed of the vehicle within a recognition range of a sensor set for the vehicle, and the recognition range of the sensor outside the safe braking distance is divided into a plurality of first sub-elements.

[0032] Please refer to Figure 3 , a lane departure collision avoidance method, comprising the following steps: S101, obtaining a lane contour of a first sub-element in a first element; S102, judging the uncertain road area based on the fuzzy entropy method, and extracting the road area using the improved region growing method, S103, generating a final lane line by fitting the main control points of the lane line edge; S104: Determine and warn if the vehicle in the first element deviates from the lane on the structured and unstructured mixed road According to the above steps, in step S101, for the first sub-element in the acquired first element, the lane contour therein is identified and acquired, and the acquisition method is to identify and enter it through a radar or a camera installed on the vehicle.

[0033] Furthermore, in step S102, the uncertain road area is determined by the fuzzy entropy method, and the fuzzy entropy of the road area is calculated to determine whether the area is an uncertain road area. If the fuzzy entropy value is high, it indicates that there are many uncertain factors in the area, which may be due to shadows, puddles, stains, stones, etc.: Furthermore, an improved region growing method is used to extract the road area, including: For the determined non-uncertain road area, the improved and growing method is used to extract the road area to avoid judging the non-road area as the road area, so as to realize the judgment of the road area and the driving path.

[0034] Step S103 and step S104 generate the final lane line by fitting the main control points of the lane line edge, which is used for lane departure warning of the vehicle on structured and unstructured mixed roads.

[0035] Ensure the safety of drivers and passengers by quickly and accurately identifying lane lines, lane markings, vehicles, pedestrians, etc.

[0036] Please refer to Figure 4 As shown, a three-dimensional visualization detection method includes the following steps: S201, obtaining a first element and a second element, and obtaining a threshold value of a dangerous element in the first element and the second element by using an image enhancement algorithm; S202, binarizing the binocular depth data to segment the dangerous elements; S203, mapping the segmented multiple hazardous elements into the first element to generate a complete hazardous element area; S204, obtaining a mapping of hazardous elements and hazardous element areas through three-dimensional visualization to obtain a three-dimensional hazardous element area, and feeding it back to the hazardous warning system.

[0037] According to the above steps, due to the stationary dangerous elements or the dangerous elements in motion, after the vehicle travels near the dangerous elements, the dangerous elements will have a certain degree of state deviation or the state of the dangerous elements will change. Therefore, it is necessary to set the boundary prefabrication for the dangerous elements. In addition, it can also avoid the calculation deviation caused by the influence of external factors during the calculation process of the auxiliary driving system, thereby improving the safety of the route during the planning process.

[0038] As shown in step S201, the image enhancement algorithm can firstly improve the image accuracy. On the other hand, for certain elements such as stones, telephone poles, trees, roadblocks, etc., the fuzzy threshold can be used to realize fuzzy calculation, and image enhancement calculation can be performed on elements with higher danger. This method is used to improve the application of computing power in the system operation process, improve the targetedness of the operation, and improve the accuracy of the operation on dangerous elements, thereby further improving driving safety.

[0039] Furthermore, the setting of the fuzzy threshold can increase the ability of the vehicle to make early judgments on dangerous factors and take braking measures during driving.

[0040] Optionally, the image enhancement algorithm uses a contourlet transform technique, which is suitable for low-contrast and noisy images such as infrared images. By defining a contrast with parameters in the contourlet transform technique NSCT domain and performing nonlinear gain function enhancement on the high-frequency coefficients, image details can be highlighted and contrast can be improved.

[0041] In addition, different from the blur threshold in this embodiment, the threshold is set by estimating the noise level, which can suppress noise and improve image quality.

[0042] In step S202, at least two cameras recognize the same element, and by giving an obstacle determination threshold, binarize the binocular depth data, so as to segment the obstacle from the background, so as to improve the accuracy and robustness of obstacle detection.

[0043] Optionally, an iterative Normalized Cut segmentation method is used to fuse fragmented and irregular obstacles and generate a complete obstacle region. The iterative Normalized Cut segmentation method can effectively cluster similar obstacle regions and improve the integrity of obstacle detection.

[0044] By identifying each element in step S202 and further mapping each element to the first element, a complete danger zone is generated.

[0045] The distribution of the dangerous area is further determined by the relative regional position of the dangerous elements mapped in step S203 with the first element. The dangerous elements and the dangerous element areas are further mapped through three-dimensional visualization to obtain a three-dimensional visualized dangerous element area, which is further fed back to the early warning system.

[0046] Furthermore, the early warning system also includes an image display device, which can give priority to reminding the driver to operate or assisting the driving system to operate by displaying the three-dimensional visual danger factor area.

[0047] Due to over-reliance on the assisted driving system, when encountering some specific scenarios, the assisted driving system cannot completely achieve safe passage. At this time, the driver's operation is required. Therefore, the driver needs to frequently perform vehicle driving operations to deal with special situations.

[0048] Reference Figure 5 As shown, a driver behavior recognition method comprises the following steps: S301, acquiring the posture features of the driver, and inputting the overall depth image of the driver in the cab through a depth camera; S302, positioning the steering wheel using a one-dimensional Hough voting deep space ellipse detection method based on inverse mapping space growth, and further extracting the driver target; S303, extracting behavioral features from the driver's depth image by combining the improved Zhang-Suen algorithm with hybrid adaptive DBSCAN; S304. Apply FAST feature extraction and positioning of multi-mapped interconnected domain cluster analysis to generate driver joint points, optimize driver status recognition, identify and warn the driver's dangerous behavior in real time, and comprehensively improve driving safety.

[0049] In step S301, a depth camera is used to obtain an overall depth image of the driver in the cab. This image can provide three-dimensional information about the distance between objects in the scene and the camera, which is helpful for subsequent processing and analysis.

[0050] In step S302, an improved one-dimensional Hough voting deep space ellipse detection method of inverse mapping space growth is used to locate the steering wheel. Through the inverse mapping space growth technology, the position of the steering wheel in the image can be determined more accurately. One-dimensional Hough voting is a commonly used straight line detection method, but it is used for ellipse detection here to adapt to the shape characteristics of the steering wheel.

[0051] After locating the steering wheel, the driver is further extracted from the depth image as the target area, using technologies including but not limited to image segmentation and target tracking to ensure accurate identification of the driver's position and posture.

[0052] In step S303, an improved Zhang-Suen algorithm of hybrid adaptive DBSCAN is used, which combines the advantages of DBSCAN (Density-Based Spatial Clustering of Applications with Noise) and Zhang-Suen algorithm to extract the behavioral features of the driver in the depth image. DBSCAN is a density-based spatial clustering algorithm, and Zhang-Suen algorithm is often used for image segmentation. This hybrid method can more accurately identify and extract the behavioral features of the driver.

[0053] In step S304, FAST (Features from Accelerated Segment Test) is a fast feature point detection algorithm. In this embodiment, it is used to extract key points from the driver image and determine the correlation between these key points through multi-mapping connected domain clustering analysis, which helps to more accurately locate the driver's joint points, such as the head, shoulders, elbows, etc.

[0054] According to the above steps, the features and key point information extracted through the above steps can be used to more finely identify the driver's status. For example, it can be determined whether the driver is in a normal driving state, whether there are signs of distraction or fatigue, etc.

[0055] The system can monitor changes in driver behavior in real time and issue warnings when potential dangerous behavior is detected, which helps to remind the driver to pay attention to safety in a timely manner and avoid traffic accidents. By integrating the above technologies and algorithms, the system can significantly improve driving safety. It can not only reduce the risk of accidents caused by driver negligence or incorrect operation, but also improve the driver's perception of the road environment, thereby making more reasonable driving decisions.

[0056] Please refer to Figure 6 As shown, the following steps are also included: S401, obtaining the size of the driver's eyeball mapping area through training, and the head rotation direction data based on the generated driver's joint points; S402: Compare the intersection relationship between the dangerous elements and the dangerous element areas according to the driver's field of vision capture area to determine whether the driver is in a safe driving state.

[0057] In step S401, refer to Figure 6First, the driver's eye mapping area a is measured when the driver does not turn his head and looks straight ahead. In the figure, b is the field of vision that the driver can recognize when the driver turns his view. Based on the generated data on the driver's joint-hour head rotation direction, combined with the eye mapping area a and area b in the figure when the driver turns his view, the area c that can be recognized when the driver turns his head is obtained.

[0058] In step S402, the driver's field of vision capture area is first compared with the dangerous element area between the first element and the second element, and the intersection between the driver's field of vision capture area and the dangerous element area is compared. Before judgment, the driver's current field of vision is first determined to be one of a, b, and c. The current field of vision is compared with the dangerous elements and the dangerous element areas, and the intersection is calculated to determine whether the intersection covers all dangerous elements or dangerous element areas, so as to determine whether the driver is in a safe driving state.

[0059] An assisted driving system based on visual recognition, comprising: A detection module, the detection module comprising at least one set of two detection modules, at least one depth camera mounted inside the cab, and at least one eye tracking camera; Assisted driving system, including lane keeping assist system, adaptive cruise control system, smart parking system, blind spot monitoring system; The hazard warning system is used to remind the driver of unidentified dangerous factors.

[0060] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. An assisted driving method based on visual recognition, characterized in that: The steps include: Acquire the first element information and the second element information during the vehicle driving process; By respectively establishing models for the road elements and road edge elements in the second element information, the road edge elements that affect the identification of the road elements are identified, and used to plan the movement path of the first element between the second elements; Segmentation is performed according to the model established for the road element and the road edge element, so that the road element is segmented into at least two mutually related first sub-elements, and the road edge elements are respectively established into independent second sub-elements; A top view of the first sub-element and the second sub-element is obtained through inverse projection transformation, and a three-dimensional visualization model of the first element and the second element is constructed by mapping them to the auxiliary driving system; Based on the obtained three-dimensional models of the visualized first element and the second element, obtain a vehicle position model of the first element relative to the first element and the second element; The driving route is planned and avoided based on the dangerous factors determined among the second factors.

2. The assisted driving method based on visual recognition according to claim 1, characterized in that: Also includes: An overall depth image of the driver in the cab is inputted through a depth camera, and an eye tracking camera is set to obtain the range of environmental information recorded by the eyeball based on the eyeball position, and obtain the amount of environmental information obtained by the driver, wherein the amount of environmental information covers the amount of information of the first element and the second element; When the amount of information is greater than the sum of the amounts of environmental information in the first factor and the second factor, it is determined that the driver is in a focused driving mode; When the amount of information is less than the set of environmental confidence amounts in the first factor and the second factor, it is determined that the driver is in a non-focused driving mode, and then it is started to determine whether the driver has obtained the dangerous factors included in the set of environmental information amounts in the first factor and the second factor; If yes, it is considered safe; If not, it is judged as dangerous and the auxiliary driving system is used.

3. A method for assisting driving based on visual recognition according to any one of claims 1 or 2, characterized in that: The first element information includes the vehicle's own profile information, vehicle speed information, weather conditions, vehicle braking distance and performance information; The first sub-element information includes a safe braking distance after braking at the current speed of the vehicle within a sensor recognition range set for the vehicle, and the sensor recognition range outside the safe braking distance is divided into a plurality of first sub-elements.

4. The assisted driving method based on visual recognition according to claim 2, characterized in that: The risk factor comprises a risk factor in which the second factor is stationary relative to the first factor, and the dangerous element of relative motion with respect to the first element.

5. A lane departure collision avoidance method, characterized in that: The steps include: Get the lane profile of the first sub-element in the first element; The uncertain road area is judged based on the fuzzy entropy method, and the road area is extracted using the improved region growing method. The final lane line is generated by fitting the main control points of the lane line edge, and a judgment and warning is made on the vehicle deviation from the lane in the structured and unstructured mixed road in the first element.

6. A three-dimensional visualization detection method, characterized in that: The steps include: Obtaining the first element and the second element, obtaining the threshold of the dangerous element in the first element and the second element through an image enhancement algorithm, and performing binarization processing on the binocular depth data to segment the dangerous element; Mapping the segmented plurality of hazardous elements into the first element to generate a complete hazardous element region; The three-dimensional hazardous element area is obtained by mapping hazardous elements and hazardous element areas through three-dimensional visualization, and is fed back to the hazardous warning system.

7. A driver behavior recognition method, characterized in that: The steps include: Acquire driver posture characteristics; One-dimensional Hough voting deep space ellipse detection method based on inverse mapping space growth to locate the steering wheel; The improved Zhang-Suen algorithm combined with hybrid adaptive DBSCAN is used to extract the behavioral features of the driver in the depth image; The FAST feature extraction and positioning of multi-map connected domain clustering analysis is applied to generate driver joint points and optimize the driver's state recognition.

8. A driver behavior recognition method according to claim 7, characterized in that: The following steps are also included: The size of the driver's eyeball mapping area and the head rotation direction data in the generated driver's joint points are obtained through training; The intersection relationship between the dangerous factors and the dangerous factor areas is compared according to the driver's field of vision capture area to determine whether the driver is in a safe driving state.

9. An assisted driving system based on visual recognition, characterized in that: include: A detection module, the detection module comprising at least one set of two detection modules, at least one depth camera mounted inside the cab, and at least one eye tracking camera; Assisted driving system, including lane keeping assist system, adaptive cruise control system, smart parking system, blind spot monitoring system; Hazard warning system.

Images