Automatic calibration method and device for distraction area, road vehicle, and electronic equipment

Abstract

The present invention discloses a method and device for automatically calibrating distraction zones, as well as road vehicles and electronic equipment. The automatic calibration method includes: collecting multiple facial images of drivers within a current vehicle within a preset time period; combining the multiple facial images to determine the driver's normal driving angle within the current vehicle; and calibrating the driver's non-distracting and distracting zones within the current vehicle based on the normal driving angle and a predetermined critical distraction deflection angle for the current vehicle. This invention addresses the technical problem in related technologies where distraction zones change due to driving posture or driver changes, while distraction detection still uses a fixed distraction zone, leading to false detections.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method and device for automatically calibrating a distraction area, a road vehicle, and an electronic device. Background Art

[0002] In related technologies, drivers are often distracted by fatigue, external factors, and other factors while driving, which can easily lead to traffic accidents. Therefore, it is necessary to monitor whether the driver is distracted. Before monitoring whether the driver is distracted, it is necessary to quickly locate the distraction area of ​​each vehicle. The current calibration of distraction areas is easily affected by differences in the driver (driving posture or personnel changes), driving habits, and each vehicle model, resulting in large calibration errors. In addition, the current distraction area calibration method usually uses a fixed area threshold to calibrate the distraction area, which is prone to errors in the calibrated distraction area due to driving posture or personnel changes.

[0003] The current calibration method uses eye movement data to detect driver distraction in real time, providing early warning of driver distraction and effectively improving road safety. However, this method, which relies on traditional machine learning, suffers from poor accuracy due to the influence of lighting conditions and individual driver differences. Furthermore, it fails to account for specific real-world scenarios, resulting in a high incidence of false positives.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] Embodiments of the present invention provide a method and apparatus for automatically calibrating a distraction area, a road vehicle, and an electronic device, to at least address the technical problem in related technologies of misdetection caused by changes in the distraction area due to driving posture or driver changes, while distraction detection still uses a fixed distraction area.

[0006] According to one aspect of an embodiment of the present invention, a method for automatically calibrating distraction areas is provided, comprising: collecting multiple facial images of a driver contained in a current vehicle within a preset time period; determining a normal driving angle of the driver in the current vehicle based on the multiple facial images; and calibrating a non-distracting area and a distracting area of ​​the driver in the current vehicle based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle.

[0007] Optionally, the normal driving angle of the driver in the current vehicle is determined in combination with the multiple facial images, including: determining the driver's abnormal driving state in combination with abnormal driving information, and eliminating images corresponding to the abnormal driving state from all facial images to obtain a normal driving image set; and counting and updating the normal driving angle based on the normal driving image set.

[0008] Optionally, before determining the normal driving angle of the driver in the current vehicle in combination with the multiple facial images, the method further includes: initializing the normal driving angle, including: initializing the normal driving angle using a factory preset value; or initializing the normal driving angle using a first sight angle when the driver looks at a first marker point.

[0009] Optionally, the abnormal driving information includes at least one of the following: low vehicle speed, turn signal triggering, distraction deflection, and grip strength.

[0010] Optionally, determining the abnormal driving state of the driver in combination with the low vehicle speed includes: collecting the vehicle speed of the current vehicle; if the vehicle speed is lower than a preset speed threshold, determining that the driver is in an abnormal driving state.

[0011] Optionally, the abnormal driving state of the driver is determined in combination with the turn signal trigger, including: collecting the signal trigger state of the turn signal of the current vehicle; if the signal trigger state indicates that the turn signal has not been triggered, determining that the driver is in a normal driving state; if the signal trigger state indicates that the turn signal has been triggered, determining that the current vehicle is in a turning state, and determining that the driver is in an abnormal driving state.

[0012] Optionally, the abnormal driving state of the driver is determined in combination with distraction deflection, including: collecting the driver's face and sight angle; counting the time the driver's face and sight angle are in a preset abnormal driving area; if the time in the abnormal driving area reaches a first time threshold, it is determined that the driver is in an abnormal driving state.

[0013] Optionally, the abnormal driving state of the driver is determined in combination with grip strength, including: collecting the driver's steering wheel grip strength; if the steering wheel grip strength is lower than a preset grip strength threshold, determining that the driver is in an abnormal driving state.

[0014] Optionally, the normal driving angle is counted and updated according to the normal driving image set, including: each image in the normal driving image set is passed through a face angle and sight line angle model to output the normal face and sight line angle values ​​corresponding to each image; all the normal face and sight line angle values ​​are counted to update the normal driving angle.

[0015] Optionally, the predetermined critical distraction deflection angle of the current vehicle includes: calibrating a non-distracting marked area and a distracting marked area inside the current vehicle according to a preset area of ​​interest, wherein the non-distracting marked area includes: a normal gaze identification point, and the distracting area includes at least: a boundary identification point; collecting images of multiple drivers facing the normal gaze identification point and the boundary identification point respectively to obtain normal gaze images and distracted gaze images; analyzing the normal gaze images and the distracted gaze images to obtain the critical distraction deflection angle of the distracting area of ​​the current vehicle.

[0016] Optionally, the non-distracting marked area and the distracting marked area inside the current vehicle are calibrated according to a preset area of ​​interest, including: characterizing the preset area of ​​interest as the non-distracting marked area inside the current vehicle; determining the center point of the non-distracting marked area to obtain the normal gaze identification point; determining multiple boundaries of the non-distracting marked area, and determining the distracting marked area outside the non-distracting marked area with each boundary as an edge, wherein any point on the boundary is characterized as the boundary identification point.

[0017] Optionally, analyzing the normal gaze image and the distracted gaze image to obtain the critical distraction deflection angle of the distraction area of ​​the current vehicle includes: analyzing the normal gaze image to determine the sight angle of the normal gaze identification point, and obtaining a first normal driving angle based on the distribution of the sight angle of the normal gaze identification point; analyzing all the distracted gaze images to obtain the distribution of the critical distraction driving angles of the distraction area, and calculating the average of the critical distraction driving angles of the distraction area based on the distribution of the critical distraction driving angles of the distraction area; and calculating the difference between the first normal driving angle and the average of the critical distraction driving angles to obtain all the critical distraction deflection angles of the current vehicles.

[0018] Optionally, based on the normal driving angle and the predetermined critical distraction deflection angle of the current vehicle, the non-distracting area and distracting area of ​​the driver in the current vehicle are calibrated, including: adding the critical distraction deflection angle of the current vehicle on the basis of the normal driving angle to obtain the critical position of the distraction area boundary; calibrating the area included in the critical position of the distraction area boundary as the non-distracting area, and calibrating the area outside the area included in the critical position of the distraction area boundary as the distraction area.

[0019] According to another aspect of an embodiment of the present invention, an automatic calibration device for distraction areas is provided, including: an acquisition unit for acquiring multiple facial images of drivers in a current vehicle within a preset time period; a determination unit for determining a normal driving angle of the driver in the current vehicle based on the multiple facial images; and a calibration unit for calibrating a non-distraction area and a distraction area of ​​the driver in the current vehicle based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle.

[0020] Optionally, the determination unit includes: a first determination module, used to determine the abnormal driving state of the driver in combination with abnormal driving information, and eliminate images corresponding to the abnormal driving state from all facial images to obtain a normal driving image set; an update module, used to count and update the normal driving angle based on the normal driving image set.

[0021] Optionally, the automatic calibration device also includes: an initialization unit, used to initialize the normal driving angle before determining the normal driving angle of the driver in the current vehicle in combination with the multiple facial images, the initialization unit including: a first initialization module, used to initialize the normal driving angle using a factory preset value; or, a second initialization module, used to initialize the normal driving angle using a first line of sight angle when the driver looks at a first marker point.

[0022] Optionally, the abnormal driving information includes at least one of the following: low vehicle speed, turn signal triggering, distraction deflection, and grip strength.

[0023] Optionally, the first determination module includes: a first acquisition submodule, used to collect the vehicle speed of the current vehicle; and a first determination submodule, used to determine that the driver is in an abnormal driving state when the vehicle speed is lower than a preset speed threshold.

[0024] Optionally, the first determination module includes: a second acquisition submodule, used to collect the signal trigger status of the turn signal of the current vehicle; a second determination submodule, used to determine that the driver is in a normal driving state when the signal trigger status indicates that the turn signal has not been triggered; and a third determination submodule, used to determine that the current vehicle is in a turning state and that the driver is in an abnormal driving state when the signal trigger status indicates that the turn signal has been triggered.

[0025] Optionally, the first determination module includes: a third acquisition submodule, used to collect the driver's face and sight angle; a first statistics submodule, used to count the time the driver's face and sight angle are in a preset abnormal driving area; and a fourth determination submodule, used to determine that the driver is in an abnormal driving state when the time in the abnormal driving area reaches a first time threshold.

[0026] Optionally, the first determination module includes: a fourth acquisition submodule, used to collect the driver's steering wheel grip; and a fifth determination submodule, used to determine that the driver is in an abnormal driving state when the steering wheel grip is lower than a preset grip threshold.

[0027] Optionally, the update module includes: an output submodule, which is used to output the normal face and sight angle values ​​corresponding to each image in the normal driving image set through a face angle and sight angle model; and a second statistical submodule, which is used to count all the normal face and sight angle values ​​and update the normal driving angle.

[0028] Optionally, the automatic calibration device for distraction areas further includes: an area calibration module, used to calibrate the non-distracting marked area and the distracting marked area inside the current vehicle according to a preset area of ​​interest, wherein the non-distracting marked area includes: a normal gaze identification point, and the distracting area includes at least: a boundary identification point; an image acquisition module, used to acquire images of multiple drivers facing the normal gaze identification point and the boundary identification point respectively, to obtain normal gaze images and distracted gaze images; an image analysis module, used to analyze the normal gaze images and the distracted gaze images to obtain the critical distraction deflection angle of the distraction area of ​​the current vehicle.

[0029] Optionally, the area calibration module includes: a sixth determination submodule, used to characterize the preset area of ​​interest as the non-distracting marked area inside the current vehicle; a seventh determination submodule, used to determine the center point of the non-distracting marked area to obtain the normal gaze identification point; an eighth determination submodule, used to determine multiple boundaries of the non-distracting marked area, and determine the distracting marked area outside the non-distracting marked area with each boundary as an edge, wherein any point on the boundary is characterized as the boundary identification point.

[0030] Optionally, the image analysis module includes: an analysis submodule, used to analyze the normal gaze image, determine the sight angle of the normal gaze identification point, and obtain a first normal driving angle based on the distribution of the sight angle of the normal gaze identification point; a first calculation submodule, used to analyze all the distracted gaze images, obtain the distribution of critical distracted driving angles of the distracted area, and calculate the average of the critical distracted driving angles of the distracted area based on the distribution of the critical distracted driving angles of the distracted area; a second calculation submodule, used to calculate the difference between the first normal driving angle and the average of the critical distracted driving angles, and obtain the critical distracted deflection angles of all the current vehicles.

[0031] Optionally, the calibration unit includes: a first calibration module, used to add the critical distraction deflection angle of the current vehicle on the basis of the normal driving angle to obtain the critical position of the distraction area boundary; a second calibration module, used to calibrate the area included in the critical position of the distraction area boundary as the non-distraction area, and calibrate the area outside the area included in the critical position of the distraction area boundary as the distraction area.

[0032] According to another aspect of an embodiment of the present invention, a road vehicle is provided, comprising: an on-board camera mounted on a windshield in front of the vehicle for capturing road images of the road ahead; and an on-board control unit connected to the on-board camera for executing any one of the above-described methods for automatically calibrating distraction areas.

[0033] According to another aspect of an embodiment of the present invention, a vehicle-mounted electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods for automatically calibrating distraction areas by executing the executable instructions.

[0034] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for automatically calibrating distracting areas.

[0035] The embodiment of the present invention can use automatic calibration to perform follow-up processing for different drivers and driving conditions. Compared with the fixed threshold method, it improves the accuracy of distraction detection and effectively reduces false detections.

[0036] The embodiment of the present invention adopts a multi-information fusion solution to filter abnormal driving states during driving to avoid false detection.

[0037] In an embodiment of the present invention, multiple facial images of drivers present in the current vehicle during a preset time period are combined to determine the driver's normal driving angle within the current vehicle. Based on the normal driving angle and a predetermined critical distraction deflection angle for the current vehicle, the driver's non-distracting and distracting areas within the current vehicle are calibrated. This embodiment analyzes multiple facial images of the driver present in the current vehicle to automatically calibrate the driver's non-distracting and distracting areas within the current vehicle. This dynamically processes the images based on individual drivers and driving states, improving the accuracy of distraction area calibration and, consequently, the accuracy of distraction state detection. This addresses the technical issue in related technologies where distraction areas change due to driving posture or driver changes, while distraction detection still uses a fixed distraction area, leading to false detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0039] Figure 1 is a flow chart of an optional method for automatically calibrating distracting areas according to an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of an optional preset abnormal driving area according to an embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of an optional demarcated distraction area according to an embodiment of the present invention;

[0042] Figure 4 2 is a schematic diagram of an optional automatic calibration device for distraction areas according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] The present invention can be applied to various types of transportation vehicles (cars, buses, motorcycles, airplanes, trains, etc.). This embodiment uses a vehicle as an example for schematic illustration to realize the detection of vehicle distraction areas and driver distraction states. Vehicle types include but are not limited to: sedans, trucks, sports cars, SUVs, and MINI cars. The present invention is adaptable to various parking spaces and areas, automatically calibrates distraction areas, and at the same time, analyzes whether the driver is in a distracted state based on the calibrated distraction areas. It performs follow-up processing for different individual drivers and driving states, and improves the accuracy of distraction detection and effectively reduces false detections compared to the fixed threshold method. The present invention can adaptively adjust the distraction area to avoid false detection of distraction due to driving posture or personnel changes. The present invention is described in detail below in conjunction with various embodiments.

[0046] Example 1

[0047] According to an embodiment of the present invention, an embodiment of a method for automatically calibrating distracting areas is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 1 is a flow chart of an optional method for automatically calibrating distracting areas according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0049] Step S102, collecting multiple facial images of drivers in the current vehicle within a preset time period;

[0050] Step S104, determining the normal driving angle of the driver in the current vehicle by combining multiple facial images;

[0051] Step S106 , based on the normal driving angle and the predetermined critical distraction deflection angle of the current vehicle, calibrate the non-distracting area and distracting area of ​​the driver in the current vehicle.

[0052] Through the above steps, multiple facial images of drivers in the current vehicle within a preset time period are collected. The driver's normal driving angle within the current vehicle is determined by combining these multiple facial images. Based on the normal driving angle and a predetermined critical distraction deflection angle for the current vehicle, the driver's non-distracting and distracting areas within the current vehicle are calibrated. In this embodiment, multiple facial images of the driver in the current vehicle are analyzed to automatically calibrate the driver's non-distracting and distracting areas within the current vehicle. Dynamic processing is performed based on individual drivers and driving states, improving the accuracy of distraction area calibration and, consequently, the accuracy of distraction state detection. This addresses the technical issue in related technologies where distraction areas change due to driving posture or driver changes, while distraction detection still uses a fixed distraction area, leading to false detections.

[0053] The embodiments of the present invention are described in detail below in conjunction with the above implementation steps.

[0054] Step S102 , collecting multiple facial images of drivers in the current vehicle within a preset time period.

[0055] To accurately demarcate distraction zones, the system combines the vehicle's pre-set deflection zones (shape and size) with the driver's driving attributes (body shape and angle) in real time. For example, varying driver builds and heights can cause variations in the demarcated distraction zones. A camera module installed within the vehicle captures multiple facial images of the driver over a pre-set time period, recovering real-time information from these images to accurately demarcate the distraction zones.

[0056] The preset time period of this embodiment can be pre-set. The specific duration of the preset time period is related to the actual detection accuracy requirements, for example, within 1 minute or within 30 seconds. In addition, the camera module captures the video stream, and the present application slides and processes the overlay frames of the preset time length. After processing the current frame, the next frame is input and the earliest frame is eliminated to update the image set, and then the image set is processed again, thereby achieving real-time automatic calibration.

[0057] In this embodiment, at least one camera module can be installed in the vehicle near the driving area. The specific installation location of the camera module is not limited, and it only needs to capture images including the driver's face. The types of camera modules include, but are not limited to, cameras, depth cameras, infrared cameras, etc. The types of facial images captured include, but are not limited to, standard RGB images, depth images, thermal images, etc. In this embodiment, standard RGB images are used for schematic illustration.

[0058] Step S104 , determining the normal driving angle of the driver in the current vehicle by combining multiple facial images.

[0059] Optionally, multiple facial images are combined to determine the normal driving angle of the driver in the current vehicle, including: combining abnormal driving information to determine the driver's abnormal driving state, and eliminating images corresponding to the abnormal driving state from all facial images to obtain a normal driving image set; and counting and updating the normal driving angle based on the normal driving image set.

[0060] Specifically, the normal driving angle refers to the current driver's line of sight angle when he or she is focused on driving in the current vehicle. Focused driving means that the driver's line of sight remains in a state of directly ahead, without making phone calls, turning the head, or looking at the rearview mirror. Furthermore, the normal driving angle will change as the scene changes. When driving in clear weather, the normal driving angle remains directly ahead, with a wide field of vision. When entering a tunnel or driving in foggy weather, the driver will unconsciously lower his or her head to focus on the environment in front of him or her, and the field of vision will decrease. The normal driving angle will move downward compared to the directly ahead position in clear weather. This application collects video streams in real time and slides and processes the collected video streams in real time to ensure that the normal driving angle information is updated in real time. In addition, in actual situations, not all image sets collected within a preset time period are in a state corresponding to the state of a focused driver. This application obtains a normal driving image set by deleting images corresponding to abnormal driving states from all facial images, and statistics and updates the long-term normal driving angle based on the normal driving image set.

[0061] Alternatively, the abnormal driving information includes at least one of the following: low vehicle speed, turn signal activation, distracted deflection, and grip strength. The following describes how to determine the driver's abnormal driving state in conjunction with each abnormal driving information.

[0062] Optionally, the abnormal driving state of the driver is determined in combination with the low vehicle speed, including: collecting the current vehicle speed of the vehicle; if the vehicle speed is lower than a preset speed threshold, determining that the driver is in an abnormal driving state.

[0063] In this embodiment, in scenarios such as starting and pulling over, it is often necessary to drive at a low speed and look around to make the right decision. This stage does not belong to the normal driving stage. The present application determines whether the vehicle is in a low-speed operation state by comparing the vehicle speed with a preset speed threshold. The preset speed threshold can be set in advance, and the specific setting of the preset speed threshold is determined according to actual user needs. For example, the preset speed threshold is set to 30km / h. If the current vehicle speed is greater than 30km / h, it is considered to be in a normal speed operation state. If the speed is less than 30km / h, it is considered to be in a low-speed operation state and the driver is in an abnormal driving state. The corresponding frames of the abnormal driving stage are further removed from the image set.

[0064] Optionally, the driver's abnormal driving state is determined in combination with the turn signal trigger, including: collecting the signal trigger state of the current vehicle's turn signal; if the signal trigger state indicates that the turn signal is not triggered, determining that the driver is in a normal driving state; if the signal trigger state indicates that the turn signal has been triggered, determining that the current vehicle is in a turning state, and determining that the driver is in an abnormal driving state.

[0065] Since the turn signal needs to be on when turning or making a U-turn, and the driver needs to look around to make the right decision, this application detects the triggering state of the turn signal to determine the driving state. If the turn signal is not triggered, the driver is in a normal driving state; if the turn signal is triggered, the vehicle is in a turning state. Frames corresponding to abnormal driving stages are further removed from the image set.

[0066] Optionally, the driver's abnormal driving state is determined in combination with distraction deflection, including: collecting the driver's face and sight angle; counting the time the driver's face and sight angle are in a preset abnormal driving area; if the time in the abnormal driving area reaches a first time threshold, it is determined that the driver is in an abnormal driving state.

[0067] In this embodiment, the area of ​​the distraction area is greater than or equal to the preset abnormal driving area. The preset abnormal driving area is set by the user and indicates an area where the driver is obviously distracted driving.

[0068] Figure 2 is a schematic diagram of an optional preset abnormal driving area according to an embodiment of the present invention, such as Figure 2 As shown, the area outside the distraction zone is called the non-distracting area, and the area outside the preset normal driving zone is called the preset abnormal driving zone. Compared to the distraction zone, which requires accurate automatic calibration, the preset abnormal driving zone is a roughly located area. When the sight angle falls within this area, it indicates that the driver is clearly distracted. For example, if the driver's eyes are drawn to a roadside billboard for a long time while driving, if they stay in the preset abnormal driving zone for a long time, the driver is determined to be in an abnormal driving state, and the corresponding frames of the abnormal driving state are further removed from the image set.

[0069] Another optional method is to determine the driver's abnormal driving state in combination with grip strength, including: collecting the driver's steering wheel grip strength; if the steering wheel grip strength is lower than a preset grip strength threshold, determining that the driver is in an abnormal driving state.

[0070] In practice, when a driver is in a state of fatigue or other distraction, their grip on the steering wheel decreases. Based on this, an embodiment of the present invention uses a sensor to obtain grip force values, and combines this with the driver's abnormal driving state to determine the driver's abnormal driving state. This embodiment pre-sets a grip force threshold adaptively based on the vehicle type. If the steering wheel grip force falls below the preset grip force threshold, the driver is determined to be in an abnormal driving state, and the corresponding frames in this abnormal driving state are removed from the image collection.

[0071] The embodiments of the present application do not limit any combination of the above-mentioned abnormal driving information inclusion categories, nor do they limit the priority order of the abnormal driving information inclusion categories when screening the image set.

[0072] Optionally, the normal driving angle is counted and updated based on the normal driving image set, including: each image in the normal driving image set passes through the face angle and sight angle model, and outputs the normal face and sight angle values ​​corresponding to each image; all normal face and sight angle values ​​are counted, and the normal driving angle is updated.

[0073] After collecting a set of normal driving images in the vehicle, each image is analyzed to determine the normal face and sight angle values ​​corresponding to each image, thereby updating the normal driving angle through all the face and sight angle values ​​during normal driving. In this embodiment, the output face and sight angle output values ​​can be obtained by inputting the image into the face angle and sight angle model. The present invention does not limit the form and type of the face angle and sight angle model, and a traditional geometric model or a neural network model can be used. This embodiment can count and update the normal driving angle for a long time. For example, the normal driving face and sight angle of the driver in the first 60 seconds are counted, and the average is taken and updated.

[0074] As an optional implementation of this embodiment, before determining the normal driving angle of the driver in the current vehicle by combining multiple facial images, the method also includes: initializing the normal driving angle, including: initializing the normal driving angle using the factory preset value; or initializing the normal driving angle using the first line of sight angle when the driver looks at the first mark point.

[0075] In this embodiment, the first marker point can be a single point or the average of multiple points. For example, the first marker point can be directly in front of the vehicle's windshield, or the average of any multiple points within a specific area of ​​the windshield. Initializing the normal driving angle helps the automatic distraction zone quickly and smoothly enter the normal driving angle update mode, avoiding excessive errors in the initial calculated normal driving angle due to extreme deviations in the initial data, which would require significant time and resources to return to normal operation.

[0076] Step S106 , based on the normal driving angle and the predetermined critical distraction deflection angle of the current vehicle, calibrate the non-distracting area and distracting area of ​​the driver in the current vehicle.

[0077] Another optional method for predetermining the critical distraction deflection angle of the current vehicle includes: calibrating a non-distracting marked area and a distracting marked area inside the current vehicle based on a preset area of ​​interest, wherein the non-distracting marked area includes: a normal gaze identification point, and the distracting area includes at least: a boundary identification point; collecting images of multiple drivers facing the normal gaze identification point and the boundary identification point respectively to obtain normal gaze images and distracted gaze images; and analyzing the normal gaze images and distracted gaze images to obtain a critical distraction deflection angle of the distracting area of ​​the current vehicle.

[0078] In this embodiment, the non-distracting labeled area and the distracting labeled area inside the current vehicle are calibrated according to the preset area of ​​interest, including: characterizing the preset area of ​​interest as the non-distracting labeled area inside the current vehicle; determining the center point of the non-distracting labeled area to obtain a normal gaze identification point; determining multiple boundaries of the non-distracting labeled area, and determining the distracting labeled area outside the non-distracting labeled area with each boundary as an edge, wherein any point on the boundary is characterized as a boundary identification point.

[0079] Figure 3 is a schematic diagram of an optional demarcated distraction area according to an embodiment of the present invention, such as Figure 3 As shown, the image of the driving area inside the car is divided into regions of interest (such as Figure 3 The shape of the region of interest is not limited in this application and can be circular, triangular, square, etc. It is generally set according to the vehicle attributes at the factory. Figure 3 A square is used as an example. Marker 1 is set within the non-distracting labeled area. Marker 1 is the center point of the area, i.e., the normal gaze marker. Multiple boundaries of the non-distracting labeled area are determined, and any point on the boundary is designated as a boundary marker. Figure 3 Positions 2, 3, 4, and 5 are set on the top, bottom, left, and right edges of the non-distracting area. Positions 2, 3, 4, and 5 represent the upper, right, lower, and left distracting area boundaries, respectively. The area beyond these boundaries is the distracting area. Specifically, position 1 is the center of the windshield, position 2 is the top edge of the windshield, position 3 is the right edge of the rearview mirror, position 4 is the center of the steering wheel, and position 5 is the left rearview mirror.

[0080] Optionally, the normal gaze image and each distracted gaze image are analyzed to obtain the critical distraction deflection angle of the current vehicle distraction area, including: analyzing the normal gaze image to determine the sight angle of the normal gaze identification point, and obtaining a first normal driving angle based on the distribution of the sight angle of the normal gaze identification point; analyzing all distracted gaze images to obtain the distribution of the critical distracted driving angle of each distraction area, and calculating the average critical distracted driving angle of each distraction area based on the distribution of the critical distracted driving angle of each distraction area; calculating the difference between the first normal driving angle and the average critical distracted driving angle to obtain the critical distraction deflection angle of all current vehicles.

[0081] by Figure 3 For example, this embodiment collects multiple images of the driver facing toward marker positions 1-5, inputs the images into a pre-trained face angle and sight line angle model, and outputs face and sight line angle output values. Specifically, based on the image set collected for gaze marker position 1, the distribution of sight line angles when gazing at the normal gaze mark point can be determined by calculating the mean value to obtain Mean_A1, i.e., the first normal angle. Similarly, for the image set collected for gaze marker positions 2-5, the critical distracted driving angle mean values ​​Mean_A2, Mean_A3...Mean_A5 are determined, and the difference between the first normal driving angle and the critical distracted driving angle mean values ​​in the yaw and pitch directions is calculated respectively to obtain the critical distraction deflection angles of all current vehicles. For example, the critical distraction deflection angle at marker position 2 is calculated by the following formula:

[0082] Yaw_A2_1=Yaw(Mean_A2)-Yaw(Mean_A1);

[0083] Pitch_A2_1=Pitch(Mean_A2)-Pitch(Mean_A1).

[0084] According to different vehicles or distraction areas, the distraction deflection angle for acquiring faces and sights is set to avoid re-collecting data to train the model and achieve better applicability.

[0085] In this embodiment, the average of the angles at which each driver looks toward the normal gaze marker and the boundary marker can be calculated, and then the difference between the first normal driving angle and the average of the critical distracted driving angles can be calculated to obtain the critical distraction deflection angles of all current vehicles. Based on the distraction deflection angles and the driver's normal driving angles, the driver's non-distracted area and distracted area in the current vehicle can be calibrated.

[0086] Optionally, based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle, the driver's non-distracting area and distracting area in the current vehicle are calibrated, including: adding the critical distraction deflection angle of the current vehicle on the basis of the normal driving angle to obtain the critical position of the distraction area boundary; calibrating the area included in the critical position of the distraction area boundary as the non-distracting area, and calibrating the area outside the area included in the critical position of the distraction area boundary as the distraction area.

[0087] After calibrating the driver's non-distracted and distracted areas within the vehicle, the system outputs the driver's prolonged distraction status. For example, if the driver remains in the distracted area for 5 consecutive seconds, it is considered distracted driving; otherwise, it is considered normal driving. When the vehicle speed reaches a threshold or the turn signal is engaged, the driver is considered non-distracted. By detecting whether the driver is distracted, the system can send timely reminders (e.g., voice reminders, alarm sounds) when the driver is confirmed to be distracted, reducing the probability of accidents and improving vehicle driving safety.

[0088] The embodiment of the present invention adopts a multi-information fusion solution to filter abnormal driving conditions during driving to avoid false detections. At the same time, the embodiment of the present invention also adopts automatic calibration to perform follow-up processing for different individual drivers and driving conditions. Compared with the fixed threshold calibration method, the present application can improve the accuracy of distraction detection and effectively reduce false detections.

[0089] The present invention is described below in conjunction with another optional embodiment.

[0090] Example 2

[0091] This embodiment provides an automatic calibration device for distracting areas. The various units included in the automatic calibration device correspond to the various implementation steps in the above-mentioned embodiment 1.

[0092] Figure 4 is a schematic diagram of an optional automatic calibration device for distraction areas according to an embodiment of the present invention. Figure 4 As shown, the automatic calibration device may include: a collection unit 41, a determination unit 43, and a calibration unit 45, wherein:

[0093] The collecting unit 41 is used to collect multiple facial images of drivers in the current vehicle within a preset time period;

[0094] A determination unit 43, configured to determine a normal driving angle of the driver in the current vehicle by combining the multiple facial images;

[0095] The calibration unit 45 is configured to calibrate the non-distraction area and the distraction area of ​​the driver in the current vehicle based on a normal driving angle and a predetermined critical distraction deflection angle of the current vehicle.

[0096] The above-mentioned automatic calibration device for distraction areas can collect multiple facial images of drivers in the current vehicle within a preset time period through the collection unit 41, determine the normal driving angle of the driver in the current vehicle through the determination unit 43 in combination with the multiple facial images, and calibrate the non-distracting area and distracting area of ​​the driver in the current vehicle based on the normal driving angle and the predetermined critical distraction deflection angle of the current vehicle through the calibration unit 45. In this embodiment, multiple facial images of the driver in the current vehicle can be analyzed to automatically calibrate the non-distracting area and distracting area of ​​the driver in the current vehicle, perform follow-up processing for different individual drivers and driving states, improve the calibration accuracy of the distraction area, and further improve the detection accuracy of the distraction state, thereby solving the technical problem in the related art that the distraction area changes due to driving posture or personnel changes, while the distraction detection still uses a fixed distraction area, resulting in false detection.

[0097] Optionally, the determination unit includes: a first determination module, used to determine the driver's abnormal driving state in combination with abnormal driving information, and eliminate images corresponding to the abnormal driving state from all facial images to obtain a normal driving image set; an update module, used to count and update the normal driving angle based on the normal driving image set.

[0098] Optionally, it also includes: an initialization unit, which is used to initialize the normal driving angle before determining the normal driving angle of the driver in the current vehicle by combining multiple facial images, and the initialization unit includes: a first initialization module, which is used to initialize the normal driving angle using a factory preset value; or, a second initialization module, which is used to initialize the normal driving angle using a first line of sight angle when the driver looks at a first marker point.

[0099] Optionally, the abnormal driving information includes at least one of the following: low vehicle speed, turn signal triggering, distraction deflection, and grip strength.

[0100] Optionally, the first determination module includes: a first acquisition submodule, used to collect the current vehicle speed of the vehicle; and a first determination submodule, used to determine that the driver is in an abnormal driving state when the vehicle speed is lower than a preset speed threshold.

[0101] Optionally, the first determination module includes: a second acquisition submodule, used to collect the signal trigger status of the current vehicle's turn signal; a second determination submodule, used to determine that the driver is in a normal driving state when the signal trigger status indicates that the turn signal is not triggered; a third determination submodule, used to determine that the current vehicle is in a turning state and that the driver is in an abnormal driving state when the signal trigger status indicates that the turn signal has been triggered.

[0102] Optionally, the first determination module includes: a third acquisition submodule, used to collect the driver's face and sight angle; a first statistics submodule, used to count the time the driver's face and sight angle are in a preset abnormal driving area; and a fourth determination submodule, used to determine that the driver is in an abnormal driving state when the time in the abnormal driving area reaches a first time threshold.

[0103] Optionally, the first determination module includes: a fourth acquisition submodule, used to collect the driver's steering wheel grip; and a fifth determination submodule, used to determine that the driver is in an abnormal driving state when the steering wheel grip is lower than a preset grip threshold.

[0104] Optionally, the update module includes: an output submodule, which is used to output the normal face and sight angle values ​​corresponding to each image in the normal driving image set through the face angle and sight angle model; a second statistical submodule, which is used to count all normal face and sight angle values ​​and update the normal driving angle.

[0105] Optionally, the automatic calibration device for distraction areas further includes: an area calibration module, used to calibrate non-distracting marked areas and distracting marked areas inside the current vehicle according to a preset area of ​​interest, wherein the non-distracting marked areas include: normal gaze identification points, and the distracting areas include at least: boundary identification points; an image acquisition module, used to capture images of multiple drivers facing normal gaze identification points and boundary identification points respectively, to obtain normal gaze images and distracted gaze images; an image analysis module, used to analyze the normal gaze images and distracted gaze images to obtain the critical distraction deflection angle of the current vehicle distraction area.

[0106] Optionally, the area calibration module includes: a sixth determination submodule, used to characterize the preset area of ​​interest as a non-distracting marked area inside the current vehicle; a seventh determination submodule, used to determine the center point of the non-distracting marked area to obtain a normal gaze identification point; an eighth determination submodule, used to determine multiple boundaries of the non-distracting marked area, and determine a distracting marked area outside the non-distracting marked area with each boundary as an edge, wherein any point on the boundary is characterized as a boundary identification point.

[0107] Optionally, the image analysis module includes: an analysis submodule, used to analyze the normal gaze image, determine the sight angle of the normal gaze identification point, and obtain a first normal driving angle based on the distribution of the sight angle of the normal gaze identification point; a first calculation submodule, used to analyze all distracted gaze images, obtain the distribution of critical distracted driving angles of the distracted area, and calculate the average critical distracted driving angle of the distracted area based on the distribution of the critical distracted driving angles of the distracted area; a second calculation submodule, used to calculate the difference between the first normal driving angle and the average critical distracted driving angle to obtain the critical distracted deflection angle of all current vehicles.

[0108] Optionally, the calibration unit includes: a first calibration module, used to add the current vehicle's critical distraction deflection angle on the basis of the normal driving angle to obtain the critical position of the distraction area boundary; a second calibration module, used to calibrate the area included in the critical position of the distraction area boundary as a non-distraction area, and calibrate the area outside the area included in the critical position of the distraction area boundary as a distraction area.

[0109] The above-mentioned automatic calibration device for distraction areas may also include a processor and a memory. The above-mentioned acquisition unit 41, determination unit 43, calibration unit 45, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0110] The processor includes a core that retrieves corresponding program units from memory. One or more cores may be configured to adjust kernel parameters to calibrate the driver's non-distracting and distracting areas within the vehicle based on a normal driving angle and a predetermined critical distraction angle for the vehicle.

[0111] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0112] According to another aspect of an embodiment of the present invention, a road vehicle is also provided, comprising: an on-board camera installed on the windshield in front of the vehicle, for collecting road images of the road ahead; and an on-board control unit connected to the on-board camera, for executing any one of the above-mentioned methods for automatically calibrating distraction areas.

[0113] According to another aspect of an embodiment of the present invention, a vehicle-mounted electronic device is also provided, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any of the above-mentioned methods for automatically calibrating distraction areas by executing the executable instructions.

[0114] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above-mentioned methods for automatically calibrating distracting areas.

[0115] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: collecting multiple facial images of drivers contained in a current vehicle within a preset time period; determining the normal driving angle of the driver in the current vehicle by combining the multiple facial images; and calibrating the non-distracting area and distracting area of ​​the driver in the current vehicle based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle.

[0116] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0117] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0119] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0120] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0122] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for automatically calibrating distracting areas, characterized in that: include: Collecting multiple facial images of drivers in the current vehicle within a preset time period; Determining the abnormal driving state of the driver based on the abnormal driving information, removing images corresponding to the abnormal driving state from all facial images, obtaining a set of normal driving images, and calculating and updating the normal driving angle of the driver in the current vehicle based on the set of normal driving images; calibrating a non-distraction zone and a distraction zone of the driver in the current vehicle based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle; The critical distraction deflection angle of the current vehicle is predetermined, including: calibrating a non-distracting marked area and a distracting marked area inside the current vehicle according to a preset region of interest, wherein the non-distracting marked area includes: a normal gaze identification point, and the distracting area includes at least: a boundary identification point; collecting images of multiple drivers facing the normal gaze identification point and the boundary identification point respectively to obtain normal gaze images and distracted gaze images; and analyzing the normal gaze images and the distracted gaze images to obtain a critical distraction deflection angle of the distracting area of ​​the current vehicle.

2. The method according to claim 1, characterized in that Before determining the abnormal driving state of the driver in combination with the abnormal driving information, the method further includes: initializing the normal driving angle, including: Initializing the normal driving angle using a factory preset value; or, The normal driving angle is initialized using a first sight line angle when the driver looks at the first marking point.

3. The method according to claim 1, characterized in that The abnormal driving information includes at least one of the following: low vehicle speed, turn signal triggering, distraction deflection, and grip strength.

4. The method according to claim 3, characterized in that Determining the abnormal driving state of the driver in combination with the low vehicle speed includes: collecting the vehicle speed of the current vehicle; If the vehicle's driving speed is lower than a preset speed threshold, it is determined that the driver is in an abnormal driving state.

5. The method according to claim 3, characterized in that Determining the abnormal driving state of the driver in combination with the turn signal trigger includes: collecting a signal triggering state of a turn signal of the current vehicle; If the signal triggering status indicates that the turn signal is not triggered, determining that the driver is in a normal driving state; If the signal triggering status indicates that the turn signal has been triggered, it is determined that the current vehicle is in a turning state and that the driver is in an abnormal driving state.

6. The method according to claim 3, characterized in that Determining the driver's abnormal driving state in combination with distraction deflection includes: Collecting the driver's face and sight angle; Counting the time the driver's face and sight angle are in a preset abnormal driving area; If the duration of being in the abnormal driving area reaches a first duration threshold, it is determined that the driver is in an abnormal driving state.

7. The method according to claim 3, characterized in that Determining the abnormal driving state of the driver in combination with the grip strength includes: collecting the driver's steering wheel grip; If the steering wheel grip force is lower than a preset grip force threshold, it is determined that the driver is in an abnormal driving state.

8. The method according to claim 3, characterized in that Counting and updating the normal driving angle according to the normal driving image set includes: Each image in the normal driving image set is passed through a face angle and sight angle model to output normal face and sight angle values ​​corresponding to each image; Count all the normal face and sight angle values, and update the normal driving angle.

9. The method according to claim 1, characterized in that The non-distracting marked area and the distracting marked area inside the current vehicle are calibrated according to a preset region of interest, including: Characterizing the preset region of interest as a non-distracting marked area inside the current vehicle; Determine the center point of the non-distracting marked area to obtain the normal gaze marker point; A plurality of boundaries of the non-distracting labeled area are determined, and a distracting labeled area outside the non-distracting labeled area is determined with each boundary as an edge, wherein any point on the boundary is characterized as the boundary identification point.

10. The method according to claim 1, characterized in that Analyzing the normal gaze image and the distracted gaze image to obtain a critical distraction deflection angle of the current vehicle distraction area includes: Analyzing the normal gaze image to determine a sight line angle of the normal gaze identification point, and obtaining a first normal driving angle based on a distribution of sight line angles of the normal gaze identification point; analyzing all the distracted gaze images to obtain a distribution of critical distracted driving angles of the distracted area, and calculating a mean of the critical distracted driving angles of the distracted area based on the distribution of the critical distracted driving angles of the distracted area; The difference between the first normal driving angle and the average of the critical distracted driving angles is calculated to obtain the critical distracted deflection angles of all the current vehicles.

11. The method according to claim 1, characterized in that Based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle, calibrating a non-distracting area and a distracting area of ​​the driver in the current vehicle, including: Adding the current critical distraction deflection angle of the vehicle to the normal driving angle to obtain a critical position of the distraction area boundary; The area included in the critical position of the distracting area boundary is marked as the non-distracting area, and the area other than the area included in the critical position of the distracting area boundary is marked as the distracting area.

12. An automatic demarcation device for distraction areas, characterized in that: include: a collection unit, configured to collect multiple facial images of drivers in a current vehicle within a preset time period; a determining unit, configured to determine a normal driving angle of the driver in the current vehicle based on the abnormal driving state of the driver and the multiple facial images; The determining unit includes: a first determining module for determining the abnormal driving state of the driver in combination with the abnormal driving information, and removing images corresponding to the abnormal driving state from all facial images to obtain a normal driving image set; an updating module for counting and updating the normal driving angle based on the normal driving image set; a calibration unit, configured to calibrate a non-distraction area and a distraction area of ​​the driver in the current vehicle based on the normal driving angle and a predetermined critical distraction deflection angle of the current vehicle; An area calibration module is configured to calibrate non-distracting marked areas and distracting marked areas within the current vehicle based on a preset area of ​​interest, wherein the non-distracting marked areas include normal gaze identification points, and the distracting areas include at least boundary identification points; an image acquisition module is configured to capture images of multiple drivers facing the normal gaze identification points and the boundary identification points, respectively, to obtain normal gaze images and distracting gaze images; and an image analysis module is configured to analyze the normal gaze images and the distracting gaze images to obtain a critical distraction deflection angle of the current vehicle's distracting area.

13. The device according to claim 12, characterized in that Also includes: an initialization unit, configured to initialize the normal driving angle before determining the abnormal driving state of the driver in combination with the abnormal driving information, the initialization unit comprising: A first initialization module, configured to initialize the normal driving angle using a factory preset value; or, The second initialization module is configured to initialize the normal driving angle using a first sight line angle when the driver is gazing at the first marking point.

14. The device according to claim 12, characterized in that The abnormal driving information includes at least one of the following: low vehicle speed, turn signal triggering, distraction deflection, and grip strength.

15. A road vehicle, characterized in that include: The vehicle-mounted camera is installed on the windshield in front of the vehicle and is used to collect road images of the road ahead; An on-vehicle control unit is connected to the on-vehicle camera and executes the automatic calibration method of the distraction area according to any one of claims 1 to 11.

16. An in-vehicle electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the method for automatically calibrating distracting areas according to any one of claims 1 to 11 by executing the executable instructions.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for automatically calibrating distracting areas according to any one of claims 1 to 11.

Images