Hybrid data set construction method and system for driver behavior monitoring

Abstract

The invention relates to a mixed data set construction method and system for driver behavior monitoring, and the method comprises the steps: defining a human body library according with human body distribution according to the man-machine engineering requirements, constructing a behavior process virtual video with vehicle environment constraints based on the human body library and typical non-driving behaviors, and virtualizing the true value label of each frame of the video; the driving posture of a driver and passengers is captured through a camera in the vehicle, and the driver is prompted to execute tasks of all non-standard driving actions in sequence, so that actually measured video data corresponding to each task is obtained; and randomly mixing the virtual image data and the actually measured image data to obtain a mixed data set for training a driving behavior monitoring model to perform driving behavior monitoring. Compared with the prior art, the driving behavior data set with the true value label can be generated at low cost, and a foundation is laid for development of a high-precision driving behavior recognition algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of driver and passenger behavior monitoring, and in particular to a hybrid data set construction method and system for driver behavior monitoring. Background Art

[0002] At present, there are some data sets for the development of driver and occupant monitoring algorithms, including real vehicle data sets and virtual data sets. However, real vehicle data has the problems of expensive collection cost and complex labeling. The setting cost of its true value labels is high and the quantity is scarce. Compared with the actual human body, virtual data is less realistic and lacks environmental constraints. Therefore, the sample size of effective driver behavior data sets is small, making it difficult to fully train driver and occupant monitoring algorithms. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for constructing a hybrid data set for driver behavior monitoring in order to overcome the defects of the above-mentioned prior art, namely, the lack of true value labels of driving behavior data sets and the lack of environmental constraints on virtual data.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] A method for constructing a hybrid dataset for driver behavior monitoring includes the following steps:

[0006] Virtual data generation step: define a human body library that conforms to human body distribution according to ergonomic requirements, construct a virtual video of the behavior process with vehicle environment constraints based on the human body library and typical non-driving behaviors, and generate true value labels of the initial position and extreme position of the video process, and obtain the true value label of each frame of the virtual video of the behavior process by interpolation method;

[0007] Measured image data collection steps: Capture the driving posture of the driver and passengers through the camera in the vehicle, and prompt the driver to perform various non-standard driving action tasks in sequence, so as to obtain the measured video data corresponding to each task and set the corresponding true value label;

[0008] Data mixing step: obtaining virtual image data and its true value label according to the virtual video of the behavior process, obtaining measured image data and its true value label according to the measured video data, randomly mixing the virtual image data with the measured image data to obtain a mixed data set for training a driving behavior monitoring model to perform driver behavior monitoring.

[0009] Furthermore, the data mixing step also includes: generating a corresponding grayscale image according to the virtual image data, and randomly mixing the virtual image data, the measured image data and the grayscale image to obtain a mixed data set.

[0010] Furthermore, the driving behavior monitoring model is a machine learning algorithm model for cockpit distraction identification tasks.

[0011] Furthermore, the true value label includes joint point coordinates, angle vectors and behavior types.

[0012] Furthermore, the camera is a camera with a fisheye lens.

[0013] Furthermore, the camera is arranged above the interior rearview mirror at the symmetry center of the vehicle.

[0014] Furthermore, the measured image data acquisition step acquires measured video data of drivers and passengers of various size types, and the size types of the drivers and passengers are selected according to three dimensions: height, proportion, and fatness.

[0015] Furthermore, the gender of the driver and passenger includes male and female.

[0016] Furthermore, the construction of the virtual video of the behavior process with vehicle environment constraints is implemented using RAMSIS software.

[0017] The present invention also provides a hybrid data set construction device for driver behavior monitoring, comprising a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the method described above.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] (1) The present invention improves the accuracy of virtual data while fully considering the technical requirements of vehicle ergonomics layout, and greatly expands the sample size of the overall data set through the virtual data method; this is crucial in the subsequent training of algorithm functions, because the judgment of driving posture and behavior risk requires a relatively accurate data set as algorithm training, so as to achieve the purpose of reliable quantitative analysis. The data set provided by the present invention contains a very rich range of non-driving activities, covering all common typical non-driving activities, which can provide a good input for subsequent algorithm training, and can be used as an alternative to the existing in-vehicle data set, improving the accuracy of the algorithm and shortening the development cycle.

[0020] (2) The method of the present invention can realize the low-cost generation of a large number of driving behavior data sets with true value labels, laying the foundation for the development of high-precision driving behavior recognition algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a flow chart of a method for constructing a hybrid data set for driver behavior monitoring provided in an embodiment of the present invention;

[0022] Figure 2A schematic diagram of a flow chart of a virtual data generation step provided in an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a human body library definition provided in an embodiment of the present invention;

[0024] Figure 4 It is a schematic diagram of a mixed data set provided in an embodiment of the present invention, wherein the real-person measured picture on the far left is blurred. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0028] Example 1

[0029] like Figure 1 As shown, this embodiment provides a method for constructing a hybrid data set for driver behavior monitoring, comprising the following steps:

[0030] Virtual data generation step S1: define a human body library that conforms to human body distribution according to ergonomic requirements, construct a virtual video of the behavior process with vehicle environment constraints based on the human body library and typical non-driving behaviors, and generate true value labels of the initial position and extreme position of the video process, and obtain the true value label of each frame of the virtual video of the behavior process by interpolation method;

[0031] Measured image data acquisition step S2: Capturing the driving posture of the driver and passenger through a camera in the vehicle, and prompting the driver to perform various non-standard driving action tasks in sequence, thereby obtaining the measured video data corresponding to each task, and setting the corresponding true value label;

[0032] Data mixing step S3: obtain virtual image data and its true value label according to the virtual video of the behavior process, obtain measured image data and its true value label according to the measured video data, randomly mix the virtual image data with the measured image data to obtain a mixed data set for training the driving behavior monitoring model to monitor the driver behavior.

[0033] Specifically, in the virtual data generation step S1, the construction of the virtual video of the behavior process with vehicle environment constraints is implemented using RAMSIS software.

[0034] The ground-truth labels include joint point coordinates, angle vectors, and behavior types.

[0035] Specifically, Figure 2 As shown, in the virtual data generation step S1 of this embodiment, a simulation environment for the non-driving behavior motion process is built based on RAMSIS (an ergonomics technology analysis software). First, a human body library that conforms to the human body distribution is defined according to the ergonomics requirements, such as Figure 3 As shown in the figure, based on the human body library and typical non-driving behaviors, virtual videos of the behavior process with vehicle environment constraints are produced respectively, and the true value labels of the initial and extreme positions of the video process are output, such as joint point coordinates, angle vectors and behavior types. Finally, the true value label of each frame of the process is obtained through interpolation, thereby realizing the generation of a large amount of virtual data with high-precision true value labels, laying the foundation for subsequent algorithm development.

[0036] This method can realize the rapid generation of large quantities of data sets at low cost, and the true value labels are realized under the constraints of environmental layout conditions, which has very important application value.

[0037] In the measured image data acquisition step S2, the camera is arranged above the interior rearview mirror at the symmetry center of the vehicle and is a camera with a fisheye lens.

[0038] In this step, measured video data of drivers and passengers of various sizes are collected, and the size types of the drivers and passengers are selected according to three dimensions: height, proportion, and fatness.

[0039] Specifically, in the measured image data acquisition step S2 of this embodiment, the camera is arranged above the rearview mirror at the symmetric center of the vehicle, and a fisheye camera is used to expand the viewing angle, so that all driving postures of the front-seat occupants can be fully captured. The driver performs non-standard driving actions in sequence according to the test process and operating specifications. After the camera records the video of each task, the corresponding image is obtained through the video frame.

[0040] like Figure 3As shown, volunteers should select people of specific sizes as much as possible, covering the 5th, 50th and 95th percentiles of height, proportion and fatness, with a total of at least 10 men and women (5 men and 5 women). If conditions permit, the ideal state can expand the special population to 18 people (focusing on two dimensions) or 54 people (three dimensions).

[0041] The data mixing step S3 also includes: generating a corresponding grayscale image according to the virtual image data, and randomly mixing the virtual image data, the measured image data and the grayscale image to obtain a mixed data set.

[0042] Specifically, Figure 4 As shown in the figure, the image data generated by virtual data is randomly mixed with the measured image data to obtain the required mixed data set. This data set can be used for algorithm training in common cockpit distraction recognition tasks, and can also be used to develop new functions according to actual needs. Three pictures are obtained, one real car image, one virtual image, and one grayscale image. In the construction of the mixed data set, the sample size of the mixed data set can be greatly expanded through virtual data, and the measured data and virtual data can be randomly scattered before algorithm training to provide support for the generalization of the training algorithm.

[0043] Preferably, after the mixed data set is obtained by the above method, a driving behavior monitoring step S4 may be performed: a preset driving behavior monitoring model is trained using the mixed data set, and the trained driving behavior monitoring model is used to perform driving behavior monitoring.

[0044] In the driving behavior monitoring step S4, the driving behavior monitoring model is a machine learning algorithm model for the cockpit distraction recognition task, such as a neural network model for the cockpit distraction recognition task.

[0045] The above is an introduction to a method embodiment. The following is a further explanation of the solution of the present invention through an apparatus embodiment.

[0046] This embodiment also relates to a hybrid data set construction device for driver behavior monitoring, including a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the hybrid data set construction method for driver behavior monitoring as described above.

[0047] It should be noted that the specific content and beneficial effects of the device of the present application can be found in the above-mentioned method embodiment, which will not be repeated here.

[0048] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A hybrid data set construction method for driver behavior monitoring, characterized in that: The following steps are involved: Virtual data generation step: define a human body library that conforms to human body distribution according to ergonomic requirements, construct a virtual video of the behavior process with vehicle environment constraints based on the human body library and typical non-driving behaviors, and generate true value labels of the initial position and extreme position of the video process, and obtain the true value label of each frame of the virtual video of the behavior process by interpolation method; Measured image data collection steps: Capture the driving posture of the driver and passengers through the camera in the vehicle, and prompt the driver to perform various non-standard driving action tasks in sequence, so as to obtain the measured video data corresponding to each task and set the corresponding true value label; Data mixing step: obtaining virtual image data and its true value label according to the virtual video of the behavior process, obtaining measured image data and its true value label according to the measured video data, randomly mixing the virtual image data with the measured image data to obtain a mixed data set for training a driving behavior monitoring model to perform driver behavior monitoring.

2. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The data mixing step also includes: generating a corresponding grayscale image according to the virtual image data, and randomly mixing the virtual image data, the measured image data and the grayscale image to obtain a mixed data set.

3. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The driving behavior monitoring model is a machine learning algorithm model used for cockpit distraction identification tasks.

4. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The true value label includes joint point coordinates, angle vectors and behavior types.

5. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The camera is a camera with a fisheye lens.

6. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The camera is arranged above the interior rearview mirror at the symmetry center of the vehicle.

7. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The measured image data acquisition step acquires measured video data of drivers and passengers of various size types, and the size types of the drivers and passengers are selected according to three dimensions: height, proportion, and fatness.

8. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The gender of the driver and passenger includes male and female.

9. The method for constructing a hybrid data set for driver behavior monitoring according to claim 1, characterized in that: The construction of the virtual video of the behavior process with vehicle environment constraints is achieved by using RAMSIS software.

10. A hybrid data set construction device for driver behavior monitoring, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of any one of the methods according to claims 1 to 9.

Images