Driving scene human factor safety monitoring experiment platform and method

Abstract

The invention discloses a driving scene human factor safety monitoring experiment platform and method, and belongs to the technical field of safe driving, the driving scene human factor safety monitoring experiment platform is composed of a plurality of acquisition devices, an experiment machine, a server and a display, the plurality of acquisition devices are used for acquiring monitoring data in an experiment process of a user, and the monitoring data correspond to current simulation driving scene information; according to the method, the current behavior mode and the physiological signal of the driver are comprehensively monitored and correspond to the current simulated complex driving scene information, and the problems that the human factor safety monitoring is poor in universality, and the human factor safety monitoring experience is poor are solved. And the physiological signal of the driver cannot be comprehensively monitored, so that the human factor safety monitoring experiment result is more accurate and reliable.

Description

Technical Field

[0001] The present invention relates to the field of safe driving technology, and in particular to a driving scenario human safety monitoring experimental platform and method. Background Art

[0002] Road traffic safety is a complex system engineering, involving many factors such as vehicles, roads, environment and drivers. Among them, human factors play a particularly prominent role in traffic accidents. The World Health Organization's report in 2018 showed that more than 93% of traffic accidents in the road traffic system are related to human factors. Therefore, the monitoring and management of driver behavior has become the key to improving road traffic safety. Human safety monitoring can detect potential safety hazards in a timely manner by real-time monitoring of the driver's physiological and psychological state, behavior patterns, etc., so as to take corresponding prevention and intervention measures to effectively reduce the occurrence of traffic accidents.

[0003] Human safety monitoring is of great significance in road traffic safety. Through real-time monitoring and analysis of drivers' physiological and psychological states, behavior patterns, etc., potential safety hazards can be discovered and eliminated in a timely manner, reducing the occurrence of traffic accidents. The application of human safety monitoring technology can not only improve drivers' safety awareness, reduce traffic accidents and casualties, and promote the development of intelligent transportation technology, but also promote social harmony and sustainable development, and provide strong guarantees for coping with future traffic challenges.

[0004] Since it is difficult to obtain data on real driving scenarios, especially data on complex and extreme scenarios are even more scarce, the data sets used by existing human factors monitoring algorithms are often small and not comprehensive enough. This lack of data leads to poor versatility of the algorithm, making it difficult to adapt to various complex and changeable actual driving environments. For example, existing algorithms may perform well under common urban road conditions, but their performance and reliability may drop significantly under extreme conditions such as bad weather, night driving, or emergencies. In addition, existing human factors safety monitoring systems usually only monitor the driver's current behavior patterns, however, these systems often ignore the comprehensive monitoring of the driver's physiological signals, such as key physiological indicators such as heart rate, breathing rate, and skin galvanic response. These physiological signals can provide more in-depth and accurate information on the driver's status, such as fatigue, emotional changes, and stress levels, but the monitoring of these signals is not yet universal and comprehensive enough. Summary of the invention

[0005] The purpose of an embodiment of the present invention is to provide a driving scene human safety monitoring experimental platform and method. The invention is composed of multiple acquisition devices, a test machine, a server and a display. The monitoring data during the user experiment is collected by multiple acquisition devices, and corresponds to the current simulated driving scene information. A dual-branch multi-frame temporal feature aggregation network model is used to finally obtain the human safety monitoring experimental results. The invention comprehensively monitors the driver's current behavior patterns and physiological signals, and corresponds to the current simulated complex driving scene information, which solves the problems of poor versatility of human safety monitoring, difficulty in adapting to various complex and changeable driving environments, and inability to comprehensively monitor the driver's physiological signals, making the human safety monitoring experimental results more accurate and reliable.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] The embodiment of the present invention provides a driving scene human safety monitoring experimental platform, the driving scene human safety monitoring experimental platform comprises: a plurality of acquisition devices, a test machine, a server and a display;

[0008] The acquisition device is used to collect monitoring data during the user experiment, and the monitoring data includes: facial movement information, micro-expression information, physiological signals and driving operation information;

[0009] The experimental machine is used to simulate driving scenes for user testing, obtain the monitoring data collected by the acquisition device and send the monitoring data and current simulated driving scene information to the server;

[0010] The server is used to call a dual-branch multi-frame temporal feature aggregation network model to analyze the monitoring data and the current simulated driving scene information to obtain a human safety monitoring experiment result;

[0011] The display is used to display the monitoring data and the human safety monitoring experiment results.

[0012] Optionally, the acquisition equipment includes: a depth camera, a visible light camera, a physiological instrument, an eye tracker, a driving steering wheel, pedals and a gear lever.

[0013] Optionally, the depth camera is used to perform face modeling on the user and collect facial movement information of the user;

[0014] The visible light camera is used to collect a facial image of the user, wherein the facial image includes facial micro-expression information;

[0015] The physiological instrument is used to collect physiological signals of the user, wherein the physiological signals include pulse signals, electrocardiogram signals and skin electrical signals;

[0016] The eye tracker is used to collect the points on the display that the user is looking at;

[0017] The driving steering wheel, the pedal and the gear lever are used to collect the user's control operation information on the driving steering wheel, the pedal and the gear lever during the test.

[0018] Optionally, the dual-branch multi-frame temporal feature aggregation network model includes: a target detection sub-network model, a feature aggregation sub-network model and a target correction sub-network model;

[0019] The target detection subnetwork model is used to detect the input face image and obtain the image candidate region and the image candidate region features;

[0020] The feature aggregation subnetwork model is used to perform multi-frame feature fusion on the candidate region features to obtain fused candidate region features;

[0021] The target correction sub-network model is used to correct and refine the prediction of the image candidate region and the image candidate region features according to the fused candidate region features to obtain the human safety monitoring experiment results.

[0022] Optionally, the process of the object detection sub-network model outputting the image candidate region and the image candidate region features includes:

[0023] S51, obtaining a priori frames to be tested of different sizes at different positions of the input face image according to preset a priori frame parameters;

[0024] S52, extracting the image region feature maps of the to-be-tested priori frames of different sizes to obtain each image region feature map;

[0025] S53, according to the feature maps of each image region, outputting the number of length micro-detection categories of each image through a convolutional layer to obtain a classification confidence vector of each image;

[0026] S54. According to the classification confidence vectors of the respective images, screening is performed by using a preset probability threshold to obtain respective image candidate regions and image candidate region features.

[0027] Optionally, the feature aggregation subnetwork model includes: a local branch and a global branch;

[0028] The feature aggregation objects of the local branch include: adjacent frames of the face image frame to be tested;

[0029] The feature aggregation object of the global branch includes: a face image frame whose distance from the face image frame to be tested to a preset step length in the time dimension.

[0030] Optionally, the target correction subnetwork model includes: a classification prediction branch and a position prediction branch;

[0031] The classification prediction branch is used to input the candidate region features into a first fully connected layer, and the output dimension of the first fully connected layer is equal to the number of classification categories;

[0032] The position prediction branch is used to input the candidate region features into the second fully connected layer to predict the target position, and the output dimension of the second fully connected layer is four times the number of classification categories.

[0033] The embodiment of the present invention further provides a driving scene human safety monitoring experimental method, which is implemented by a driving scene human safety monitoring experimental platform, and the method includes:

[0034] S1. Turn on the experimental machine and simulate the user's driving scenario;

[0035] S2. Turn on the collection device to collect monitoring data during the user's experiment;

[0036] S3, obtaining current simulated driving scene information and monitoring data information according to the user driving scene and the monitoring data during the user experiment;

[0037] S4. According to the current simulated driving scene information and the monitoring data information, a dual-branch multi-frame temporal feature aggregation network model is used to obtain human safety monitoring experiment results;

[0038] S5. Display the monitoring data and the human safety monitoring experiment results through a display screen according to the monitoring data information and the human safety monitoring experiment results.

[0039] Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects:

[0040] The above scheme consists of multiple acquisition devices, test machines, servers and displays. On the one hand, the monitoring data during the user experiment is collected through multiple acquisition devices, which solves the problem that the existing technology cannot comprehensively monitor the driver's physiological signals. On the other hand, the monitoring data during the user experiment corresponds to the complex driving scene information of the current simulation, and a dual-branch multi-frame temporal feature aggregation network model is adopted to finally obtain the human safety monitoring experiment results, which solves the problem that human safety monitoring has poor versatility and is difficult to adapt to various complex and changeable driving environments, making the human safety monitoring experiment results more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a system block diagram of a driving scenario human safety monitoring experimental platform according to an embodiment of the present application;

[0042] Figure 2 is a flowchart showing a method for human safety monitoring in a driving scenario according to an embodiment of the present application;

[0043] Figure 3 is a flowchart showing the steps of operating a target detection subnetwork model of a driving scenario human safety monitoring experimental platform according to an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of a driving scenario human safety monitoring experimental platform according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0046] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0047] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0048] On the one hand, if Figure 1 As shown, the driving scene human safety monitoring experimental platform of the embodiment of the present application includes: an experimental machine 101, multiple acquisition devices 102, a server 103 and a display 104;

[0049] Specifically, the test machine 101 simulates driving scenes for users to test; the test machine 101 can simulate a variety of driving scenes for users to conduct human safety monitoring tests. The test machine 101 is mainly responsible for the operation of the simulated driving scenes and the overall control of the acquisition equipment. The test machine 101 is equipped with a graphics card to achieve smooth operation of the simulated driving scenes, and at the same time, a data storage array is mounted to store the data captured by each acquisition device.

[0050] Each collection device 102 collects basic data of human safety monitoring during the user test process; the human safety monitoring data includes: facial movement information, micro-expression information, physiological signals and driving operation information.

[0051] The experimental machine 101 obtains the human safety monitoring data collected by each acquisition device 102, and sends the human safety monitoring data and the current simulated driving scene information to the server 103; in addition to obtaining the information collected by each acquisition device, the experimental machine also collects its own video output, that is, the simulated environment controlled by the subject (user) (that is, the current simulated driving scene).

[0052] The server 103 calls the dual-branch multi-frame temporal feature aggregation network model to analyze the human safety monitoring data information and the current simulated driving scene information to obtain the human safety monitoring experiment results; the server 103 sends the human safety monitoring experiment results to the display 104 for display. The test machine 101 sends the collected monitoring data and the current simulated driving scene information to the display for display.

[0053] The display 104 displays the human safety monitoring experiment results and monitoring data, which includes: information of each acquisition device and current simulated driving scene information. The display 104 is used as a demonstration device in the platform.

[0054] Specifically, each acquisition device may include, but is not limited to: a depth camera, a visible light camera, a physiological instrument, an eye tracker, a driving steering wheel, pedals, and a gear lever.

[0055] The depth camera is used to perform face modeling on the user and collect facial movement information of the user.

[0056] The visible light camera is used to collect the user's facial image, which includes facial micro-expression information. During the test collection, the visible light camera faces the test machine user to collect image information and capture facial micro-expressions.

[0057] The physiological instrument is used to collect physiological signals of the user, and the physiological signals include at least one of the following: a pulse signal, an electrocardiogram signal, and a skin electrical signal.

[0058] An eye tracker is used to collect the points on the display where the user is looking.

[0059] The steering wheel, pedals and gear lever are used to collect the user's control operation information on the steering wheel, pedals and gear lever during the test. During the test, the user simulates the driving scene by operating the steering wheel, pedals and gear lever.

[0060] Due to the high dynamic and unpredictable characteristics of the simulation environment, the factors brought to the subjects by the simulation environment are constantly changing. Therefore, this platform has designed a driving scene human safety factor rating module to detect targets on the video output of the experimental machine, that is, the visible part of the simulation environment controlled by the subjects, and derive the pressure intensity felt by the subjects during the experiment based on the number of targets, target positions, and target sizes. In driving scenes, common difficult samples include occlusion and blur, and in multi-frame image sequences, the feature information of other unoccluded and clear frames can be used to enhance the detection features of difficult samples, thereby improving the detection performance of the experimental platform.

[0061] The server receives various information collected by the experimental machine, implements the detection of human safety factors of the subjects based on a specific paradigm, and generates a test report output based on the test results.

[0062] Specifically, a dual-branch multi-frame temporal feature aggregation network model is set up in the server. The server analyzes the data collected by the experimental machine through the network model and finally obtains the detection results. The dual-branch multi-frame temporal feature aggregation network model extracts features from adjacent frame images based on the local branches of adjacent frames, and aggregates them with the features of the candidate regions of the frames to be tested. It mainly solves problems such as short-term occlusion and image blur of the target in the frames to be tested. The main feature aggregation object of the global branch based on the global frame is the global frame with a larger interval, which is used to solve difficult samples where the target is occluded for a long time and the adjacent frames are still of low quality.

[0063] Specifically, the dual-branch multi-frame temporal feature aggregation network model includes: a target detection sub-network model, a feature aggregation sub-network model, and a target correction sub-network model; the target detection sub-network model is used to detect the input face image and output the image candidate region and candidate region features; the feature aggregation sub-network model is used to perform multi-frame feature fusion on the candidate region features to obtain the fused candidate region features; the target correction sub-network model is used to correct and refine the prediction of the image candidate region and candidate region features based on the fused candidate region features to obtain the human safety monitoring experiment results. In the actual implementation process, the feature aggregation sub-network model performs multi-frame feature fusion on the candidate region features obtained in the initial inspection according to different aggregation ranges. The human safety monitoring experiment results include: target detection classification results, i.e., target positions.

[0064] More specifically, the target detection sub-network model uses a residual neural network to extract high-dimensional features of the input face image, such as Figure 3 As shown in FIG. 1 , the process of the target detection subnetwork model outputting the image candidate region and the image candidate region features includes:

[0065] S51, obtaining a priori frames to be tested of different sizes at different positions of the input face image according to preset a priori frame parameters;

[0066] S52, extracting the image region feature maps of the to-be-tested priori frames of different sizes to obtain each image region feature map;

[0067] S53, according to the feature maps of each image region, output the length micro-detection category number of each image through the convolution layer to obtain the classification confidence vector of each image (that is, the probability that the current priori region is classified as a certain category);

[0068] S54. According to the classification confidence vectors of the respective images, screening is performed by using a preset probability threshold to obtain respective image candidate regions and image candidate region features.

[0069] More specifically, the feature aggregation subnetwork model includes: local branches and global branches; the feature aggregation objects of the local branches are adjacent frames of the face image frame to be tested, such as the previous 1 frame and the previous 2 frames. The purpose of introducing adjacent frames is to solve the problems of temporary occlusion and image blur of the target in the frame to be tested. The feature aggregation objects of the global branch are face image frames with a preset step length in the time dimension of the face image frame to be tested, such as information of 6 frames and 10 frames in time interval. The long time interval can avoid the influence of continuous occlusion in adjacent frames. The feature aggregation method calculates the feature similarity of candidate regions in different frames as weights for feature aggregation, but it is not sufficient to guide the feature aggregation of two candidate regions based only on the image feature similarity measurement, which may increase redundant or erroneous information and reduce the detection accuracy. Therefore, the position and shape feature vectors are added to encode the center point position and length and width of the candidate region. After linear mapping through a fully connected layer, they are added with the image feature similarity as feature weights for feature aggregation.

[0070] More specifically, the target correction subnetwork model is mainly used for target correction. The purpose of target correction is to use the aggregated information to correct the results of the initial target inspection and improve the accuracy of the final detection results. The target correction subnetwork model includes: a classification prediction branch and a position prediction branch; the classification prediction branch is used to input the candidate area features into the first fully connected layer, where the output dimension of the first fully connected layer is equal to the number of classification categories; the probability of the candidate area being predicted as each category is output; and the category to which the candidate area belongs is determined based on the probability and probability threshold corresponding to each category. In the actual implementation process, if the maximum probability of the probability corresponding to each category is less than the threshold, it is considered that there is no target in the candidate area, and if it is greater than the threshold, it is considered that there is a target in the candidate area, and it is classified into the category corresponding to the maximum probability. The position prediction branch is used to input the candidate area features into the second fully connected layer to predict the target position. The output dimension of the second fully connected layer is four times the number of classification categories. The position prediction branch determines whether there is a target based on the classification result, and outputs the corresponding target position based on the maximum confidence category of the classification prediction.

[0071] The driving scenario human safety monitoring experimental platform provided in the embodiment of the present application visualizes the model detection results on a display and saves the human safety monitoring experimental results.

[0072] on the other hand, Figure 2 A flowchart of a driving scenario human safety monitoring experimental method for implementing an embodiment of the present application is provided. The method is implemented by a driving scenario human safety monitoring experimental platform. The method includes:

[0073] S1. Turn on the experimental machine and simulate the user's driving scenario;

[0074] Figure 4 This is a schematic diagram of the driving scenario human safety monitoring experimental platform. Figure 4 As shown, the subject, i.e. the user, operates the experimental machine, and at the same time, each acquisition device collects simulation scene information and subject's physiological information, and transmits the collected information back to the server. The server processes and analyzes the data, and finally outputs the experimental results, i.e. the test results, and displays the original data on the demonstration device (i.e., the display).

[0075] In this step, the test machine simulates various driving scenarios for users to conduct human safety monitoring in different driving scenarios.

[0076] S2. Turn on the collection device to collect monitoring data during the user's experiment;

[0077] The monitoring data includes: facial information, expression information, physiological signals, and driving operation information.

[0078] For the specific collection method of monitoring data collected by each collection device, please refer to the aforementioned related embodiments and will not be described in detail here.

[0079] S3, obtaining current simulated driving scene information and monitoring data information according to the user driving scene and the monitoring data during the user experiment;

[0080] The experimental machine obtains the monitoring data collected by each acquisition device, and obtains the driving scene information of its own simulation, obtains the current simulated driving scene information and monitoring data information, and sends the obtained current simulated driving scene information and monitoring data information to the server for the server to perform human safety monitoring and analysis.

[0081] S4. According to the current simulated driving scene information and the monitoring data information, a dual-branch multi-frame temporal feature aggregation network model is used to obtain human safety monitoring experiment results;

[0082] For the specific analysis method of the server performing the human safety monitoring experiment based on the current simulated driving scene information and monitoring data information sent by the testing machine, please refer to the relevant instructions in the aforementioned embodiments, which will not be repeated here.

[0083] S5. Displaying the monitoring data and the human safety monitoring experiment results through a display screen according to the monitoring data information and the human safety monitoring experiment results;

[0084] The server sends the experimental results to the display for display, and the display also displays the monitoring data collected by the experimental machine, so that users or others can understand the experimental situation in a timely manner.

[0085] The driving scenario human safety monitoring experimental method provided in the embodiment of the present application can conveniently and flexibly simulate and test the monitoring data information under various driving scenarios. The monitoring data information is analyzed by the dual-branch multi-frame time series feature aggregation network model set in the server, and reliable and comprehensive user human safety monitoring experimental results can be obtained.

[0086] The driving scenario human safety monitoring experimental platform and method provided in the embodiment of the present application provide a scientific way to evaluate and understand the human safety factors of individuals in the face of complex and changeable driving environments. Through this experimental platform, the emotional response, cognitive ability, decision-making ability, and behavioral control of individuals can be measured to fully understand the human safety factors of individuals. Specifically, it can be reflected in the following aspects:

[0087] Identify potential mental health risks: The human safety monitoring experimental platform can help identify potential mental health risks. When individuals deal with complex and changing driving environments, they may face the risk of psychological problems, such as anxiety and depression. Through the experimental platform, these risks can be discovered and identified early, and corresponding interventions and support can be provided to individuals to promote the development of mental health.

[0088] Personalized training and intervention: The results of the human safety monitoring experimental platform can provide individuals with personalized training and intervention programs. According to the individual's test results, a targeted training plan can be formulated to help individuals develop and improve their abilities. These training and interventions can include emotion management skills, cognitive reconstruction, coping strategies, etc., to help individuals better cope with challenges.

[0089] Preventing driving accidents and improving driving safety: In the field of driving, the human safety monitoring experimental platform can help prevent driving accidents and improve driving safety. By evaluating the driver's ability to cope with complex and changing driving environments, potential driving risk factors can be discovered, and corresponding training and intervention can be provided to the driver. This helps the driver improve his coping ability, decision-making ability and reaction speed, thereby reducing the occurrence of driving accidents.

[0090] Promoting individual growth and development: The human safety monitoring experimental platform helps promote individual growth and development. Through the experimental results, individuals can understand their performance and coping methods when facing pressure, recognize their own strengths and challenges, and develop and improve in a targeted manner. This helps individuals improve their adaptability, enhance their self-confidence, and better realize their potential when facing challenges.

[0091] Providing scientific basis and reference: The human safety monitoring experimental platform can provide scientific basis and reference for research and practice. By collecting a large amount of experimental data, researchers can conduct in-depth research on the mechanisms, influencing factors and development patterns of human safety, and provide support for theoretical construction in related fields. At the same time, the results of the experimental platform can also provide a basis for decision makers in the fields of education, vocational training and mental health, and help formulate more effective intervention measures and policies.

[0092] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0093] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A driving scene human safety monitoring experimental platform, characterized in that: The experimental platform includes: multiple acquisition devices, an experimental machine, a server and a display; The acquisition device is used to collect monitoring data during the user experiment, and the monitoring data includes: facial movement information, micro-expression information, physiological signals and driving operation information; The experimental machine is used to simulate driving scenes for user testing, obtain the monitoring data collected by the acquisition device and send the monitoring data and current simulated driving scene information to the server; The server is used to call a dual-branch multi-frame temporal feature aggregation network model to analyze the monitoring data and the current simulated driving scene information to obtain a human safety monitoring experiment result; The display is used to display the monitoring data and the human safety monitoring experiment results.

2. The driving scene human safety monitoring experimental platform according to claim 1 is characterized in that: The acquisition equipment includes: a depth camera, a visible light camera, a physiological instrument, an eye tracker, a driving steering wheel, pedals and a gear lever.

3. The driving scene human safety monitoring experimental platform according to claim 2 is characterized by: The depth camera is used to perform face modeling on the user and collect facial movement information of the user; The visible light camera is used to collect a facial image of the user, wherein the facial image includes facial micro-expression information; The physiological instrument is used to collect physiological signals of the user, wherein the physiological signals include pulse signals, electrocardiogram signals and skin electrical signals; The eye tracker is used to collect the points on the display that the user is looking at; The driving steering wheel, the pedal and the gear lever are used to collect the user's control operation information on the driving steering wheel, the pedal and the gear lever during the test.

4. The driving scene human safety monitoring experimental platform according to claim 1 is characterized in that: The dual-branch multi-frame temporal feature aggregation network model includes: a target detection sub-network model, a feature aggregation sub-network model and a target correction sub-network model; The target detection subnetwork model is used to detect the input face image and obtain the image candidate region and the image candidate region features; The feature aggregation subnetwork model is used to perform multi-frame feature fusion on the candidate region features to obtain fused candidate region features; The target correction sub-network model is used to correct and refine the prediction of the image candidate region and the image candidate region features according to the fused candidate region features to obtain the human safety monitoring experiment results.

5. The driving scene human safety monitoring experimental platform according to claim 4 is characterized in that: The process of the target detection sub-network model outputting the image candidate region and the image candidate region features includes: S51, obtaining a priori frames to be tested of different sizes at different positions of the input face image according to preset a priori frame parameters; S52, extracting the image region feature maps of the to-be-tested priori frames of different sizes to obtain each image region feature map; S53, according to the feature maps of each image region, outputting the number of length micro-detection categories of each image through a convolutional layer to obtain a classification confidence vector of each image; S54. According to the classification confidence vectors of the respective images, screening is performed by using a preset probability threshold to obtain respective image candidate regions and image candidate region features.

6. The driving scene human safety monitoring experimental platform according to claim 4 is characterized in that: The feature aggregation sub-network model includes: a local branch and a global branch; The feature aggregation objects of the local branch include: adjacent frames of the face image frame to be tested; The feature aggregation object of the global branch includes: a face image frame whose distance from the face image frame to be tested to a preset step length in the time dimension.

7. The driving scene human safety monitoring experimental platform according to claim 4 is characterized in that: The target correction sub-network model includes: a classification prediction branch and a position prediction branch; The classification prediction branch is used to input the candidate region features into a first fully connected layer, and the output dimension of the first fully connected layer is equal to the number of classification categories; The position prediction branch is used to input the candidate region features into the second fully connected layer to predict the target position, and the output dimension of the second fully connected layer is four times the number of classification categories.

8. A driving scene human safety monitoring experimental method, characterized in that: The experimental method includes: S1. Turn on the experimental machine and simulate the user's driving scenario; S2. Turn on the collection device to collect monitoring data during the user's experiment; S3, obtaining current simulated driving scene information and monitoring data information according to the user driving scene and the monitoring data during the user experiment; S4. According to the current simulated driving scene information and the monitoring data information, a dual-branch multi-frame temporal feature aggregation network model is used to obtain human safety monitoring experiment results; S5. Display the monitoring data and the human safety monitoring experiment results through a display screen according to the monitoring data information and the human safety monitoring experiment results.

Images