Train-mounted auxiliary system based on personnel behavior recognition and safety monitoring

Abstract

The invention relates to the technical field of safety monitoring, in particular to a train-mounted auxiliary system based on personnel behavior recognition and safety monitoring. The system comprises a vehicle state monitoring module, an acquisition module, an analysis host, an alarm device, a communication module and a monitoring terminal module, firstly, a vehicle state monitoring module monitors whether a vehicle is in a running state or not; secondly, when the vehicle is in a running state, vehicle running data are acquired in real time through an acquisition module; and thirdly, the analysis host receives the vehicle operation data transmitted by the acquisition module in real time, analyzes the driving behavior of the driver and vehicle operation obstacles, obtains an analysis result, and uploads the analysis result to the monitoring terminal module through the communication module. According to the system, whether the driving behavior of a driver is abnormal or not can be monitored in real time, meanwhile, obstacles in the driving process are monitored, the driver is reminded in time, and the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring, and in particular to a train-mounted auxiliary system based on personnel behavior recognition and safety monitoring. Background Art

[0002] With the rapid development of the global economy, rail transportation has become increasingly valued as an efficient and environmentally friendly mode of transportation. The safe operation of trains, improving operational efficiency and reducing accident risks are the main goals of the railway industry. The behavior of train drivers has a direct impact on train safety, so monitoring and analyzing driver behavior has become an important research direction for the railway system.

[0003] The existing technology monitors the driver's behavior through video surveillance and sensors. Although it can realize the monitoring of the driver's behavior, there are still problems such as untimely monitoring and inaccurate identification. For example, fatigue monitoring often relies on simple eye closure time and lacks in-depth analysis, and the identification of distracted behavior is also limited by the accuracy and real-time performance of the algorithm. The present invention aims to provide an advanced monitoring system that can realize real-time and accurate monitoring of train driver behavior through deep learning algorithms and high-precision sensors, effectively warn of unsafe behaviors such as fatigue, and improve driving safety.

[0004] Therefore, a train on-board assistance system based on personnel behavior recognition and safety monitoring is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a train on-board auxiliary system based on personnel behavior recognition and safety monitoring, the system comprising a vehicle status monitoring module, a collection module, an analysis host, an alarm device, a communication module and a monitoring terminal module; first, the vehicle status monitoring module is used to monitor whether the vehicle is in a running state; secondly, the vehicle operation data is obtained in real time through the collection module when the vehicle is in a running state; thirdly, the analysis host receives the vehicle operation data transmitted in real time by the collection module, analyzes the driving behavior of the driver and the obstacles to the vehicle operation, obtains the analysis results, and uploads them to the monitoring terminal module through the communication module.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A train-mounted auxiliary system based on personnel behavior recognition and safety monitoring includes: a vehicle status monitoring module, a collection module, an analysis host, an alarm device, a communication module and a monitoring terminal module.

[0008] The vehicle status monitoring module is used to monitor whether the vehicle is in a running state;

[0009] The acquisition module is used to acquire vehicle operation data in real time when the vehicle is in operation;

[0010] The analysis host is used to receive the vehicle operation data transmitted in real time by the acquisition module, and analyze the driving behavior of the driver and the vehicle operation obstacles to obtain the analysis results;

[0011] The analysis host includes a driver identity authentication unit, a driving behavior analysis unit and a vehicle operation obstacle analysis unit; the analysis results include a driver identification result, a driving behavior identification result and a vehicle operation obstacle identification result.

[0012] The vehicle running obstacle analysis unit is used to identify and obtain the vehicle running obstacles and calculate the running obstacle hazard level; the specific process includes:

[0013] Calculating the driving behavior risk according to the driving behavior recognition result obtained by the driving behavior analysis unit;

[0014] Obtaining the danger degree of the running obstacle according to the driving behavior danger degree and the characteristic data of the vehicle running obstacle; the characteristic data of the vehicle running obstacle includes the size and position of each of the vehicle running obstacles;

[0015] According to the running obstacle danger degree, the running obstacle danger level is obtained; according to the running obstacle danger level, the alarm device is controlled to issue a corresponding alarm prompt.

[0016] Furthermore, the acquisition module includes: a driver identity feature acquisition unit, an RFID collector, a first camera, a second camera, a millimeter wave radar, an ultra-long-range laser radar, and a data transmission unit;

[0017] The driver identity characteristic acquisition unit is used to acquire the driving characteristics of the current target driver of the vehicle;

[0018] The RFID collector is used to collect the driver's RFID tag information;

[0019] The first camera is used to collect the front driving video of the driver;

[0020] The second camera is used to collect the driver's hand driving video;

[0021] The millimeter wave radar is used to transmit millimeter wave signals into the cab and receive echo signals from the cab;

[0022] The ultra-long-range laser radar is used to obtain the operation point cloud data within the vehicle operation area;

[0023] The data transmission unit is used to transmit the vehicle operation data to the analysis host in real time; the vehicle operation data includes: the driver's RFID tag information, the driver's front driving video, the driver's hand driving video, the cab echo signal, the operation point cloud data and the driving characteristics of the target driver.

[0024] Furthermore, the driving characteristics of the current target driver of the vehicle include: identity characteristics and driving behavior characteristics of the target driver in a pre-collected driver database; the identity characteristics include the name, work number and facial image of the target driver; the driving behavior characteristics include standard driving eye appearance feature data and standard driving breathing feature data corresponding to the target driver.

[0025] Furthermore, the driver identity verification unit is used to verify the identity of the driver, and the specific process includes: comparing the driver's RFID tag information with the driving characteristics of the target driver; the RFID tag information includes the driver's name, work number and face image;

[0026] When the RFID tag information and the identity characteristics in the driving characteristics of the target driver are inconsistent, the driver identity verification unit determines that the driver identification result is abnormal;

[0027] When the RFID tag information is consistent with the identity characteristics in the driving characteristics of the target driver, obtain The frontal driving video within the time period is recorded as the real-time frontal driving video. Represents the current time, Expressed as the first time threshold; extracting the front driving video frame image in the real-time front driving video; extracting the facial features of each of the front driving video frame images, and comparing the facial features with the facial features in the identity features to obtain the facial feature similarity; when there are consecutive Z frames corresponding to the front driving video frame images whose facial feature similarity is less than the set facial similarity threshold, judging that the driver recognition result is abnormal; otherwise, obtaining the clothing features of the front driving video frame images, and obtaining the clothing feature similarity; when there are consecutive F frames corresponding to the front driving video frame images whose clothing feature similarity is less than the set clothing feature similarity threshold, judging that the driver recognition result is abnormal; otherwise, judging that the driver recognition result is normal.

[0028] Furthermore, the driving behavior analysis unit is used to analyze the driving behavior of the driver when the driver identification result is normal, and obtain a driving behavior identification result; the driving behavior identification result includes normal driving and abnormal driving;

[0029] The abnormal driving includes fatigue driving and abnormal hand movements; the steps of obtaining the fatigue driving include:

[0030] Based on the real-time front driving video, obtain the driver's When there is a face image in each frame of the real-time front driving video, obtain the driver's face image in the The eye feature data and mouth feature data within a time period; the eye feature data includes blinking frequency, eye closing time, sight deviation time and eye appearance features; the eye appearance features include white eye color features, eyelid color features and eye periorbital color features; the mouth feature data includes The number of yawns during a period of time;

[0031] Obtain a dynamic eye fatigue value based on the blinking frequency, eye closing time and line of sight deviation time; obtain a static eye fatigue value based on the eye appearance characteristics and standard driving eye appearance characteristic data; obtain an eye fatigue value based on the dynamic eye fatigue value and the static eye fatigue value; obtain a first fatigue driving value of the driver based on the eye fatigue value and the mouth feature data, and determine whether the driver is fatigued driving based on the first fatigue driving value.

[0032] Furthermore, when there is no face image in each frame of the real-time front driving video, obtaining The echo signal within the time period is used to construct an echo signal set; based on the echo signal set, the Driving breathing characteristic data of the driver in a time period; obtaining a second fatigue driving value of the driver based on the driving breathing characteristic data and standard driving breathing characteristic data in the driving behavior characteristics, and judging whether the driver is fatigued driving based on the second fatigue driving value.

[0033] Furthermore, the step of obtaining abnormal hand movements includes: obtaining The hand driving video within the time period is recorded as the real-time hand driving video; based on the real-time hand driving video, the Whether the driver has any abnormal hand movements during the time period.

[0034] Furthermore, the vehicle running obstacle analysis unit includes:

[0035] Acquire the running point cloud data at the current time, and pre-process the running point cloud data;

[0036] Inputting the preprocessed running point cloud data into the vehicle DGCNN model to obtain local running point cloud features;

[0037] The vehicle DGCNN model is based on the DGCNN model, and a self-attention module is added after each edge convolution module of the backbone network to extract the first operating point cloud feature of the operating point cloud data; at the same time, GCN is used to extract the second operating point cloud feature of the operating point cloud data, and the first operating point cloud feature and the second operating point cloud feature are weighted to obtain the operating point cloud feature; the operating obstacles are identified according to the operating point cloud features; when the operating obstacles are identified, the size of each of the operating obstacles is obtained; when the size of the operating obstacle is greater than the set size threshold, the operating obstacle is marked as a vehicle operating obstacle; the feature data of each of the vehicle operating obstacles is obtained, and the danger level of the operating obstacle is calculated.

[0038] Furthermore, the communication module is used to receive the analysis results and upload them to the monitoring terminal module; at the same time, the analysis host controls the alarm device to issue a corresponding alarm prompt;

[0039] Furthermore, the monitoring terminal module is used to record, store, and display the acquired analysis results and to remind the backstage monitoring personnel.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The present invention obtains a real-time frontal driving video of the driver. When a face image exists in each frame of the real-time frontal driving video, the blinking frequency, eye closing time, sight line deviation time and eye appearance characteristics of the driver in the real-time frontal driving video are obtained, and the dynamic eye fatigue value and the static eye fatigue value are obtained. This method can combine the dynamic eye features and the static eye features, can more accurately reflect the degree of eye fatigue, and can timely identify whether it is fatigue driving, thereby improving driving safety.

[0042] 2. The present invention constructs a vehicle DGCNN model, which can more accurately obtain the local features of train running point cloud data by adding a self-attention mechanism to the backbone network; at the same time, integrating GCN into the edge convolution module can make full use of the global feature aggregation capability of GCN and the local feature extraction capability of DGCNN, which can improve the representation capability of point cloud features, thereby more accurately identifying obstacles and improving driving safety.

[0043] 3. The present invention obtains the danger level of vehicle running obstacles based on the characteristic data of vehicle running obstacles and the abnormal behavior of the driver. When the driver has abnormal behavior, the driver's reaction time will be prolonged. This method can adjust the range of vehicle running obstacles that need to be monitored according to the abnormal behavior of the driver, calculate the potential risks in real time, and remind the driver through the early warning system to improve driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A structural diagram of a train-mounted auxiliary system based on personnel behavior recognition and safety monitoring provided by the present invention;

[0045] Figure 2 A schematic diagram of the structure of a collection module provided in an embodiment of the present invention;

[0046] Figure 3 A detection flow chart of an off-post detection unit provided in an embodiment of the present invention;

[0047] Figure 4 A flowchart of driver identity verification provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention provides a train-mounted auxiliary system based on personnel behavior recognition and safety monitoring. Figure 1 A train-mounted auxiliary system based on personnel behavior recognition and safety monitoring includes a vehicle status monitoring module, a collection module, an analysis host, an alarm device, a communication module and a monitoring terminal module. Figure 2, the acquisition module includes a driver identity feature acquisition unit, an RFID collector, a first camera, a second camera, a millimeter-wave radar, an ultra-long-range laser radar and a data transmission unit. The vehicle status monitoring module, the driver identity feature acquisition unit, the RFID collector, the first camera, the second camera, the millimeter-wave radar, the data transmission unit, the analysis host, the alarm device and the communication module are installed in the cab; the ultra-long-range laser radar is installed in the external area on both sides of the vehicle; the monitoring terminal is installed in the ground monitoring center; the vehicle status monitoring module is used to monitor whether the vehicle is in operation; the acquisition module is used to obtain vehicle operation data in real time when the vehicle is in operation; the vehicle operation data includes: the driver's RFID tag information, the driver's front driving video and the driver's hand driving video, the cab echo signal, the operation The analysis host is used to receive the vehicle operation data transmitted by the acquisition module in real time, and analyze the driving behavior of the driver and the vehicle operation obstacles to obtain the analysis results; the communication module is used to receive the analysis results and upload them to the monitoring terminal module; at the same time, the analysis host controls the alarm device to issue a corresponding alarm prompt; the alarm device includes a speaker, a microphone and a flash light, the alarm device receives the control signal sent by the analysis host, and sends a corresponding alarm message through the speaker and turns on the flash light for light reminder, and the alarm device can use the microphone to communicate with the background monitoring personnel.

[0050] The acquisition unit obtains various data in the cab and in front of the vehicle, and inputs the acquired data into the analysis host. The analysis host monitors the driver's driving behavior and vehicle running obstacles based on different data, and combines the driving behavior and vehicle running obstacles to obtain the danger level of running obstacles and issue an early warning. This method can combine various data in vehicle operation to achieve joint monitoring of the vehicle's interior and exterior, and improve vehicle driving safety.

[0051] Embodiment 1

[0052] This embodiment applies the system to the subway to monitor the driver's behavior and vehicle running obstacles during the operation of the subway.

[0053] Reference Figure 1 The vehicle status monitoring module is used to monitor whether the subway is in operation; when the subway is started or the subway is in a non-zero speed state, the vehicle status monitoring module determines that the subway is in operation, and the on-board auxiliary system automatically turns on monitoring and identification; otherwise, the vehicle status monitoring module determines that the subway is not in operation, and the on-board auxiliary system automatically turns off monitoring and identification.

[0054] Reference Figure 1 The acquisition module is used to obtain vehicle operation data in real time when the vehicle is in operation.

[0055] Furthermore, the acquisition module includes: a driver identity feature acquisition unit, an RFID collector, a first camera, a second camera, a millimeter wave radar, an ultra-long-range laser radar, and a data transmission unit;

[0056] The driver identity characteristic acquisition unit is used to acquire the driving characteristics of the current target driver of the vehicle;

[0057] The RFID collector is used to collect the driver's RFID tag information;

[0058] The first camera is used to collect the front driving video of the driver;

[0059] The second camera is used to collect the driver's hand driving video;

[0060] The millimeter wave radar is used to transmit millimeter wave signals into the cab and receive echo signals from the cab;

[0061] The ultra-long-range laser radar is used to obtain the operation point cloud data within the vehicle operation area;

[0062] The data transmission unit is used to transmit the vehicle operation data to the analysis host in real time; the vehicle operation data includes: the driver's RFID tag information, the driver's front driving video, the driver's hand driving video, the cab echo signal, the operation point cloud data and the driving characteristics of the target driver.

[0063] In this embodiment, this step acquires various data through the acquisition module, including data in the cab, driver data, and data in the vehicle operation area. This step provides a data basis for the host to analyze driving behavior and vehicle operation obstacles.

[0064] Furthermore, the driving characteristics of the current target driver of the vehicle include: identity characteristics and driving behavior characteristics of the target driver in a pre-collected driver database; the identity characteristics include the name, work number and facial image of the target driver; the driving behavior characteristics include standard driving eye appearance feature data and standard driving breathing feature data corresponding to the target driver; the standard driving eye appearance feature data are eye appearance features collected from the target driver in a non-fatigue state; the standard driving breathing feature data are driving breathing frequency and driving heart rate collected from the target driver in a non-fatigue state, recorded as standard driving breathing frequency and standard driving heart rate.

[0065] This step of the present embodiment obtains the identity characteristics and driving behavior characteristics of the target driver. This method can provide a data basis for subsequent driver identity recognition and subsequent driving behavior monitoring.

[0066] Reference Figure 1 The analysis host is used to receive the vehicle operation data transmitted in real time by the acquisition module, and analyze the driving behavior of the driver and the vehicle operation obstacles to obtain the analysis results.

[0067] Furthermore, the analysis host is a 1U model, installed in the cab, and is electrically connected to the acquisition module, alarm device and communication module; the analysis host includes a driver identity authentication unit, a driving behavior analysis unit and a vehicle operation obstacle analysis unit; the analysis results include driver identification results, driving behavior identification results and vehicle operation obstacle identification results.

[0068] Furthermore, the analysis host also includes an off-duty detection unit; the analysis result also includes an off-duty identification result; when the off-duty detection unit detects that the driver is off-duty, the driver identity authentication unit and the driving behavior analysis unit do not perform analysis.

[0069] Further, refer to Figure 3 , the detection process of the off-post detection unit includes:

[0070] right The frontal driving video within the time period is analyzed; Represents the current time, It is expressed as the time threshold for off-duty detection; Can be set to 60 seconds and 120 seconds, etc. When a face is detected in the front driving video within the time period, the off-duty recognition result of the off-duty detection unit is that the driver has not left his post; when When no face is detected in the front driving video within the time period, obtain The cab echo signal received by the millimeter-wave radar within the time period; the cab echo signal is identified to obtain the cab echo signal identification result; the cab echo signal identification result includes whether there is someone in the cab and whether there is no one in the cab; when the cab echo signal identification result is that there is someone in the cab, the off-duty identification result of the off-duty detection unit is that the driver has not left his post; otherwise, the off-duty identification result of the off-duty detection unit is that the driver has left his post. When the off-duty detection unit identification result is that the driver has left his post, the communication module uploads the off-duty detection unit identification result to the monitoring terminal module, and at the same time, the analysis host controls the alarm device to issue an off-duty alarm prompt.

[0071] Further, when the off-duty detection unit identifies that the driver is on duty, the driver identity verification unit is used to verify the identity of the driver, referring to Figure 4 The specific process includes:

[0072] Comparing the driver's RFID tag information with the target driver's driving characteristics; the RFID tag information includes the driver's name, work number and facial image;

[0073] When the RFID tag information and the identity characteristics in the driving characteristics of the target driver are inconsistent, the driver identity verification unit determines that the driver identification result is abnormal;

[0074] When the RFID tag information is consistent with the identity characteristics in the driving characteristics of the target driver, obtain The frontal driving video within the time period is recorded as the real-time frontal driving video. Represents the current time, It is represented as the first time threshold; It can be set to 60 seconds and 120 seconds, etc. to extract the front driving video frame image in the real-time front driving video; extract the facial features of each of the front driving video frame images, and compare the facial features with the facial features in the identity features to obtain the facial feature similarity; when there are continuous Z frames corresponding to the front driving video frame images whose facial feature similarity is less than the set facial similarity threshold, the driver recognition result is judged to be abnormal; the facial similarity threshold is obtained based on experience; Z can be set to 60 and 120, etc.; otherwise, obtain the clothing features of the front driving video frame images, compare the clothing features with the standard clothing features, and obtain the clothing feature similarity; when there are continuous F frames corresponding to the front driving video frame images whose clothing feature similarity is less than the set clothing feature similarity threshold, the driver recognition result is judged to be abnormal; otherwise, the driver recognition result is judged to be normal; F can be set to 60 and 120, etc.; the clothing feature similarity threshold is obtained based on experience. When the identification result of the driver identity authentication unit is that the driver identification is abnormal, the communication module uploads the identification result of the driver identity authentication unit to the monitoring terminal module, and at the same time, the analysis host controls the alarm device to issue an identity abnormality alarm prompt.

[0075] In this embodiment, the step identifies the driver's identity based on the driver's RFID information and front image information, and the identity identification includes not only face image recognition but also clothing compliance recognition. This method can significantly improve the security and accuracy of driver identification through this multiple identity authentication mechanism.

[0076] Furthermore, the driving behavior analysis unit is used to analyze the driving behavior of the driver when the driver identification result is normal, and obtain a driving behavior identification result; the driving behavior identification result includes normal driving and abnormal driving;

[0077] The abnormal driving includes fatigue driving and abnormal hand movements; the steps of obtaining the fatigue driving include:

[0078] Based on the real-time front driving video, obtain the driver's When there is a face image in each frame of the real-time front driving video, obtain the driver's face image in the The eye feature data and mouth feature data within a time period; the eye feature data includes blinking frequency, eye closing time, sight deviation time and eye appearance features; the eye appearance features include white eye color features, eyelid color features and eye periorbital color features; the mouth feature data includes The number of yawns during a period of time;

[0079] Obtaining a dynamic eye fatigue value according to the blinking frequency, eye closing time and line of sight deviation time; obtaining a static eye fatigue value according to the eye appearance characteristics and standard driving eye appearance characteristic data; obtaining an eye fatigue value according to the dynamic eye fatigue value and the static eye fatigue value;

[0080] The calculation formula of the dynamic eye fatigue value is:

[0081] ;

[0082] in, Indicated as a basis The dynamic eye fatigue value at the current moment obtained from the face image within the time period; Expressed as The driver's blinking frequency during the time period; Expressed as The driver’s eye closing time during the time period; Expressed as The driver’s sight deviation time during the time period; , and They are respectively expressed as blink frequency coefficient, eye closure time coefficient and gaze deviation coefficient; , , and The default is , which can also be changed according to the actual application;

[0083] The calculation process of the static eye fatigue value is as follows:

[0084] Randomly select M video frame images with face images in the real-time front driving video, and extract eye appearance features in the M video frame images; compare the eye appearance features with the standard driving eye appearance features corresponding to the target driver to obtain the similarity of the eye appearance features; average the similarities of the eye appearance features corresponding to the M video frame images to obtain the average value of the eye appearance feature similarities; the calculation formula of the static eye fatigue value is:

[0085] ;

[0086] in, Indicated as a basis The static eye fatigue value at the current moment obtained from the face image within the time period; It is expressed as the average value of the similarity of the eye appearance features;

[0087] The calculation formula of the eye fatigue value is:

[0088] ;

[0089] in, Indicated as a basis The eye fatigue value at the current moment obtained from the face image within the time period; and Expressed as dynamic fatigue coefficient and static fatigue coefficient respectively; and The default value is 0.5, which can be changed according to the actual application; Expressed as an exponential with a natural constant as base.

[0090] According to the eye fatigue value and the mouth feature data, a first fatigue driving value of the driver is obtained, and according to the first fatigue driving value, whether the driver is driving fatigued is determined; the calculation formula of the first fatigue driving value is:

[0091] ;

[0092] in, It is represented by the first driving fatigue value of the driver at the current time obtained based on the real-time front driving video; Expressed as The number of times the driver yawns during the time period; and They are expressed as eye fatigue coefficient and mouth fatigue coefficient respectively; and The default value is 0.5, which can be changed according to the actual application.

[0093] When the first fatigue driving value is not greater than the set first fatigue threshold, the driver is judged to be non-fatigued driving; when the first fatigue driving value is greater than the set first fatigue threshold, the driver is judged to be fatigued driving, and fatigue is graded according to the first fatigue driving value; the fatigue grades include mild fatigue, moderate fatigue and severe fatigue; the first fatigue threshold is obtained based on experience.

[0094] This step of the present embodiment obtains the real-time front driving video of the driver. When there is a face image in each frame of the real-time front driving video, the blinking frequency, eye closing time, line of sight deviation time and eye appearance characteristics of the driver in the real-time front driving video are obtained, and the dynamic eye fatigue value and the static eye fatigue value are obtained. This method can combine the dynamic features of the eyes and the static features of the eyes, and can more accurately reflect the degree of eye fatigue. At the same time, the first fatigue driving value is obtained in combination with the eye fatigue value and the mouth feature data, and it is judged whether the driver is fatigued driving and the corresponding fatigue level. This method can comprehensively obtain the driver's facial information, so as to accurately evaluate the driver's fatigue state.

[0095] Furthermore, when there is no face image in each frame of the real-time front driving video, obtaining The echo signal within the time period is used to construct an echo signal set; based on the echo signal set, the The driving breathing characteristic data of the driver in the time period; the driving breathing characteristic data includes the breathing frequency and the heart rate; the standard driving breathing characteristic data includes the standard driving breathing frequency and the standard driving heart rate; according to the driving breathing characteristic data and the standard driving breathing characteristic data in the driving behavior characteristics, the second fatigue driving value of the driver is obtained, and according to the second fatigue driving value, whether the driver is fatigue driving is determined; the calculation formula of the second fatigue driving value is:

[0096] ;

[0097] in, It is represented by the second driving fatigue value of the driver at the current time obtained according to the echo signal set; Expressed as The breathing rate of the driver during the time period; Expressed as The driver's heart rate during the time period; and They are expressed as standard driving breathing rate and standard driving heart rate respectively; and All are nonlinear functions; and They are expressed as respiratory rate coefficient and heart rate coefficient respectively; and The default value is 0.5, which can be changed according to the actual application.

[0098] When the second fatigue driving value is not greater than the set second fatigue threshold, the driver is judged to be non-fatigued driving; when the second fatigue driving value is greater than the set second fatigue threshold, the driver is judged to be fatigued driving, and fatigue is graded according to the second fatigue driving value; the fatigue grades include mild fatigue, moderate fatigue and severe fatigue; the second fatigue threshold is obtained based on experience.

[0099] In this step of the present embodiment, when there are no facial images in the front driving video, the millimeter wave radar is used to obtain the driver's breathing rate and heart rate, and it is compared with the standard driving breathing rate and standard driving heart rate of the target driver to obtain the second fatigue driving value, and determine whether the driver is fatigued driving and the corresponding fatigue level. This method can evaluate the driver's fatigue state when the driver's facial information cannot be obtained.

[0100] Furthermore, the step of obtaining abnormal hand movements includes: obtaining The hand driving video within the time period is recorded as the real-time hand driving video; based on the real-time hand driving video, the Whether the driver has any abnormal hand movements during the time period; abnormal hand movements include: leaving the hands on the driving console for more than 30 seconds and holding a mobile phone for more than 30 seconds, etc.

[0101] This step of the present embodiment monitors the driver's hand movements, identifies whether the driver has abnormal hand movements, and issues a warning if the hand movements are abnormal, thereby ensuring driving safety.

[0102] Furthermore, when the driving behavior analysis unit analyzes the driving behavior and obtains that the driver is not driving fatigued and there is no abnormal hand movement, the driving behavior recognition result is normal driving. When the driving behavior analysis unit obtains that the driver is driving abnormally, the alarm device is controlled to issue a corresponding alarm prompt.

[0103] Furthermore, the vehicle running obstacle analysis unit includes:

[0104] Acquire the running point cloud data at the current time, and pre-process the running point cloud data;

[0105] Inputting the preprocessed running point cloud data into the vehicle DGCNN model to obtain local running point cloud features;

[0106] The vehicle DGCNN model is based on the DGCNN model, and a self-attention module is added after each edge convolution module of the backbone network to extract the first operating point cloud feature of the operating point cloud data; at the same time, GCN is used to extract the second operating point cloud feature of the operating point cloud data, and the first operating point cloud feature and the second operating point cloud feature are weighted to obtain the operating point cloud feature; the operating obstacles are identified according to the operating point cloud features; when the operating obstacles are identified, the size of each of the operating obstacles is obtained, and the size is the volume of the operating obstacle; when the size of the operating obstacle is greater than the set size threshold, the operating obstacle is marked as a vehicle operating obstacle; the feature data of each of the vehicle operating obstacles is obtained, and the danger level of the operating obstacle is calculated.

[0107] In order to verify the effectiveness of the vehicle DGCNN model provided in this embodiment, this embodiment selected 100 point cloud data in vehicle operation, each point cloud data contains at least one operation obstacle, 100 point cloud data contain 340 operation obstacles, and 340 operation obstacles contain 175 vehicle operation obstacles. The 340 point cloud data are input into four different models respectively, and each model is repeated five times, and the average recognition accuracy of different models for operation obstacles and vehicle operation obstacles in the five repeated tests is obtained. The specific results are shown in Tables 1 and 2.

[0108] Table 1 Operation obstacle identification verification table

[0109] Model Model 1 Model 2 Model 3 Model 4 Average precision 0.907 0.853 0.827 0.728 Average recall 0.916 0.835 0.843 0.734 Average F1-Score 0.911 0.841 0.835 0.732

[0110] Table 2 Vehicle operation obstacle identification verification table

[0111] Model Model 1 Model 2 Model 3 Model 4 Average precision 0.935 0.862 0.813 0.737 Average recall 0.927 0.843 0.827 0.751 Average F1-Score 0.929 0.849 0.816 0.745

[0112] Model 1 in Table 1 and Table 2 is the vehicle DGCNN model provided in this embodiment; Model 2 is based on Model 1 without adding a self-attention module after each edge convolution module of the backbone network; Model 3 is based on Model 1 without extracting the second running point cloud features of the running point cloud data through GCN; Model 4 is the DGCNN model. It can be seen from Table 1 and Table 2 that the vehicle DGCNN model provided in this embodiment can improve the recognition accuracy of running obstacles and vehicle running obstacles.

[0113] This step of the present embodiment constructs a vehicle DGCNN model. The vehicle DGCNN model can more accurately obtain the local features of the train running point cloud data by adding a self-attention mechanism to the backbone network. At the same time, integrating GCN into the edge convolution module can make full use of the global feature aggregation capability of GCN and the local feature extraction capability of DGCNN, which can improve the representation capability of point cloud features and thus more accurately identify obstacles.

[0114] Furthermore, the calculation process of the running obstacle risk includes:

[0115] According to the driving behavior recognition result obtained by the driving behavior analysis unit, the driving behavior risk is calculated, and the calculation formula is:

[0116] ;

[0117] in, It is represented by the driving behavior risk level at the current time obtained based on the driving behavior recognition result; Indicates the label value corresponding to the fatigue level; in particular, when the driver is not driving fatigue-free, ; When the driver is driving fatigued, the corresponding label value is obtained according to the fatigue level; for example: when the fatigue level is mild fatigue, ; When the fatigue level is moderate fatigue, ; When the fatigue level is severe fatigue, ; Indicates the label value corresponding to the hand movement; when there is an abnormal hand movement ;otherwise, ; and They are respectively expressed as fatigue driving coefficient and hand motion coefficient; and The default value is 0.5, which can be changed according to the actual application.

[0118] According to the driving behavior danger degree and the characteristic data of the vehicle running obstacle, the danger degree of the running obstacle is obtained; the characteristic data of the vehicle running obstacle includes the size and position of each of the vehicle running obstacles; according to the position of the vehicle running obstacle, the distance between each of the vehicle running obstacles and the current position of the vehicle is obtained; the characteristic data of the vehicle running obstacles whose distance is less than the driving danger distance are screened out; the calculation formula of the driving danger distance is:

[0119] ;

[0120] in, It is indicated as driving danger distance; Represented as the set distance threshold; Expressed as distance coefficient;

[0121] The danger degree of the running obstacle is obtained according to the characteristic data of the vehicle running obstacle within the driving danger distance, and the calculation formula is:

[0122] ;

[0123] in, It is represented as the danger degree of running obstacles; W is represented as the number of running obstacles within the driving danger distance; in particular, when W=0, ; It is represented as the distance between the wth vehicle running obstacle and the current position of the vehicle; It is represented as the size of the current position of the w-th vehicle;

[0124] According to the running obstacle danger degree, the running obstacle danger level is obtained; according to the running obstacle danger level, the alarm device is controlled to issue a corresponding alarm prompt;

[0125] The communication module is used to receive the analysis result and upload it to the monitoring terminal module; at the same time, the analysis host controls the alarm device to issue a corresponding alarm prompt.

[0126] This embodiment connects the analysis host and the alarm device. When the analysis host analyzes that there is an abnormality in the driving process, the alarm device can promptly remind the driver according to the abnormal situation, thereby ensuring driving safety.

[0127] The monitoring terminal module is used to record, store, display and alert the backstage monitoring personnel of the acquired analysis results.

[0128] This embodiment provides a monitoring terminal module, which records and stores abnormal behaviors during driving, so as to generate regular reports on driving behaviors and evaluate driving safety. At the same time, the monitoring terminal module displays abnormal behaviors during driving and reminds background monitoring personnel, which can help background monitoring personnel identify potential safety risks in a timely manner and ensure driving safety.

[0129] Embodiment 2

[0130] This embodiment applies the system to trains to monitor driver behavior and vehicle running obstacles during train operation.

[0131] Reference Figure 1The vehicle status monitoring module in the vehicle status monitoring module is used to monitor whether the train is in a running state; when the train is started or the train is in a non-zero speed state, the vehicle status monitoring module determines that the train is in a running state, and the on-board auxiliary system automatically turns on monitoring and identification; otherwise, the vehicle status monitoring module determines that the train is in a non-running state, and the on-board auxiliary system automatically turns off monitoring and identification.

[0132] Reference Figure 1 The acquisition module is used to obtain vehicle operation data in real time when the vehicle is in operation.

[0133] Furthermore, the acquisition module includes: a driver identity feature acquisition unit, an RFID collector, a first camera, a second camera, a millimeter wave radar, an ultra-long-range laser radar, and a data transmission unit;

[0134] The driver identity characteristic acquisition unit is used to acquire the driving characteristics of the current target driver of the vehicle;

[0135] The RFID collector is used to collect the driver's RFID tag information;

[0136] The first camera is used to collect the front driving video of the driver;

[0137] The second camera is used to collect the driver's hand driving video;

[0138] The millimeter wave radar is used to transmit millimeter wave signals into the cab and receive echo signals from the cab;

[0139] The ultra-long-range laser radar is used to obtain the operation point cloud data within the vehicle operation area;

[0140] The data transmission unit is used to transmit the vehicle operation data to the analysis host in real time; the vehicle operation data includes: the driver's RFID tag information, the driver's front driving video, the driver's hand driving video, the cab echo signal, the operation point cloud data and the driving characteristics of the target driver.

[0141] Furthermore, the driving characteristics of the current target driver of the vehicle include: identity characteristics and driving behavior characteristics of the target driver in a pre-collected driver database; the identity characteristics include the name, work number and facial image of the target driver; the driving behavior characteristics include standard driving eye appearance feature data and standard driving breathing feature data corresponding to the target driver; the standard driving eye appearance feature data are eye appearance features collected from the target driver in a non-fatigue state; the standard driving breathing feature data are driving breathing frequency and driving heart rate collected from the target driver in a non-fatigue state, recorded as standard driving breathing frequency and standard driving heart rate.

[0142] Reference Figure 1 The analysis host is used to receive the vehicle operation data transmitted in real time by the acquisition module, and analyze the driving behavior of the driver and the vehicle operation obstacles to obtain the analysis results.

[0143] Furthermore, the analysis host is a 1U model, installed in the cab, and the analysis host is electrically connected to the acquisition module, alarm device, and communication module; the analysis host includes a driver identity authentication unit, a driving behavior analysis unit, and a vehicle operation obstacle analysis unit; the analysis results include driver identification results, driving behavior identification results, and vehicle operation obstacle identification results.

[0144] Furthermore, the analysis host also includes an off-duty detection unit; the analysis result also includes an off-duty identification result; when the off-duty detection unit detects that the driver is off-duty, the driver identity authentication unit and the driving behavior analysis unit do not perform analysis.

[0145] Furthermore, the detection process of the off-duty detection unit includes:

[0146] right The frontal driving video within the time period is analyzed; Represents the current time, It is represented as the off-duty detection time threshold; when When a face is detected in the front driving video within the time period, the off-duty recognition result of the off-duty detection unit is that the driver has not left his post; when When no face is detected in the front driving video within the time period, obtain The cab echo signal received by the millimeter-wave radar within the time period; the cab echo signal is identified to obtain the cab echo signal identification result; the cab echo signal identification result includes whether there is someone in the cab and whether there is no one in the cab; when the cab echo signal identification result is that there is someone in the cab, the off-duty identification result of the off-duty detection unit is that the driver has not left his post; otherwise, the off-duty identification result of the off-duty detection unit is that the driver has left his post. When the off-duty detection unit identification result is that the driver has left his post, the communication module uploads the off-duty detection unit identification result to the monitoring terminal module, and at the same time, the analysis host controls the alarm device to issue an off-duty alarm prompt.

[0147] Further, when the identification result of the off-duty detection unit is that the driver is on duty, the driver identity verification unit is used to verify the identity of the driver, and the specific process includes:

[0148] Comparing the driver's RFID tag information with the target driver's driving characteristics; the RFID tag information includes the driver's name, work number and facial image;

[0149] When the RFID tag information and the identity characteristics in the driving characteristics of the target driver are inconsistent, the driver identity verification unit determines that the driver identification result is abnormal;

[0150] When the RFID tag information is consistent with the identity characteristics in the driving characteristics of the target driver, obtain The frontal driving video within the time period is recorded as the real-time frontal driving video. Represents the current time, It is represented as the first time threshold; It can be set to 60 seconds and 120 seconds, etc. to extract the front driving video frame image in the real-time front driving video; extract the facial features of each of the front driving video frame images, and compare the facial features with the facial features in the identity features to obtain the facial feature similarity; when there are continuous Z frames corresponding to the front driving video frame images whose facial feature similarity is less than the set facial similarity threshold, the driver recognition result is judged to be abnormal; the facial similarity threshold is obtained based on experience; Z can be set to 60 and 120, etc.; otherwise, obtain the clothing features of the front driving video frame images, and obtain the clothing feature similarity; when there are continuous F frames corresponding to the front driving video frame images whose clothing feature similarity is less than the set clothing feature similarity threshold, the driver recognition result is judged to be abnormal; otherwise, the driver recognition result is judged to be normal; F can be set to 60 and 120, etc.; the clothing feature similarity threshold is obtained based on experience. When the identification result of the driver identity authentication unit is that the driver identification is abnormal, the communication module uploads the identification result of the driver identity authentication unit to the monitoring terminal module, and at the same time, the analysis host controls the alarm device to issue an identity abnormality alarm prompt.

[0151] Furthermore, the driving behavior analysis unit is used to analyze the driving behavior of the driver when the driver identification result is normal, and obtain a driving behavior identification result; the driving behavior identification result includes normal driving and abnormal driving;

[0152] The abnormal driving includes fatigue driving and abnormal hand movements; the steps of obtaining the fatigue driving include:

[0153] Based on the real-time front driving video, obtain the driver's When there is a face image in each frame of the real-time front driving video, obtain the driver's face image in the The eye feature data and mouth feature data within a time period; the eye feature data includes blinking frequency, eye closing time, sight deviation time and eye appearance features; the eye appearance features include white eye color features, eyelid color features and eye periorbital color features; the mouth feature data includes The number of yawns during a period of time;

[0154] Obtaining a dynamic eye fatigue value according to the blinking frequency, eye closing time and line of sight deviation time; obtaining a static eye fatigue value according to the eye appearance characteristics and standard driving eye appearance characteristic data; obtaining an eye fatigue value according to the dynamic eye fatigue value and the static eye fatigue value;

[0155] The calculation formula of the dynamic eye fatigue value is:

[0156] ;

[0157] in, Indicated as a basis The dynamic eye fatigue value at the current moment obtained from the face image within the time period; Expressed as The driver's blinking frequency during the time period; Expressed as The driver’s eye closing time during the time period; Expressed as The driver’s sight deviation time during the time period; , and They are respectively expressed as blink frequency coefficient, eye closure time coefficient and gaze deviation coefficient; , , and The default is , which can also be changed according to the actual application;

[0158] The calculation process of the static eye fatigue value is as follows:

[0159] Randomly select M video frame images with face images in the real-time front driving video, and extract eye appearance features in the M video frame images; compare the eye appearance features with the standard driving eye appearance features corresponding to the target driver to obtain the similarity of the eye appearance features; average the similarities of the eye appearance features corresponding to the M video frame images to obtain the average value of the eye appearance feature similarities; the calculation formula of the static eye fatigue value is:

[0160] ;

[0161] in, Indicated as a basis The static eye fatigue value at the current moment obtained from the face image within the time period; It is expressed as the average value of the similarity of the eye appearance features;

[0162] The calculation formula of the eye fatigue value is:

[0163] ;

[0164] in, Indicated as a basis The eye fatigue value at the current moment obtained from the face image within the time period; and Expressed as dynamic fatigue coefficient and static fatigue coefficient respectively; and The default value is 0.5, which can be changed according to the actual application; Expressed as an exponential with a natural constant as base.

[0165] According to the eye fatigue value and the mouth feature data, a first fatigue driving value of the driver is obtained, and according to the first fatigue driving value, whether the driver is driving fatigued is determined; the calculation formula of the first fatigue driving value is:

[0166] ;

[0167] in, It is represented by the first driving fatigue value of the driver at the current time obtained based on the real-time front driving video; Expressed as The number of times the driver yawns during the time period; and They are expressed as eye fatigue coefficient and mouth fatigue coefficient respectively; and The default value is 0.5, which can be changed according to the actual application.

[0168] When the first fatigue driving value is not greater than the set first fatigue threshold, the driver is judged to be non-fatigued driving; when the first fatigue driving value is greater than the set first fatigue threshold, the driver is judged to be fatigued driving, and fatigue is graded according to the first fatigue driving value; the fatigue grades include mild fatigue, moderate fatigue and severe fatigue; the first fatigue threshold is obtained based on experience.

[0169] Furthermore, when there is no face image in each frame of the real-time front driving video, obtaining The echo signal within the time period is used to construct an echo signal set; based on the echo signal set, the The driving breathing characteristic data of the driver in the time period; the driving breathing characteristic data includes the breathing frequency and the heart rate; the standard driving breathing characteristic data includes the standard driving breathing frequency and the standard driving heart rate; according to the driving breathing characteristic data and the standard driving breathing characteristic data in the driving behavior characteristics, the second fatigue driving value of the driver is obtained, and according to the second fatigue driving value, whether the driver is fatigue driving is determined; the calculation formula of the second fatigue driving value is:

[0170] ;

[0171] Wherein, represents the second fatigue driving value of the driver at the current time obtained according to the echo signal set; Expressed as The breathing rate of the driver during the time period; Expressed as The driver's heart rate during the time period; and They are expressed as standard driving breathing rate and standard driving heart rate respectively; and All are nonlinear functions; and They are expressed as respiratory rate coefficient and heart rate coefficient respectively;

[0172] When the second fatigue driving value is not greater than the set second fatigue threshold, the driver is judged to be non-fatigued driving; when the second fatigue driving value is greater than the set second fatigue threshold, the driver is judged to be fatigued driving, and fatigue classification is performed based on the second fatigue driving value; the fatigue classification includes mild fatigue, moderate fatigue and severe fatigue.

[0173] Furthermore, the step of obtaining abnormal hand movements includes: obtaining The hand driving video within the time period is recorded as the real-time hand driving video; based on the real-time hand driving video, the Whether the driver has any abnormal hand movements during the time period.

[0174] Further, when the driving behavior analysis unit analyzes the driving behavior and obtains that the driver is not driving while fatigued and there is no abnormal hand movement, the driving behavior recognition result is normal driving.

[0175] Furthermore, the vehicle running obstacle analysis unit includes:

[0176] Acquire the running point cloud data at the current time, and pre-process the running point cloud data;

[0177] Inputting the preprocessed running point cloud data into the vehicle DGCNN model to obtain local running point cloud features;

[0178] The vehicle DGCNN model is based on the DGCNN model, and a self-attention module is added after each edge convolution module of the backbone network to extract the first operating point cloud feature of the operating point cloud data; at the same time, GCN is used to extract the second operating point cloud feature of the operating point cloud data, and the first operating point cloud feature and the second operating point cloud feature are weighted to obtain the operating point cloud feature; the operating obstacles are identified according to the operating point cloud features; when the operating obstacles are identified, the size of each of the operating obstacles is obtained; when the size of the operating obstacle is greater than the set size threshold, the operating obstacle is marked as a vehicle operating obstacle; the feature data of each of the vehicle operating obstacles is obtained, and the danger level of the operating obstacle is calculated.

[0179] Furthermore, the calculation process of the running obstacle risk includes:

[0180] According to the driving behavior recognition result obtained by the driving behavior analysis unit, the driving behavior risk is calculated, and the calculation formula is:

[0181] ;

[0182] in, It is represented by the driving behavior risk level at the current time obtained based on the driving behavior recognition result; Indicates the label value corresponding to the fatigue level; in particular, when the driver is not driving fatigue-free, ; When the driver is driving fatigued, the corresponding label value is obtained according to the fatigue level; for example: when the fatigue level is mild fatigue, ; When the fatigue level is moderate fatigue, ; When the fatigue level is severe fatigue, ; Indicates the label value corresponding to the hand movement; when there is an abnormal hand movement ;otherwise, ; and They are respectively expressed as fatigue driving coefficient and hand motion coefficient;

[0183] According to the driving behavior danger degree and the characteristic data of the vehicle running obstacle, the danger degree of the running obstacle is obtained; the characteristic data of the vehicle running obstacle includes the size and position of each of the vehicle running obstacles; according to the position of the vehicle running obstacle, the distance between each of the vehicle running obstacles and the current position of the vehicle is obtained; the characteristic data of the vehicle running obstacles whose distance is less than the driving danger distance are screened out; the calculation formula of the driving danger distance is:

[0184] ;

[0185] in, It is indicated as driving danger distance; Represented as the set distance threshold; Expressed as distance coefficient;

[0186] The danger degree of the running obstacle is obtained according to the characteristic data of the vehicle running obstacle within the driving danger distance, and the calculation formula is:

[0187] ;

[0188] in, It is represented as the danger degree of running obstacles; W is represented as the number of running obstacles within the driving danger distance; in particular, when W=0, ; It is represented as the distance between the wth vehicle running obstacle and the current position of the vehicle; It is represented as the size of the current position of the w-th vehicle;

[0189] According to the running obstacle danger degree, the running obstacle danger level is obtained; according to the running obstacle danger level, the alarm device is controlled to issue a corresponding alarm prompt;

[0190] The communication module is used to receive the analysis results and upload them to the monitoring terminal module; at the same time, the analysis host controls the alarm device to issue a corresponding alarm prompt;

[0191] The monitoring terminal module is used to record, store, display and alert the backstage monitoring personnel of the acquired analysis results.

[0192] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A train-mounted auxiliary system based on personnel behavior recognition and safety monitoring, characterized in that: include: Vehicle status monitoring module, acquisition module, analysis host, alarm device, communication module and monitoring terminal module; The vehicle status monitoring module is used to monitor whether the vehicle is in a running state; The acquisition module is used to acquire vehicle operation data in real time when the vehicle is in operation; The analysis host is used to receive the vehicle operation data transmitted in real time by the acquisition module, and analyze the driving behavior of the driver and the vehicle operation obstacles to obtain the analysis results; The analysis host includes a driver identity authentication unit, a driving behavior analysis unit and a vehicle operation obstacle analysis unit; the analysis results include a driver identification result, a driving behavior identification result and a vehicle operation obstacle identification result; The vehicle running obstacle analysis unit is used to identify and obtain the vehicle running obstacles and calculate the running obstacle danger level; The specific process includes: Calculating the driving behavior risk according to the driving behavior recognition result obtained by the driving behavior analysis unit; Obtaining the danger degree of the running obstacle according to the driving behavior danger degree and the characteristic data of the vehicle running obstacle; the characteristic data of the vehicle running obstacle includes the size and position of each of the vehicle running obstacles; According to the running obstacle danger degree, the running obstacle danger level is obtained; according to the running obstacle danger level, the alarm device is controlled to issue a corresponding alarm prompt.

2. According to claim 1, a train-mounted auxiliary system based on personnel behavior recognition and safety monitoring is characterized in that: The acquisition module includes: a driver identity feature acquisition unit, an RFID collector, a first camera, a second camera, a millimeter wave radar, an ultra-long-range laser radar and a data transmission unit; The driver identity characteristic acquisition unit is used to acquire the driving characteristics of the current target driver of the vehicle; The RFID collector is used to collect the driver's RFID tag information; The first camera is used to collect the front driving video of the driver; The second camera is used to collect the driver's hand driving video; The millimeter wave radar is used to transmit millimeter wave signals into the cab and receive echo signals from the cab; The ultra-long-range laser radar is used to obtain the operation point cloud data within the vehicle operation area; The data transmission unit is used to transmit the vehicle operation data to the analysis host in real time; the vehicle operation data includes: the driver's RFID tag information, the driver's front driving video, the driver's hand driving video, the cab echo signal, the operation point cloud data and the driving characteristics of the target driver.

3. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 1 is characterized in that: The driving characteristics of the current target driver of the vehicle include: the identity characteristics and driving behavior characteristics of the target driver in a pre-collected driver database; the identity characteristics include the name, work number and facial image of the target driver; the driving behavior characteristics include standard driving eye appearance feature data and standard driving breathing feature data corresponding to the target driver.

4. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 1 is characterized in that: The driver identity verification unit is used to verify the identity of the driver, and the specific process includes: Comparing the driver's RFID tag information with the target driver's driving characteristics; the RFID tag information includes the driver's name, work number and facial image; When the RFID tag information and the identity characteristics in the driving characteristics of the target driver are inconsistent, the driver identity verification unit determines that the driver identification result is abnormal; When the RFID tag information is consistent with the identity characteristics in the driving characteristics of the target driver, obtain The frontal driving video within the time period is recorded as the real-time frontal driving video. Represents the current time, Expressed as the first time threshold; extracting the front driving video frame image in the real-time front driving video; extracting the facial features of each of the front driving video frame images, and comparing the facial features with the facial features in the identity features to obtain the facial feature similarity; when there are consecutive Z frames corresponding to the front driving video frame images whose facial feature similarity is less than the set facial similarity threshold, judging that the driver recognition result is abnormal; otherwise, obtaining the clothing features of the front driving video frame images, and obtaining the clothing feature similarity; when there are consecutive F frames corresponding to the front driving video frame images whose clothing feature similarity is less than the set clothing feature similarity threshold, judging that the driver recognition result is abnormal; otherwise, judging that the driver recognition result is normal.

5. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 1 is characterized in that: The driving behavior analysis unit is used to analyze the driving behavior of the driver and obtain the driving behavior recognition result when the driver recognition result is normal; The driving behavior recognition result includes normal driving and abnormal driving; The abnormal driving includes fatigue driving and abnormal hand movements; The step of obtaining the fatigue driving includes: Based on the real-time front driving video, obtain the driver's When there is a face image in each frame of the real-time front driving video, obtain the driver's face image in the The eye feature data and mouth feature data within a time period; the eye feature data includes blinking frequency, eye closing time, sight deviation time and eye appearance features; the eye appearance features include white eye color features, eyelid color features and eye periorbital color features; the mouth feature data includes The number of yawns during a period of time; Obtaining a dynamic eye fatigue value according to the blinking frequency, eye closing time and line of sight deviation time; obtaining a static eye fatigue value according to the eye appearance characteristics and standard driving eye appearance characteristic data; obtaining an eye fatigue value according to the dynamic eye fatigue value and the static eye fatigue value; A first fatigue driving value of the driver is acquired based on the eye fatigue value and the mouth feature data, and whether the driver is driving fatigued is determined based on the first fatigue driving value.

6. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 5 is characterized in that: When there is no face image in each frame of the real-time front driving video, obtain echo signals within a time period, constructing an echo signal set; According to the echo signal set, obtain Driving breathing characteristic data of drivers in different time periods; A second fatigue driving value of the driver is obtained according to the driving breathing characteristic data and the standard driving breathing characteristic data in the driving behavior characteristics, and whether the driver is driving fatigued is determined according to the second fatigue driving value.

7. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 5 is characterized in that: The step of obtaining abnormal hand movements includes: obtaining The hand driving video within the time period is recorded as the real-time hand driving video; based on the real-time hand driving video, the Whether the driver has any abnormal hand movements during the time period.

8. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 2 is characterized in that: The vehicle running obstacle analysis unit comprises: Acquire the running point cloud data at the current time, and pre-process the running point cloud data; Inputting the preprocessed running point cloud data into the vehicle DGCNN model to obtain local running point cloud features; The vehicle DGCNN model is based on the DGCNN model, and a self-attention module is added after each edge convolution module of the backbone network to extract the first operating point cloud feature of the operating point cloud data; at the same time, GCN is used to extract the second operating point cloud feature of the operating point cloud data, and the first operating point cloud feature and the second operating point cloud feature are weighted to obtain the operating point cloud feature; the operating obstacles are identified according to the operating point cloud features; when the operating obstacles are identified, the size of each of the operating obstacles is obtained; when the size of the operating obstacle is greater than the set size threshold, the operating obstacle is marked as a vehicle operating obstacle and the feature data of each of the vehicle operating obstacles is obtained, and the danger level of the operating obstacle is calculated.

9. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 1 is characterized in that: The communication module is used to receive the analysis result and upload it to the monitoring terminal module; at the same time, the analysis host controls the alarm device to issue a corresponding alarm prompt.

10. The train-mounted auxiliary system based on personnel behavior recognition and safety monitoring according to claim 1 is characterized in that: The monitoring terminal module is used to record, store, display and alert the backstage monitoring personnel of the acquired analysis results.

Images