A non-motor vehicle driver state monitoring and early warning method and system

Abstract

The application relates to the technical field of vehicle safe driving monitoring, and provides a non-motor vehicle driver state monitoring and early warning method and system.The method comprises the following steps: acquiring the running track information of a target driving vehicle, extracting N abnormal nodes according to the running route information, and obtaining N abnormal data sets; performing abnormal analysis on the N abnormal data sets to obtain N abnormal analysis result sets; obtaining M transfer constraint information; obtaining a real-time driver data set; matching the real-time driver data set to obtain real-time node matching data; and performing early warning analysis on the above information to obtain a real-time state monitoring and early warning result.The application solves the technical problem that the driver state cannot be monitored in real time in the prior art, traffic accidents are prone to occur, and the safety of the driver work is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle safe driving monitoring, in particular to a non-motor vehicle driver state monitoring and early warning method and system. BACKGROUND

[0002] As the number of people with cars is increasing today, road safety is a top priority. In order to ensure the safety of the traffic road, prevent traffic accidents caused by fatigue and other problems of the driver, the present application provides a non-motor vehicle driver state monitoring and early warning method and system.

[0003] In summary, the present application solves the technical problem that the state of the driver cannot be monitored in real time in the prior art, which leads to traffic accidents. SUMMARY

[0004] Therefore, it is necessary to provide a non-motor vehicle driver state monitoring and early warning method and system which can avoid safety accidents of the driver during driving and improve the safety of the driver's work, to solve the technical problem that the state of the driver cannot be monitored in real time in the prior art, which leads to traffic accidents, and to avoid safety accidents of the driver during driving and improve the safety of the driver's work.

[0005] In a first aspect, the embodiments of the present application provide a non-motor vehicle driver state monitoring and early warning method, which comprises: acquiring the running track information of a target driving vehicle, wherein the running track information comprises running route information and transfer node information; extracting N abnormal nodes according to the running route information, and matching the N abnormal nodes with an abnormal database to obtain N abnormal data sets; taking accident causes as indexes, performing abnormal analysis on the N abnormal data sets to obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label; extracting M transfer nodes based on the transfer node information, and obtaining the stay duration of the M transfer nodes as M transfer constraint information; performing real-time data collection on the driver of the target driving vehicle according to the sensor recognition module to obtain a real-time driver data set; matching the N abnormal nodes, the M transfer nodes and the real-time driver data set to obtain real-time node matching data; inputting the N abnormal nodes, the M transfer nodes, the real-time node matching data, the N abnormal analysis result sets and the M transfer constraint information into a state monitoring and early warning channel for early warning analysis to obtain a real-time state monitoring and early warning result.

[0006] In a second aspect, the embodiments of the present application provide a non-motor vehicle driver state monitoring and early warning system, comprising: a travel trajectory information acquisition module, configured to acquire travel trajectory information of a target driving vehicle, wherein the travel trajectory information comprises travel route information and transfer node information; an abnormal data set obtaining module, configured to extract N abnormal nodes according to the travel route information, match the N abnormal nodes with an abnormal database, and obtain N abnormal data sets; an abnormal analysis result obtaining module, configured to perform abnormal analysis on the N abnormal data sets with accidents as indexes, and obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label; a transfer node extraction module, configured to extract M transfer nodes based on the transfer node information, and obtain stay durations of the M transfer nodes as M transfer constraint information; a real-time driver data set obtaining module, configured to perform real-time data acquisition on a driver of the target driving vehicle according to the sensor recognition module, and obtain a real-time driver data set; a real-time node matching data obtaining module, configured to match the N abnormal nodes, the M transfer nodes, and the real-time driver data set, and obtain real-time node matching data; and a real-time state monitoring and early warning result obtaining module, configured to input the N abnormal nodes, the M transfer nodes, the real-time node matching data, the N abnormal analysis result sets, and the M transfer constraint information into a state monitoring and early warning channel for early warning analysis, and obtain a real-time state monitoring and early warning result.

[0007] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0008] First, the running track information of the target driving vehicle is acquired, wherein the running track information comprises running route information and transfer node information; secondly, N abnormal nodes are extracted according to the running route information, and the N abnormal nodes are matched with an abnormal database to obtain N abnormal data sets; next, the N abnormal data sets are subjected to abnormal analysis with accidents as indexes to obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label; then, M transfer nodes are extracted based on the transfer node information, and the stay duration of the M transfer nodes is obtained as M transfer constraint information; again, real-time data collection is performed on the driver of the target driving vehicle by the sensor recognition module to obtain a real-time driver data set; then, the N abnormal nodes, the M transfer nodes and the real-time driver data set are matched to obtain real-time node matching data; finally, the N abnormal nodes, the M transfer nodes, the real-time node matching data, the N abnormal analysis result sets and the M transfer constraint information are input into a state monitoring and early warning channel for early warning analysis to obtain a real-time state monitoring and early warning result. The present application solves the technical problem that the state of the driver cannot be monitored in real time in the prior art, leading to traffic accidents, and achieves the purpose of avoiding safety accidents of the driver during driving and improving the safety of the driver's work.

[0009] The above description is only a summary of the technical solutions of the present application. In order to enable one skilled in the art to better understand the technical means of the present application, the present application can be implemented in accordance with the contents of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A flowchart of a non-motor vehicle driver state monitoring and early warning method in an embodiment;

[0011] Figure 2 A flowchart of a state monitoring and early warning channel of a non-motor vehicle driver state monitoring and early warning method in an embodiment;

[0012] Figure 3 A structural block diagram of a non-motor vehicle driver state monitoring and early warning system in an embodiment.

[0013] REFERENCE SIGNS: Running track information acquisition module 11, abnormal data set obtaining module 12, abnormal analysis result obtaining module 13, transfer node extraction module 14, real-time driver data set obtaining module 15, real-time node matching data obtaining module 16, real-time state monitoring and early warning result obtaining module 17. DETAILED DESCRIPTION

[0014] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0015] After introducing the basic principles of the present application, the technical solutions in the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. In addition, it should be noted that, for convenience of description, only parts related to the present application are shown in the drawings, not all.

[0016] As shown in Figure 1 The present application provides a non-motor vehicle driver state monitoring and early warning method, applied to a state monitoring and early warning system, the state monitoring and early warning system being in communication connection with a sensor identification module, and the method comprising:

[0017] S100: obtaining the travel trajectory information of the target driving vehicle, wherein the travel trajectory information comprises travel route information and transfer node information;

[0018] Specifically, with the rapid development of economy, basically every family has their own vehicle. In daily driving, when the driver completes the dynamic driving task, the alertness, manipulation method, situational awareness and other abilities may not meet the driving ability requirements, and the driver may not be aware of it, which can easily cause traffic accidents. Therefore, the present application provides a non-motor vehicle driver state monitoring and early warning method, applied to a state monitoring and early warning system, the state monitoring and early warning system being in communication connection with a sensor identification module. By obtaining the driving environment of the non-motor vehicle, the warning level of the state monitoring result of the driver is determined, so as to avoid traffic accidents and improve the safety of the driver.

[0019] The target driving vehicle refers to a vehicle that is being driven and selected for study. The travel trajectory information refers to the information of the driving distance of the vehicle. The travel route information refers to the information of the route of the vehicle when driving to the destination. The transfer node information refers to the place that needs to be stopped during the process of driving to the destination.

[0020] Optionally, a vehicle is taken as a target driving vehicle, the starting point and the destination of the target driving vehicle are obtained, and driving route information of the target driving vehicle is obtained, wherein the driving route information includes road information and information of a place to be stopped when going to the destination. By obtaining the driving route of the target driving vehicle, a basis can be provided for subsequent analysis of a driver.

[0021] S200: Extract N abnormal nodes according to the travel route information, and match the N abnormal nodes with an abnormal database to obtain N abnormal data sets;

[0022] Specifically, the abnormal node refers to a road segment node where a traffic accident occurs, such as a car accident or a collision; the abnormal database refers to a database of data sets of abnormal nodes occurring on a road segment, including records of a place, information, and a cause; the matching refers to classification and division of the abnormal nodes and the abnormal database; and the abnormal data set refers to a set composed of data corresponding to the abnormal nodes in the travel route.

[0023] According to the travel route information, N abnormal nodes are extracted, and the N abnormal nodes are classified and divided with an abnormal database for matching to obtain N abnormal data sets.

[0024] S300: Abnormal analysis is performed on the N abnormal data sets with an accident cause as an index to obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label;

[0025] Specifically, the accident cause refers to a factor causing a traffic accident, such as a human factor or an environmental factor; the abnormal analysis refers to analysis of the accident cause in the abnormal data set; and the abnormal analysis result set refers to a set containing the N abnormal data sets and an abnormal mapping relationship.

[0026] Further, the step of the application further includes:

[0027] S310: A plurality of sample abnormal data sets and a plurality of sample abnormal analysis results with abnormal data labels are obtained;

[0028] S320: The abnormal data labels include an abnormal data label caused by an external cause and an abnormal data label caused by a driver state;

[0029] S330: An abnormal mapping relationship is constructed according to the plurality of sample abnormal data sets and the plurality of sample clustering results with abnormal data labels;

[0030] S340: The N abnormal analysis result sets are obtained according to the N abnormal data sets and the abnormal mapping relationship.

[0031] Specifically, the sample abnormal data set refers to a part of the abnormal data set selected from the N abnormal data sets for studying actual observation and investigation; the abnormal data label includes an abnormal data label caused by an external cause and an abnormal data label caused by a driver state, wherein the abnormal data label caused by the external cause includes other vehicles running a red light, other vehicles temporarily braking, etc., and the abnormal data label of the driver state includes the driver drinking, fatigue driving, inattentive driving, chatting with the same car, etc., and the label of the abnormal data is added to the plurality of sample abnormal data sets; clustering is not like classification with an optimization target and a learning process, but a statistical method that separates similar data and dissimilar data, and the clustering result shows a relatively stable distribution in the data, and this mode will not change due to the addition, deletion or modification of individual data points, and the data can be separated as much as possible; the mapping relationship refers to a corresponding relationship from the plurality of sample abnormal data sets to the plurality of sample clustering results with abnormal data labels, which is obtained according to the abnormal data labels.

[0032] Obtain a plurality of sample abnormal data sets and a plurality of sample abnormal analysis results with abnormal data labels, wherein the abnormal data labels include an abnormal data label caused by an external cause and an abnormal data label caused by a driver state; construct an abnormal mapping relationship according to the plurality of sample abnormal data sets and the plurality of sample clustering results with abnormal data labels; and obtain the N abnormal analysis result set according to the N abnormal data sets and the abnormal mapping relationship. By obtaining the N abnormal analysis result set, more accurate results can be obtained in combination with the driver state below.

[0033] S400: Extract M transfer nodes based on the transfer node information, and obtain the stay duration of the M transfer nodes as M transfer constraint information;

[0034] Specifically, the transfer node refers to a position node that needs to be stopped during travel; the stay duration of the M transfer nodes is obtained as M transfer constraint information through the transfer node information, for example, a time such as five minutes is specified as the time for the target driving vehicle to stop, that is, the transfer constraint information, and if the stay time of the target driving vehicle is greater than the transfer constraint information, the target driving vehicle may have an accident at the transfer node.

[0035] S500: Collect real-time data of the driver of the target driving vehicle according to the sensor recognition module to obtain a real-time driver data set;

[0036] Specifically, the sensor can perceive various things, such as temperature, humidity, angular velocity, image, and if the sensor data is read by a single-chip microcomputer, only a few sensors can be directly connected to the single-chip microcomputer, so in order to identify, a sensor module appears, which is a layer of circuit encapsulated on the basis of the sensor and can be directly connected to the single-chip microcomputer; the real-time data acquisition refers to acquiring data of the driving personnel, such as facial data, hand action data and foot data of the driving personnel, while the target driving vehicle is driving, and combining the acquired data to obtain a real-time driving personnel data set.

[0037] Further, the step of the application comprises:

[0038] S510: identifying the facial expression of the driving personnel of the target driving vehicle according to the facial recognition sensor device in the sensor identification module to obtain real-time facial recognition data, wherein the real-time facial recognition data has a time identifier;

[0039] S520: identifying the hand action of the driving personnel of the target driving vehicle according to the hand action recognition device in the sensor identification module to obtain real-time hand action recognition data;

[0040] S530: identifying the foot action of the driving personnel of the target driving vehicle according to the foot action recognition device in the sensor identification module to obtain real-time foot action recognition data;

[0041] S540: inputting the real-time facial recognition data, the real-time hand action recognition data and the real-time foot action recognition data into the action correction three-dimensional coordinate system to obtain an action correction result;

[0042] S550: obtaining a real-time driver data set according to the action correction result.

[0043] Specifically, the real-time facial data refers to the facial expression captured by a sensor such as a camera, which is mainly used to judge whether the driving personnel is tired or drunk; the real-time hand action recognition data refers to the position of the hand obtained by a camera or a pressure sensor; the real-time foot action recognition data refers to the position of the foot of the driving personnel, such as stepping on the car or stepping on the ground; the action correction result refers to the difference between the action of the driving personnel and the action of the driving personnel in a safe state after inputting the real-time facial recognition data, the real-time hand action recognition data and the real-time foot action recognition data into the action correction three-dimensional coordinate system; and the real-time driver data set refers to a collection of time points corresponding to the action data of the driver.

[0044] According to the face recognition sensor device in the sensor recognition module, the facial expression of the driver of the target driving vehicle is recognized, and real-time facial recognition data is obtained, wherein the real-time facial recognition data has a time identifier; according to the hand action recognition device in the sensor recognition module, the hand action of the driver of the target driving vehicle is recognized, and real-time hand action recognition data is obtained; according to the foot action recognition device in the sensor recognition module, the foot action of the driver of the target driving vehicle is recognized, and real-time foot action recognition data is obtained; the real-time facial recognition data, the real-time hand action recognition data and the real-time foot action recognition data are input into the action correction three-dimensional coordinate system, and an action correction result is obtained; and the real-time driver data set is obtained according to the action correction result.

[0045] Further, the step of the application further comprises:

[0046] S560: Obtain a plurality of historical real-time facial recognition data, a plurality of historical real-time hand action recognition data, a plurality of historical real-time foot action recognition data and a plurality of historical real-time states of the target driver, wherein the plurality of historical real-time states include a safe state of the target driver and an abnormal state of the target driver;

[0047] S570: The coordinate axes of the action correction three-dimensional coordinate system are constructed with the facial recognition data as the x-axis, the hand action recognition data as the y-axis, and the foot action recognition data as the z-axis;

[0048] S580: The plurality of historical real-time facial recognition data, the plurality of historical real-time hand action recognition data and the plurality of historical real-time foot action recognition data are input into the action correction three-dimensional coordinate system, and a plurality of historical coordinate points are obtained;

[0049] S590: The plurality of historical coordinate points are marked by using the plurality of historical real-time states, and a marking result is obtained;

[0050] S5100: The action correction three-dimensional coordinate system is generated based on the coordinate axes, the plurality of historical coordinate points and the marking result.

[0051] Specifically, the historical real-time state refers to the state of the target driver who has driven on the road in the past; and the marking result refers to the state display result of the target driver in the action correction three-dimensional coordinate system.

[0052] The plurality of historical real-time face recognition data, the plurality of historical real-time hand action recognition data, the plurality of historical real-time foot action recognition data and the plurality of historical real-time states of the target driver are obtained, wherein the plurality of historical real-time states include a safety state of the target driver and an abnormal state of the target driver; the coordinate axes of the action correction three-dimensional coordinate system are constructed with the face recognition data as the x-axis, the hand action recognition data as the y-axis and the foot action recognition data as the z-axis; the plurality of historical real-time face recognition data, the plurality of historical real-time hand action recognition data and the plurality of historical real-time foot action recognition data are input into the action correction three-dimensional coordinate system to obtain a plurality of historical coordinate points; the plurality of historical coordinate points are marked by using the plurality of historical real-time states to obtain a marking result; and the action correction three-dimensional coordinate system is generated based on the coordinate axes, the plurality of historical coordinate points and the marking result.

[0053] Further, the step of the application further comprises:

[0054] S5110: inputting the real-time face recognition data, the real-time hand action recognition data and the real-time foot action recognition data into the action correction three-dimensional coordinate system to obtain a real-time coordinate point;

[0055] S5120: obtaining k historical coordinate points closest to the real-time coordinate point, wherein k is an integer greater than or equal to 3;

[0056] S5130: obtaining k historical real-time states of the k historical coordinate points for mean value processing to obtain an action correction result.

[0057] Specifically, the real-time coordinate point refers to the real-time action of the target driver, which has a time identifier; the k historical coordinate points closest to the real-time coordinate point refer to the points closest to the real-time coordinate point in the action correction three-dimensional coordinate system, and k is an integer greater than 3; the mean value processing refers to average number processing of the point, that is, the average value of the point is obtained, for example, the target driver's hands are away from the steering wheel, but the driver's feet are on the ground, and at this time, the driver is in a safe state, such as parking, etc., so the driver is also in a safe state; the action correction result refers to the difference between the action of the driver after inputting the real-time face recognition data, the real-time hand action recognition data and the real-time foot action recognition data into the action correction three-dimensional coordinate system and the action of the driver in a safe state.

[0058] The real-time face recognition data, the real-time hand action recognition data and the real-time foot action recognition data are input into the action correction three-dimensional coordinate system to obtain a real-time coordinate point, and the k historical coordinate points closest to the real-time coordinate point are selected by the staff, wherein k is an integer greater than or equal to 3, which can be 3, 4, 5, etc.; the k historical real-time states of the k historical coordinate points are obtained for mean value processing to obtain an action correction result.

[0059] S600: Match based on N abnormal nodes, M relay nodes and real-time driver data set to obtain real-time node matching data;

[0060] Specifically, the real-time node matching data refers to matching the real-time state of the driver by time identification according to the need to stop and the place where the traffic accident occurs when the driver drives on the road.

[0061] Further, the steps of the application further include:

[0062] S610: Extract real-time position from the real-time driver data set;

[0063] S620: Match the real-time position with N abnormal nodes and M relay nodes to obtain the corresponding matching nodes;

[0064] S630: Take the matching nodes and the real-time driver data set as real-time node matching data.

[0065] Specifically, the real-time position refers to the actual time position of the driver in the road, and the real-time road position is extracted from the real-time driver data set. The real-time position is matched with N abnormal nodes and M relay nodes to obtain the corresponding matching nodes, which refers to the matching relationship between the position and the state of the real-time driver. The matching nodes and the real-time driver data set are taken as real-time node matching data, which refers to matching the real-time state of the driver by time identification according to the need to stop and the place where the traffic accident occurs when the driver drives on the road.

[0066] As shown in Figure 2 Further, the steps of the application further include:

[0067] S640: Take N abnormal nodes, historical real-time node matching data, N abnormal analysis result set and N historical real-time state monitoring and early warning result as the first training set;

[0068] S650: Supervised training of the framework based on BP neural network using the first training set to obtain the abnormal node early warning sub-channel of the state monitoring and early warning channel;

[0069] S660: Take M relay nodes, historical real-time node matching data, M relay constraint information and M historical relay real-time state monitoring and early warning result as the second training set;

[0070] S670: Supervised training of the framework based on BP neural network using the second training set to obtain the relay node early warning sub-channel of the state monitoring and early warning channel;

[0071] S680: generating a state monitoring and warning channel according to the abnormal node warning sub-channel and the transfer node warning sub-channel.

[0072] Specifically, the process of constructing the state monitoring and warning channel is as follows: first, based on a neural network algorithm, the network structure of the abnormal node warning sub-channel is constructed. The abnormal node warning sub-channel can form parameters such as connection weights and threshold values between simple units in a supervised training process. The trained abnormal node warning sub-channel can perform complex nonlinear logical operations according to input data, output abnormal state monitoring and warning results, and the input data of the abnormal node warning sub-channel is N abnormal nodes and historical real-time node matching data, and the output data is the abnormal state monitoring and warning result. A first training set is obtained to construct an abnormal node warning sub-channel, wherein the first training set includes N abnormal nodes, historical real-time node matching data, N abnormal analysis result sets, and N historical real-time state monitoring and warning results. The first training set is used to train and verify the abnormal node warning sub-channel to obtain the abnormal node warning sub-channel.

[0073] Similarly, the transfer node warning sub-channel can be obtained according to the above principle, and the state monitoring and warning channel is generated according to the abnormal node warning sub-channel and the transfer node warning sub-channel.

[0074] S700: inputting N abnormal nodes, M transfer nodes, real-time node matching data, N abnormal analysis result sets, and M transfer constraint information into the state monitoring and warning channel for warning analysis to obtain real-time state monitoring and warning results.

[0075] Specifically, N abnormal nodes, M transfer nodes, real-time node matching data, N abnormal analysis result sets, and M transfer constraint information are input into the state monitoring and warning channel to obtain real-time state monitoring and warning results, wherein the real-time state monitoring and warning results refer to the state monitoring of the target driver on the road, and the warning results are obtained according to the state of the target driver. The present application solves the technical problem that the state of the driver cannot be monitored in real time in the prior art, which leads to traffic accidents. The safety of the driver's work is improved.

[0076] As shown in Figure 3 The present application also provides a non-motor vehicle driver state monitoring and warning system, which is in communication connection with a sensor identification module, and the system comprises:

[0077] A travel trajectory information acquisition module 11 is configured to acquire travel trajectory information of a target driving vehicle, wherein the travel trajectory information includes travel route information and transfer node information.

[0078] An abnormal data set obtaining module 12 is configured to extract N abnormal nodes according to the travel route information, match the N abnormal nodes with an abnormal database, and obtain N abnormal data sets;

[0079] An abnormal analysis result obtaining module 13 is configured to perform abnormal analysis on the N abnormal data sets with accident causes as indexes, and obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label;

[0080] A transfer node extraction module 14 is configured to extract M transfer nodes based on the transfer node information, and obtain stay durations of the M transfer nodes as M transfer constraint information;

[0081] A real-time driver data set obtaining module 15 is configured to perform real-time data collection on a driver of a target driving vehicle according to the sensor identification module, and obtain a real-time driver data set;

[0082] A real-time node matching data obtaining module 16 is configured to match the N abnormal nodes, the M transfer nodes, and the real-time driver data set, and obtain real-time node matching data;

[0083] A real-time state monitoring and early warning result obtaining module 17 is configured to input the N abnormal nodes, the M transfer nodes, the real-time node matching data, the N abnormal analysis result sets, and the M transfer constraint information into a state monitoring and early warning channel for early warning analysis, and obtain real-time state monitoring and early warning results.

[0084] Further, the embodiments of the present application further include:

[0085] A sample abnormal analysis result obtaining module is configured to obtain a plurality of sample abnormal data sets and a plurality of sample abnormal analysis results with abnormal data labels;

[0086] An abnormal data label module is configured to include abnormal data labels caused by external reasons and abnormal data labels caused by driver states;

[0087] An abnormal mapping relationship construction module is configured to construct an abnormal mapping relationship according to the plurality of sample abnormal data sets and the plurality of sample clustering results with abnormal data labels;

[0088] An abnormality analysis result set obtaining module is configured to obtain the N abnormality analysis result sets according to the N abnormality data sets and the abnormality mapping relationship.

[0089] Further, the embodiments of the present application further include:

[0090] A facial expression recognition module is configured to recognize facial expressions of a driver of a target driving vehicle according to a facial recognition sensor device in the sensor recognition module, and obtain real-time facial recognition data, wherein the real-time facial recognition data has a time identifier.

[0091] A real-time hand action recognition data obtaining module is configured to recognize hand actions of the driver of the target driving vehicle according to a hand action recognition device in the sensor recognition module, and obtain real-time hand action recognition data.

[0092] A real-time foot action recognition data obtaining module is configured to recognize foot actions of the driver of the target driving vehicle according to a foot action recognition device in the sensor recognition module, and obtain real-time foot action recognition data.

[0093] An action correction result obtaining module is configured to input the real-time facial recognition data, the real-time hand action recognition data and the real-time foot action recognition data into an action correction three-dimensional coordinate system, and obtain an action correction result.

[0094] A real-time driver data set obtaining module is configured to obtain a real-time driver data set according to the action correction result.

[0095] Further, the embodiments of the present application further include:

[0096] A target driver historical real-time state obtaining module is configured to obtain a plurality of historical real-time facial recognition data, a plurality of historical real-time hand action recognition data, a plurality of historical real-time foot action recognition data and a plurality of historical real-time states of a target driver, wherein the plurality of historical real-time states include a safety state of the target driver and an abnormality state of the target driver.

[0097] A coordinate axis constructing module of a three-dimensional coordinate system is configured to construct coordinate axes of the action correction three-dimensional coordinate system, with the facial recognition data as an x-axis, the hand action recognition data as a y-axis, and the foot action recognition data as a z-axis.

[0098] a historical coordinate point obtaining module, configured to input a plurality of historical real-time face recognition data, a plurality of historical real-time hand action recognition data, and a plurality of historical real-time foot action recognition data into the action correction three-dimensional coordinate system to obtain a plurality of historical coordinate points;

[0099] a marking result obtaining module, configured to mark the plurality of historical coordinate points by using a plurality of historical real-time states to obtain a marking result;

[0100] an action correction three-dimensional coordinate system generating module, configured to generate the action correction three-dimensional coordinate system based on the coordinate axis, the plurality of historical coordinate points, and the marking result.

[0101] Further, the embodiment of the application further includes:

[0102] a real-time coordinate point obtaining module, configured to input real-time face recognition data, real-time hand action recognition data, and real-time foot action recognition data into the action correction three-dimensional coordinate system to obtain a real-time coordinate point;

[0103] a historical coordinate point obtaining module, configured to obtain k historical coordinate points closest to the real-time coordinate point, where k is an integer greater than or equal to 3;

[0104] an action correction result obtaining module, configured to obtain k historical real-time states of the k historical coordinate points and perform mean processing to obtain an action correction result.

[0105] Further, the embodiment of the application further includes:

[0106] a real-time position extracting module, configured to extract a real-time position from the real-time driver data set;

[0107] a corresponding matching node obtaining module, configured to perform position matching on the real-time position, N abnormal nodes, and M transfer nodes to obtain corresponding matching nodes;

[0108] a real-time node matching data module, configured to take the matching nodes and the real-time driver data set as real-time node matching data.

[0109] Further, the embodiment of the application further includes:

[0110] a first training set obtaining module, configured to take the N abnormal nodes, historical real-time node matching data, N abnormal analysis result sets, and N historical real-time state monitoring and early warning results as a first training set;

[0111] The abnormal node early warning sub-channel obtaining module is configured to perform supervised training on the framework based on the BP neural network by using the first training set, and obtain an abnormal node early warning sub-channel of the state monitoring early warning channel.

[0112] The second training set obtaining module is configured to take the M transfer nodes, the historical real-time node matching data, the M transfer constraint information and the M historical transfer real-time state monitoring early warning results as a second training set.

[0113] The transfer node early warning sub-channel obtaining module is configured to perform supervised training on the framework based on the BP neural network by using the second training set, and obtain a transfer node early warning sub-channel of the state monitoring early warning channel.

[0114] The state monitoring early warning channel obtaining module is configured to generate a state monitoring early warning channel according to the abnormal node early warning sub-channel and the transfer node early warning sub-channel.

[0115] For specific embodiments of the non-motor vehicle driver state monitoring early warning system, reference can be made to the embodiments of the non-motor vehicle driver state monitoring early warning method described above, which will not be repeated here. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0116] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0117] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A non-motor vehicle driver state monitoring and warning method, characterized in that, The method is applied to a state monitoring and early warning system in communication connection with a sensor identification module, and comprises: obtaining travel trajectory information of a target driving vehicle, wherein the travel trajectory information comprises travel route information and transfer node information; extracting N abnormal nodes according to the travel route information, and matching the N abnormal nodes with an abnormal database to obtain N abnormal data sets; performing abnormal analysis on the N abnormal data sets with accidents as indexes to obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label; extracting M transfer nodes based on the transfer node information, and obtaining the stay duration of the M transfer nodes as M transfer constraint information; performing real-time data collection on a driver of the target driving vehicle according to the sensor identification module to obtain a real-time driver data set; matching the N abnormal nodes, the M transfer nodes and the real-time driver data set to obtain real-time node matching data; inputting the N abnormal nodes, the M transfer nodes, the real-time node matching data, the N abnormal analysis result sets and the M transfer constraint information into a state monitoring and early warning channel for early warning analysis to obtain a real-time state monitoring and early warning result.

2. The method of claim 1, wherein, The abnormal analysis on the N abnormal data sets with accidents as indexes to obtain N abnormal analysis result sets comprises: obtaining a plurality of sample abnormal data sets and a plurality of sample abnormal analysis results with abnormal data labels; wherein the abnormal data labels comprise abnormal data labels caused by external reasons and abnormal data labels caused by the state of the driver; constructing an abnormal mapping relationship according to the plurality of sample abnormal data sets and the plurality of sample clustering results with abnormal data labels; obtaining the N abnormal analysis result sets according to the N abnormal data sets and the abnormal mapping relationship.

3. The method of claim 1, wherein, The real-time data collection on the driver of the target driving vehicle according to the sensor identification module to obtain a real-time driver data set comprises: recognizing the facial expression of the driver of the target driving vehicle according to a facial recognition sensor device in the sensor identification module to obtain real-time facial recognition data, wherein the real-time facial recognition data has a time identifier; recognizing the hand action of the driver of the target driving vehicle according to a hand action recognition device in the sensor identification module to obtain real-time hand action recognition data; recognizing the foot action of the driver of the target driving vehicle according to a foot action recognition device in the sensor identification module to obtain real-time foot action recognition data; inputting the real-time facial recognition data, the real-time hand action recognition data and the real-time foot action recognition data into a motion correction three-dimensional coordinate system to obtain a motion correction result; obtaining the real-time driver data set according to the motion correction result.

4. The method of claim 3, wherein, comprises: obtaining a plurality of historical real-time facial recognition data, a plurality of historical real-time hand action recognition data, a plurality of historical real-time foot action recognition data and a plurality of historical real-time states of a target driver, wherein the plurality of historical real-time states comprise the safe state of the target driver and the abnormal state of the target driver; The coordinate axes of the motion correction three-dimensional coordinate system are constructed by taking facial recognition data as the x-axis, taking hand motion recognition data as the y-axis, and taking foot motion recognition data as the z-axis; A plurality of historical coordinate points are obtained by inputting a plurality of historical real-time facial recognition data, a plurality of historical real-time hand motion recognition data, and a plurality of historical real-time foot motion recognition data into the motion correction three-dimensional coordinate system; The plurality of historical coordinate points are marked by using a plurality of historical real-time states to obtain a marking result; The motion correction three-dimensional coordinate system is generated based on the coordinate axes, the plurality of historical coordinate points, and the marking result.

5. The method of claim 4, wherein, The method comprises: Real-time facial recognition data, real-time hand motion recognition data, and real-time foot motion recognition data are input into the motion correction three-dimensional coordinate system to obtain real-time coordinate points; The k nearest historical coordinate points to the real-time coordinate points are obtained, wherein k is an integer greater than or equal to 3; The k historical real-time states of the k historical coordinate points are processed by mean value to obtain a motion correction result.

6. The method of claim 1, wherein, The method comprises: A real-time location is extracted from the real-time driver data set; The real-time location is matched with N abnormal nodes and M transfer nodes to obtain corresponding matching nodes; The matching nodes and the real-time driver data set are used as real-time node matching data.

7. The method of claim 6, wherein, The method comprises: N abnormal nodes, historical real-time node matching data, N abnormal analysis result sets, and N historical real-time state monitoring and early warning results are used as a first training set; The framework based on the BP neural network is supervised trained by using the first training set to obtain an abnormal node early warning sub-channel of the state monitoring and early warning channel; M transfer nodes, historical real-time node matching data, M transfer constraint information, and M historical transfer real-time state monitoring and early warning results are used as a second training set; The framework based on the BP neural network is supervised trained by using the second training set to obtain a transfer node early warning sub-channel of the state monitoring and early warning channel; The state monitoring and early warning channel is generated according to the abnormal node early warning sub-channel and the transfer node early warning sub-channel.

8. A non-motorized vehicle driver state monitoring and warning system, characterized by, The state monitoring and early warning system is in communication connection with a sensor recognition module, and the system comprises: A travel trajectory information acquisition module, which is used to acquire travel trajectory information of a target driving vehicle, wherein the travel trajectory information comprises travel route information and transfer node information; An abnormal data set acquisition module, which is used to extract N abnormal nodes according to the travel route information, match the N abnormal nodes with an abnormal database, and obtain N abnormal data sets; An abnormal analysis result acquisition module, which is used to perform abnormal analysis on the N abnormal data sets by taking an accident cause as an index to obtain N abnormal analysis result sets, wherein each abnormal analysis result set has an abnormal data label; A transfer node extraction module, which is used to extract M transfer nodes based on the transfer node information and obtain the stay duration of the M transfer nodes as M transfer constraint information; The real-time driver dataset obtaining module is configured to perform real-time data collection on a driver of a target driving vehicle according to the sensor identification module, and obtain a real-time driver dataset; The real-time node matching data obtaining module is configured to match the N abnormal nodes, the M transfer nodes and the real-time driver dataset, and obtain real-time node matching data. The real-time state monitoring and early warning result obtaining module is configured to input the N abnormal nodes, the M transfer nodes, the real-time node matching data, the N abnormal analysis result sets and the M transfer constraint information into a state monitoring and early warning channel for early warning analysis, and obtain a real-time state monitoring and early warning result.

Images