Driving state recognition method, device, equipment and program product

Abstract

The invention relates to the field of intelligent driving, in particular to a driving state recognition method, device and equipment and a program product. The method comprises the steps that a first event of a vehicle in the driving process is collected, wherein the first event comprises a first event attribute and a first event entity serving as a node in a map; according to the first event attribute, a first event entity of the first event is added to a first entity sequence in a graph, a first relation label between the first event and the first entity sequence is determined, and the graph comprises the first entity sequence determined by an event set before the first event; the first entity sequence represents the relationship between two adjacent event entities in the first entity sequence through a second relationship label; and updating a second relation label of the first entity sequence according to the first relation label, and identifying the driving state according to the first relation label and the second relation label. And updating the second relation label in the first entity sequence according to the first relation label, thereby being beneficial to improving the accuracy of the identified driving state.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and in particular to a driving state recognition method, device, equipment and program product. Background Art

[0002] Driving state recognition refers to the use of various sensors and algorithms to monitor and analyze the driver's behavior, physiological state, and environmental factors related to driving to determine whether the driver is in a safe driving state. This includes identifying whether the driver is fatigued, distracted, drunk, or in other states that may affect driving ability. Driving state recognition is an important part of the intelligent driving system, which helps improve road safety and reduce traffic accidents.

[0003] At present, intelligent driving systems usually identify driving status based on a single event or events that occur over a period of time. When determining driving status through a single event, the driver's behavior and physiological state may have certain randomness and individual differences. For example, emergency deceleration does not necessarily mean that the driver is in a dangerous driving state, but may be the driver's behavioral habits. Events that occur over a period of time may also be the driver's personalized habits, which is not conducive to improving the accuracy of driving status recognition. SUMMARY OF THE INVENTION Technical issues

[0004] In view of this, the embodiments of the present application provide a driving state recognition method, device, equipment and program product to solve the problem in the prior art that it is not conducive to improving the recognition accuracy of the driving state when recognizing the driving state. Technical Solutions

[0005] A first aspect of an embodiment of the present application provides a driving state recognition method, the method comprising:

[0006] Collecting a first event of the vehicle during driving, the first event including a first event attribute and a first event entity used as a node in the graph;

[0007] Adding a first event entity of the first event to a first entity sequence in the graph according to the first event attribute, determining a first relationship label between the first event and the first entity sequence, the graph including the first entity sequence determined by the event set before the first event, the first entity sequence characterizing the relationship between two adjacent event entities in the first entity sequence through a second relationship label;

[0008] The second relationship tag of the first entity sequence is updated according to the first relationship tag, and the driving state is identified according to the first relationship tag and the second relationship tag.

[0009] In combination with the first aspect, in a first possible implementation manner of the first aspect, determining a first relationship label between a first event and a first entity sequence in a graph includes:

[0010] determining a second event in the first entity sequence that is associated with the first event;

[0011] Determine a conditional event according to the first event and the event set in the first entity sequence, and obtain, based on the conditional event, the probability that the relationship between the first event and the second event is a different relationship label;

[0012] A first relationship label between the first event and the second event is determined according to the probability.

[0013] In combination with the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, obtaining, based on the conditional event, a probability that the relationship between the first event and the second event is a different relationship label includes:

[0014] According to historical statistical data, the probability of obtaining a relationship between the first event and the second event as different relationship labels based on the conditional event is determined;

[0015] Alternatively, the first event and the first entity sequence are input into a pre-trained probability prediction model to determine the probability that the relationship between the first event and the second event is a different relationship label.

[0016] In combination with the first aspect, in a third possible implementation manner of the first aspect, determining, based on historical statistical data, a probability that the relationship between the first event and the second event is a different relationship label based on the conditional event includes:

[0017] When the driver of the vehicle has been in the fleet system for longer than the predetermined time, the probability of obtaining different relationship labels for the relationship between the first event and the second event based on the conditional event is determined according to the knowledge graph determined by the historical statistical data of the fleet;

[0018] When the length of time that the driver of the vehicle has joined the fleet system is less than or equal to a predetermined length of time, the probability that the relationship between the first event and the second event is a different relationship label is determined based on the conditional event according to the knowledge graph determined based on the driver's historical statistical data.

[0019] In combination with the first possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, before adding the first event entity of the first event to the first entity sequence in the graph according to the first event attribute, the method further includes:

[0020] According to the event end time in the first event attribute, the connection order of each entity in the first entity sequence is determined.

[0021] In combination with the first aspect, in a fifth possible implementation manner of the first aspect, collecting a first event during driving of the vehicle includes at least one of the following manners:

[0022] Acquiring the action event of the driver of the vehicle through the driver monitoring system;

[0023] Acquire environmental events during vehicle driving through advanced driver assistance systems;

[0024] The motion event of the vehicle is acquired through the motion sensor of the vehicle.

[0025] In combination with the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, the action event includes at least one of an eye closing event, a yawning event, a left and right looking event, a head-down distraction event, a downward glance distraction event, a phone call event, a smoking event, an eating event, a drinking event, and a driver large-scale movement event;

[0026] The environmental event includes at least one of a front collision event, a vehicle-to-vehicle distance event, a pedestrian collision event, a lane departure event, and a lane keeping abnormality event;

[0027] The motion event includes at least one of an acceleration event, a smooth deceleration event, a sharp deceleration event, a smooth deceleration event, a sharp turn event, a smooth turn event, a collision event, and a rollover event.

[0028] A second aspect of an embodiment of the present application provides a driving state recognition device, the device comprising:

[0029] An event collection unit, used for collecting a first event of the vehicle during driving, the first event including a first event attribute and a first event entity used as a node in the graph;

[0030] a relationship label determination unit, configured to add a first event entity of a first event to a first entity sequence in a graph according to a first event attribute, and determine a first relationship label between the first event and the first entity sequence, wherein the graph includes a first entity sequence determined by an event set before the first event, and the first entity sequence represents a relationship between two adjacent event entities in the first entity sequence through a second relationship label;

[0031] The driving state determining unit is used to update the second relationship tag of the first entity sequence according to the first relationship tag, and identify the driving state according to the first relationship tag and the second relationship tag.

[0032] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements any method of the first aspect.

[0033] A fourth aspect of the embodiments of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method in the first aspect or its various implementations.

[0034] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any method of the first aspect are implemented.

[0035] The sixth aspect of the embodiment of the present application provides a chip for implementing the methods in each implementation of the first aspect. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method in the first aspect or its implementation. Beneficial Effects

[0036] Compared with the prior art, the embodiment of the present application has the following beneficial effects: when the embodiment of the present application collects the first event during driving, according to the first event attribute of the first event, the first event entity of the first event is added to the first entity sequence in the predetermined graph, the first relationship label between the first event and the first entity sequence is determined, the second relationship label of the first entity sequence is updated according to the first relationship label, and the driving state is identified according to the first relationship label and the second relationship label. Since the method can update the second relationship label of the first entity sequence according to the first relationship label between the latest generated first event and the first entity sequence generated earlier, the accuracy of the second relationship label is higher, so that a more accurate driving state can be identified according to the first relationship label and the updated second relationship label, and the recognition efficiency of the driving state can be effectively improved by searching the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 It is a schematic diagram of an implementation scenario of a driving state recognition method provided in an embodiment of the present application;

[0039] Figure 2 It is a schematic diagram of the implementation flow of a driving state recognition method provided in an embodiment of the present application;

[0040] Figure 3This is a schematic diagram of an implementation process of determining a relationship tag provided in an embodiment of the present application;

[0041] Figure 4 is a schematic diagram of determining an entity sequence of a first relationship label based on a first entity sequence and a first event provided by an embodiment of the present application;

[0042] Figure 5 is a schematic diagram of updating a second relationship tag according to a first relationship tag provided by an embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of an entity sequence for determining a first relationship label based on a first entity sequence and a first event provided by an embodiment of the present application;

[0044] Figure 7 is a schematic diagram of updating a second relationship tag according to a first relationship tag provided by an embodiment of the present application;

[0045] Figure 8 is a schematic diagram of a driving state recognition device provided in an embodiment of the present application;

[0046] Fig. 9 It is a schematic diagram of an electronic device provided in an embodiment of the present application. Embodiments of the present invention

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0048] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0049] Driving state recognition technology deploys multiple sensors and uses algorithms to monitor drivers' behavior patterns, physiological reactions, and the driving environment they are in in real time, thereby assessing whether the driver is in a suitable condition for safe driving. This technology can identify whether the driver is showing signs of fatigue, inattention, driving under the influence of alcohol, or other conditions that may impair their driving skills. As a key component of the intelligent driving system, driving state recognition technology plays a vital role in enhancing road safety and reducing the incidence of accidents.

[0050] In current intelligent driving systems, driving status is often identified by analyzing a single event or several events over a period of time. However, judging driving status based on a single event alone may be affected by the randomness and individual differences in driver behavior and physiological reactions. For example, an emergency deceleration operation does not always mean that the driver is in a dangerous state, which may be part of his personal driving habits. Similarly, several events over a period of time may also reflect the driver's personal driving habits, which may pose a challenge to improving the accuracy of driving status identification.

[0051] Figure 1 A schematic diagram of an implementation scenario of a driving state recognition method provided in an embodiment of the present application. Figure 1 As shown, the implementation scenario includes a vehicle 1 and a server 2, wherein the vehicle 1 includes a driver monitoring system (DMS in English, Driver Monitoring System in English) 10, an advanced driver assistance system 11 (ADAS in English, Advanced Driver Assistance System in English), a motion sensor 12 and a driving state recognition system 13. The driver monitoring system 10 can collect relevant information of the driver of the vehicle through the DMS camera, and determine the action events of the driver of the vehicle, including at least one of eye closing events, yawning events, left and right observation events, head-down distraction events, down-looking distraction events, phone calls, smoking events, eating events, drinking events and driver large-scale movement events. The advanced driver assistance system 11 can collect environmental events during the vehicle driving process through the ADAS camera, including at least one of front collision events, vehicle-to-vehicle close distance events, pedestrian collision events, lane departure events and lane keeping abnormal events. The motion sensor 12 can be used to collect the motion information of the vehicle and determine the motion events of the vehicle, including at least one of acceleration events, smooth deceleration events, sharp deceleration events, smooth deceleration events, sharp turn events, smooth turn events, collision events, and rollover events. The vehicle or server can generate a graph including an entity sequence based on the collected events and the attributes of the events, and characterize the relationship between two adjacent events in the entity sequence through relationship tags.

[0052] For any newly collected first event, the driving state recognition system 13 or service 2 can construct and update the knowledge graph. The first relationship tag between the first event and the first entity sequence can be determined based on the knowledge graph, and the second relationship tag of the first entity sequence previously obtained can be updated based on the first relationship tag, and the driving state can be identified based on the first relationship tag and the second relationship tag. By searching the entity sequence, the computational burden of the vehicle terminal can be effectively reduced.

[0053] Figure 2A schematic diagram of the implementation flow of a driving state recognition method provided in an embodiment of the present application is described in detail as follows:

[0054] In S201 , a first event during the driving process of the vehicle is collected.

[0055] Among them, the first event includes a first event entity and a first event attribute, wherein the first event entity is used as a node in the graph. For any event, an event entity and an event attribute may be included. When an event entity is used as a node in a graph, the name of the event entity may be used as the name of the node. For example, the name of the event entity may be determined based on the event name. The event entity name may include at least one of a closed eyes entity, a yawning entity, a left and right observation entity, a distracted head-down entity, a distracted glance down entity, a phone call entity, a smoking entity, an eating entity, a drinking entity, a front collision entity, a vehicle distance too close entity, a pedestrian collision entity, a lane departure entity, a lane keeping abnormality entity, a sudden acceleration entity, a sudden deceleration entity, a sharp turn entity, a collision entity, and a rollover entity.

[0056] When collecting the first event, image data can be collected by vehicle 1 including a DMS camera and an ADAS camera, or motion data of the vehicle can be collected by a motion sensor. Based on the collected image data or motion data, an event in the driving process is determined, and an event attribute of the event is determined.

[0057] The DMS camera can collect driver image data of the driver of the vehicle during driving, and determine the driver's action events based on the collected driver image data. The determined action events can include at least one of the driver's eye closing event, yawning event, left and right looking event, head-down distraction event, downward glance distraction event, phone call event, smoking event, eating event, drinking event, and driver's large movement event.

[0058] The image data of the vehicle's surrounding environment collected by the ADAS camera is used to determine environmental events during the vehicle's driving process. The determined environmental events may include at least one of a front collision event, a vehicle-to-vehicle distance event, a pedestrian collision event, a lane departure event, and a lane keeping abnormality event.

[0059] The motion data of the vehicle is collected by a motion sensor, including a motion state detection device such as an acceleration sensor or a gyroscope, and the motion event of the vehicle is determined according to the motion data of the vehicle. The motion event of the vehicle may include at least one of an acceleration event, a smooth deceleration event, a sharp deceleration event, a smooth deceleration event, a sharp turn event, a smooth turn event, a collision event, and a rollover event.

[0060] The attributes of any event may include one or more of the following information: event start time, event end time, speed at the start of the event, speed at the end of the event, driving time at the start of the event, and vehicle position at the start of the event.

[0061] When determining the event attributes, the duration of the detected state can be combined to determine whether the event is a valid event. For example, when detecting the eye closing event, its business process may include:

[0062] 1. When one of the eyes is detected to be closed, the timer starts.

[0063] 2. When the timing starts, if it is detected that both eyes are in an open state and the duration is greater than the first duration, for example, greater than 2 frames (that is, it can be detected that two consecutive frames are in a closed-eye state, and the 2-frame duration is related to the frame rate. After the frame rate is determined, the frame interval duration can be determined according to the frame rate, and the number of frame intervals can be determined according to the number of frames. The actual duration corresponding to a specific frame duration can be determined according to the number of frame intervals, and the first duration can be set as needed), the timing of this event ends.

[0064] 3. If the duration of eye closure is less than the second duration, such as less than 200ms (adjustable as needed), the event of this timing is invalid and discarded.

[0065] 4. When both eyes are in the open state and the duration is less than the first duration, such as less than 2 frames (which can be set as needed), and the driver is detected to be in the closed eye state again, it is considered that the closed eye event has not ended.

[0066] 5. When the last eye closing event ends, if the left eye or right eye is detected as closed again, the timing starts. If the left eye and the right eye are detected as open and the duration is greater than the first duration, such as greater than 2 frames (settable) or more, the timing ends and the time when the timing ends is determined as the timing end time.

[0067] 6. When it is detected that both eyes are in the closed state, the timing starts. When it is detected that both eyes are in the open state (both eyes of the driver are open) and the duration is greater than the first duration, such as greater than 2 frames (which can be set as needed) or more, the timing ends and the length of time the eyes are closed is calculated.

[0068] In S202, a first event entity of a first event is added to a first entity sequence in a graph according to a first event attribute, and a first relationship label between the first event and the first entity sequence is determined.

[0069] The graph includes a first entity sequence determined by an event set before a first event, and the first entity sequence represents the relationship between two adjacent event entities in the first entity sequence through a second relationship label.

[0070] The first entity sequence is determined based on the events collected during the vehicle driving process, and "first" is used to distinguish other entity sequences. In general, it is possible to determine whether the first entity sequence in the graph is in the same entity sequence based on the time interval between the events. For example, the end time of the first event is the first moment, the end time of the second event is the second moment, and the interval between the first moment and the second moment is greater than the predetermined third duration, such as greater than 100s, then it can be determined that the first event entity and the second event entity belong to different entity sequences. For the subsequent event entities, a new entity sequence can be created to identify the driving status.

[0071] In a possible implementation, the event entities in the first entity sequence may also be dynamically updated according to a preset number of event entities. For example, when a new first event is detected, the newly detected first event entity is added to the first entity sequence, and the event entity in the first entity sequence that is farthest from the end time of the first event is deleted, so that the set number of event entities is dynamically maintained in the first entity sequence.

[0072] Among them, when more than two events are included in the entity sequence (including the first entity sequence), the connection relationship between the entity sequences can be determined according to the end time of the events. That is, according to the end time of the event, one or two event associations closest to the end time can be found. For example, the event set in the event sequence includes the first event, the second event and the third event, and the corresponding event end times are t1, t2 and t3 respectively, and the corresponding event entities are the first event entity, the second event entity and the third event entity respectively, and t1 is earlier than t3, and t3 is earlier than t2, then the connection order of the event entities is the first event entity, the third event entity and the second event entity.

[0073] In the embodiment of the present application, the relationship between the event entities in the graph is characterized by relationship tags. The relationship tags may include relationship tags such as vehicle hijacking, driver severe fatigue, driver severe fatigue leading to abnormal behavior, driver severe distraction, driver mild fatigue, driver mild distraction, driver aggressive driving, driver conservative driving, driver stable driving, driver following too close, driver personalized habits, driver abnormal operation, driver normal behavior, etc.

[0074] In the embodiment of the present application, when adding the first event to the first entity sequence in the graph, if the vehicle is cold-started and the first entity sequence is empty, the first event can be directly used as the first node of the first entity sequence in the graph, and at this time, it is not necessary to determine the relationship label between the first event and the first entity sequence. Or if the interval between the end time of the first event and the previous event is too long, the first entity sequence and the first event entity belong to different entity sequences, and it is not necessary to determine the relationship label between the first event and the first entity sequence.

[0075] If the first entity sequence is not empty, and the duration between the end time of the first event and the end time of the second event of the first entity sequence (i.e., the last event in the first entity sequence) is less than the third duration, then an association between the first event entity and the second event entity can be established, and a first relationship label between the first event and the first entity sequence can be determined, which can be as follows: Figure 3 As shown, including:

[0076] In S301, a second event associated with a first event in a first entity sequence is determined.

[0077] In general, the second event in the first entity sequence that is related to the first event is the event corresponding to the event entity at the end of the first entity sequence. The end time of the second event is usually before the end time of the first event.

[0078] In S302, a conditional event is determined according to the first event and the event set in the first entity sequence, and based on the conditional event, a probability that the relationship between the first event and the second event is a different relationship label is obtained.

[0079] When determining the relationship between the first event and the second event, it is necessary to determine the number of event sets in the first entity sequence.

[0080] When the event entity in the first entity sequence is 1, the second event and the first event can be directly used as conditional events. Based on the second event, the probability that the relationship between the first event and the second event is a different relationship label is obtained, and the probability of all relationship labels on the conditional event can be calculated.

[0081] When the number of event entities in the first entity sequence is greater than 1, an event set of more than two events can be determined according to the first entity sequence, a conditional event can be determined according to the first event and the event set, and the probability of generating different relationship tags can be determined according to the conditional event.

[0082] Among them, when determining the first relationship label between the first event and the second event based on the conditional event, the probability of the current conditional event relative to different relationship labels can be found based on the knowledge graph determined by the driver's historical statistical data, or the first event and the first entity sequence can be input into a pre-trained probability prediction model to determine the probability that the relationship between the first event and the second event is different relationship labels, and the relationship label with the maximum probability is selected to determine the association relationship between the first event and the second event.

[0083] When determining the probabilities of different relationship labels based on historical statistical data, if the driver has joined the fleet system for less than the predetermined time, such as less than 1 year, the amount of historical statistical data of the driver is relatively small. In order to improve the accuracy of the historical statistical data, the probabilities of different relationship labels based on conditional events can be determined based on the knowledge graph determined by the fleet's historical statistical data.

[0084] If the driver has been in the fleet for a period of time greater than or equal to the predetermined period of time, the knowledge graph determined by the driver's historical statistical data can be directly used to determine the probabilities of different related tags based on the event conditions.

[0085] In S303, a first relationship label between the first event and the second event is determined according to the probability.

[0086] After determining the first relationship tag between the second event and the first event, the embodiment of the present application further updates the second relationship tag between the events in the first entity sequence according to the first relationship tag. The second relationship tag may include one or more than two relationship tags.

[0087] In S203, the second relationship tag of the first entity sequence is updated according to the first relationship tag, and the driving state is identified according to the first relationship tag and the second relationship tag.

[0088] Since the first relationship label between the first event entity and the second event entity corresponding to the first event and the second event is newly determined, the second relationship label between each event entity in the first entity sequence can be updated according to the first relationship label, so that the second relationship label between each event entity in the first entity sequence is more accurate. Based on the accurate first relationship label and second relationship label, the driving state of the driver can be identified according to the preset label relationship, the corresponding relationship between the entity sequence and the driving state, or the driving state of the driver can be identified based on the trained driving state recognition model.

[0089] After determining the relationship label between the first event and the second event, the second event may be updated to the first entity sequence to obtain an updated first entity sequence.

[0090] The entity sequence of the first relationship tag is determined based on the first entity sequence and the first event, as follows: Figure 4 As shown, the first entity sequence in the graph includes event entity 1, event entity 2 and event entity 3, which are eye-closing entity 1, eye-closing entity 2 and balanced deceleration entity 3. The first event detected is event entity 4, and the entity name is vehicle distance too close entity. The event attributes of each event are introduced as follows:

[0091] A1, 12.00.10 generates closed eye entity 1, the start time of its event attributes is 12.00.00, the end time is 12.00.10, and the duration of this entity is 10s.

[0092] A2, 12.00.20 generates closed-eye entity 2, whose event attribute start time is 12.00.15, end time is 12.00.20, and duration of this entity is 5s. Based on the logical relationship (which can be determined by historical statistical information or based on the relationship label prediction model), it can be known that the relationship label between entity 1 and entity 2 is that the driver is severely fatigued.

[0093] A3, 12.00.23 generates a smooth deceleration entity 3, whose event attribute starts at 12.00.18 and ends at 12.00.28. The duration of this entity is 10s, the speed is reduced from 40km / h to 0km / h, and the deceleration is 1.1m / s^2. According to the logical sequence association of the current event entity 1, event entity 2, and event entity 3, the relationship between event entity 2 and event entity 3 is still that the driver is severely fatigued.

[0094] A4, 12.00.25 generates the vehicle distance too close entity 4, whose event attribute starts at 12.00.20 and ends at 12.00.30. According to the conditional events of event entity 1, event entity 2, event entity 3, and event entity 4, the logical relationship between event entity 3 and event entity 4 is determined, and it can be known that this behavior sequence is caused by the normal action of the driver, rather than the driver's severe fatigue. Therefore, the relationship between event entity 3 and event entity 4 is determined to be the driver's stable driving.

[0095] Due to the relationship labels of event entity 3 and event entity 4, according to the pre-statistical correspondence, or according to the pre-trained driving state recognition model, it can be determined that the relationship label of event entity 2 and event entity 3 is the driver's personalized eye-closing habit, and the relationship label of entity 1 and entity 2 is also the driver's personalized eye-closing habit. At this time, the knowledge graph can be as follows Figure 5 shown.

[0096] In yet another embodiment, Figure 6 As shown:

[0097] B1, generates a driver lane change entity (event entity 1) at 12.00.05, and its event attributes are: lane change start time 12.00.02, start speed 70km / h, lane change end time 12.00.05, and lane change end speed 80km / h.

[0098] B2, at 12.00.08, generates the driver reply movement entity (event entity 2), whose event attributes are: start time 12.00.00, start speed 60km / h, end time 12.00.08, end speed 85km / h. Based on the logical relationship between event entity 1 and event entity 2, it can be inferred that the relationship label between event entity 1 and event entity 2 is caused by abnormal operation of the driver.

[0099] B3, generates the front vehicle insertion entity (event entity 3) at 12.00.20, and its event attributes are: start time 12.00.18, end time 12.00.20. The logical relationship between event entity 1, event entity 2 and event entity 3 predicts that the relationship label at this time is abnormal driver operation.

[0100] B4, 12.00.22 generates the driver's sudden deceleration entity (event entity 4), whose event attributes are: start time is 12.00.17, end time is 12.00.22, and deceleration is 23.6m / s^2. Due to the logical relationship between event entity 1, event entity 2, event entity 3 and event entity 4, it can be inferred that the relationship label between event entity 3 and event entity 4 is abnormal driver operation.

[0101] B5, 12.00.30 generates the driver getting off the car entity (event entity 5), whose event attributes are: start time is 12.00.30, end time is 12.00.30. Based on the logical dependencies of event entity 1, event entity 2, event entity 3, event entity 4, and event entity 5 (calculating the probability of relationship labels by taking event entities 1-5 as conditional events), it can be inferred that the relationship label between event entity 5 and event entity 4 is the vehicle hijacking relationship. At this time, the knowledge graph composed of event entity 1 to event entity 5 is as follows: Figure 6 shown.

[0102] B6, from the graph link relationship of event entity 1, event entity 2, event entity 3, event entity 4 and event entity 5, it can be seen that the relationship label between event entity 4 and event entity 3 is vehicle hijacked, the relationship label between event entity 2 and event entity 3 is vehicle hijacked, and the relationship label between event entity 1 and event entity 2 is vehicle hijacked. At this time, the knowledge graph composed of event entity 1 to event entity 5 with the second relationship label updated is as follows Figure 7 shown.

[0103] It can be seen from the above two examples that by updating the second relationship tag in the first entity sequence with a new first relationship tag, the driving state of the driver can be determined more accurately through the updated first relationship tag and the second relationship tag.

[0104] In addition, the embodiment of the present application constructs an entity sequence through a knowledge graph, and constructs an entity sequence for complex events through triplets of event entities, event attributes, and relationship labels, which is conducive to rapid retrieval and query of relevant information without the need for tedious event searches in massive data.

[0105] Through entity sequences, scattered risk events can be associated, and through the association of adjacent event entity pairs, the complete risk process can be traced back, which helps to obtain sufficient information to complete risk identification.

[0106] After completing the construction of the knowledge graph, new relationship labels can be quickly constructed and confirmed based on the constructed knowledge graph.

[0107] By calculating relationship tags and identifying driving status on the server side, the terminal side only needs to collect events, which can greatly reduce the complexity of the device side and reduce equipment costs.

[0108] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0109] Figure 8 A schematic diagram of a driving state recognition device provided in an embodiment of the present application, the device comprising:

[0110] The event collection unit 801 is used to collect the first event of the vehicle during driving, where the first event includes a first event attribute and a first event entity used as a node in the graph.

[0111] The relationship label determination unit 802 is used to add the first event entity of the first event to the first entity sequence in the graph according to the first event attribute, and determine the first relationship label between the first event and the first entity sequence. The graph includes the first entity sequence determined by the event set before the first event, and the first entity sequence represents the relationship between two adjacent event entities in the first entity sequence through the second relationship label.

[0112] The driving state determining unit 803 is configured to update the second relationship tag of the first entity sequence according to the first relationship tag, and identify the driving state according to the first relationship tag and the second relationship tag.

[0113] Figure 8 The driving state recognition device shown is Figure 2The driving state recognition method shown corresponds to.

[0114] Fig. 9 is a schematic diagram of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 9 of this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90, such as a driving state recognition program. When the processor 90 executes the computer program 92, the steps in the above-mentioned driving state recognition method embodiments are implemented. Alternatively, when the processor 90 executes the computer program 92, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0115] Exemplarily, the computer program 92 may be divided into one or more modules / units, one or more modules / units are stored in the memory 91, and are executed by the processor 90 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 92 in the electronic device 9.

[0116] The electronic device 9 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will appreciate that Fig. 9 It is only an example of an electronic device 9 and does not constitute a limitation on the electronic device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0117] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0118] The memory 91 may be an internal storage unit of the electronic device 9, such as a hard disk or memory of the electronic device 9. The memory 91 may also be an external storage device of the electronic device 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 9. Further, the memory 91 may also include both an internal storage unit of the electronic device 9 and an external storage device. The memory 91 is used to store computer programs and other programs and data required by the electronic device. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0119] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0126] In addition, an embodiment of the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the methods in the above-mentioned implementation modes.

[0127] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A driving state recognition method, characterized in that: Methods include: Collecting a first event of the vehicle during driving, the first event including a first event attribute and a first event entity used as a node in the graph; Adding a first event entity of the first event to a first entity sequence in the graph according to the first event attribute, determining a first relationship label between the first event and the first entity sequence, the graph including the first entity sequence determined by the event set before the first event, the first entity sequence characterizing the relationship between two adjacent event entities in the first entity sequence through a second relationship label; The second relationship tag of the first entity sequence is updated according to the first relationship tag, and the driving state is identified according to the first relationship tag and the second relationship tag.

2. The method according to claim 1, characterized in that Determining a first relationship label between a first event and a first entity sequence in a graph includes: determining a second event in the first entity sequence that is associated with the first event; Determine a conditional event according to the first event and the event set in the first entity sequence, and obtain, based on the conditional event, the probability that the relationship between the first event and the second event is a different relationship label; A first relationship label between the first event and the second event is determined according to the probability.

3. The method according to claim 2, characterized in that Based on the conditional event, the probability of the relationship between the first event and the second event being different relationship labels is obtained, including: According to historical statistical data, the probability of obtaining a relationship between the first event and the second event as different relationship labels based on the conditional event is determined; Alternatively, the first event and the first entity sequence are input into a pre-trained probability prediction model to determine the probability that the relationship between the first event and the second event is a different relationship label.

4. The method according to claim 1, characterized in that: According to historical statistical data, the probability of obtaining a relationship between the first event and the second event as different relationship labels based on the conditional event is determined, including: When the driver of the vehicle has been in the fleet system for longer than the predetermined time, the probability of obtaining different relationship labels for the relationship between the first event and the second event based on the conditional event is determined according to the knowledge graph determined by the historical statistical data of the fleet; When the length of time that the driver of the vehicle has joined the fleet system is less than or equal to a predetermined length of time, the probability of obtaining a relationship between a first event and a second event with different relationship labels based on a conditional event is determined according to a knowledge graph determined based on the driver's historical statistical data.

5. The method according to claim 1, characterized in that Adding a first event entity of the first event before a first entity sequence in the graph according to the first event attribute, the method further comprising: According to the event end time in the first event attribute, the connection order of each entity in the first entity sequence is determined.

6. The method according to claim 1, characterized in that The first event of the vehicle during driving is collected, including at least one of the following methods: Acquiring the action event of the driver of the vehicle through the driver monitoring system; Acquire environmental events during vehicle driving through advanced driver assistance systems; The motion event of the vehicle is acquired through the motion sensor of the vehicle.

7. The method according to claim 6, characterized in that Action events include eye closing events, At least one of yawning, looking around, looking down, looking down, making phone calls, smoking, eating, drinking, and large movements of the driver; The environmental event includes at least one of a front collision event, a vehicle-to-vehicle distance event, a pedestrian collision event, a lane departure event, and a lane keeping abnormality event; The motion event includes at least one of an acceleration event, a smooth deceleration event, a sharp deceleration event, a smooth deceleration event, a sharp turn event, a smooth turn event, a collision event, and a rollover event.

8. A driving state recognition device, characterized in that: The device includes: An event collection unit, used for collecting a first event of the vehicle during driving, the first event including a first event attribute and a first event entity used as a node in the graph; a relationship label determination unit, configured to add a first event entity of a first event to a first entity sequence in a graph according to a first event attribute, and determine a first relationship label between the first event and the first entity sequence, wherein the graph includes a first entity sequence determined by an event set before the first event, and the first entity sequence represents a relationship between two adjacent event entities in the first entity sequence through a second relationship label; The driving state determining unit is used to update the second relationship tag of the first entity sequence according to the first relationship tag, and identify the driving state according to the first relationship tag and the second relationship tag.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer program product comprising computer program instructions, characterized in that When the computer program is executed, the method according to any one of claims 1 to 7 is performed.

Images