Driving state identification method and apparatus, and device and program product

Abstract

The present application relates to the field of intelligent driving, and particularly relates to a driving state identification method and apparatus, and a device and a program product. The method comprises: collecting a first event of a vehicle during driving, wherein the first event comprises a first event attribute and a first event entity which is used as a node in a graph; on the basis of the first event attribute, adding the first event entity of the first event to a first entity sequence in the graph, and determining a first relationship label between the first event and the first entity sequence, wherein the graph comprises the first entity sequence determined on the basis of a set of events before the first event, and the first entity sequence represents a relationship between two adjacent event entities in the first entity sequence by means of a second relationship label; and on the basis of the first relationship label, updating the second relationship label of the first entity sequence, and on the basis of the first relationship label and the second relationship label, identifying a driving state. The second relationship label in the first entity sequence is updated on the basis of the first relationship label, thus being conducive to improving the accuracy of an identified driving state.

Description

Driving status recognition methods, devices, equipment and procedures products Technical Field

[0001] This application relates to the field of intelligent driving, and in particular to a driving state recognition method, device, equipment, and program product. Background Technology

[0002] Driving state recognition refers to the use of various sensors and algorithms to monitor and analyze the driver's behavior, physiological state, and driving-related environmental factors to determine whether the driver is in a safe driving state. This includes identifying whether the driver is fatigued, distracted, intoxicated, or in other states that may impair driving ability. Driving state recognition is an important component of intelligent driving systems, contributing to improved road safety and reduced traffic accidents.

[0003] Currently, intelligent driving systems typically identify driving states based on single events or events occurring over a period of time. When determining driving states through a single event, the driver's behavior and physiological state can be subject to chance and individual differences. For example, emergency deceleration does not necessarily indicate dangerous driving; it could simply be a driver's habitual behavior. Events occurring over a period of time may also reflect individual driver habits, which is detrimental to improving the accuracy of driving state identification. Technical issues

[0004] In view of this, embodiments of this application provide a driving state recognition method, apparatus, device, and program product to solve the problem that the prior art is not conducive to improving the accuracy of driving state recognition. Technical solutions

[0005] A first aspect of this application provides a driving state recognition method, the method comprising:

[0006] The first event during the vehicle's driving process is collected. The first event includes first event attributes and first event entities used as nodes in the graph.

[0007] Based on the attributes of the first event, the first event entity of the first event is added to the first entity sequence in the graph, and the first relation label between the first event and the first entity sequence is determined. The graph includes the first entity sequence determined by the event set before the first event. The first entity sequence is characterized by the relationship between two adjacent event entities in the first entity sequence through the second relation label.

[0008] Update the second relation label of the first entity sequence according to the first relation label, and identify the driving state according to the first relation label and the second relation label.

[0009] In conjunction with the first aspect, in a first possible implementation of the first aspect, determining the first relation label between the first event and the first entity sequence in the graph includes:

[0010] Identify the second event in the first entity sequence that is associated with the first event;

[0011] Based on the event set in the first event and the first entity sequence, determine the condition event, and based on the condition event, obtain the probability that the relationship between the first event and the second event is a different relationship label;

[0012] The first relationship label between the first event and the second event is determined based on probability.

[0013] In conjunction with the first possible implementation of the first aspect, in the second possible implementation of the first aspect, based on the conditional events, the probability that the relationship between the first event and the second event has different relation labels is obtained, including:

[0014] Based on historical statistical data, determine the probability that the relationship between the first event and the second event is labeled differently, based on the conditional events;

[0015] Alternatively, the first event and the first entity sequence can be input into a pre-trained probability prediction model to determine the probability that the relationship between the first event and the second event is a different relationship label.

[0016] In conjunction with the first aspect, in the third possible implementation of the first aspect, based on historical statistical data, the probability of obtaining different relation labels for the relationship between the first event and the second event, based on the conditional events, is determined, including:

[0017] When the duration of a vehicle driver's time in the fleet system exceeds the predetermined duration, the probability of obtaining different relationship labels between the first event and the second event based on the knowledge graph determined by the fleet's historical statistical data is determined, based on the conditional events.

[0018] When the driver of a vehicle has been in the fleet system for less than or equal to the predetermined duration, the probability of obtaining different relational labels between the first event and the second event is determined based on the knowledge graph determined by the driver's historical statistical data, on the basis of the conditional events.

[0019] In conjunction with the first possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the method further includes adding the first event entity of the first event to the first entity sequence in the graph based on the first event attribute:

[0020] The connection order of each entity in the first entity sequence is determined based on the event end time in the first event attribute.

[0021] In conjunction with the first aspect, in the fifth possible implementation of the first aspect, the first event of the vehicle during driving is collected, including at least one of the following:

[0022] The driver's action events are acquired through a driver monitoring system;

[0023] Acquire environmental events during vehicle driving through advanced driver assistance systems;

[0024] Vehicle motion events are acquired through the vehicle's motion sensors.

[0025] In conjunction with the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, the action event includes at least one of the following: closing eyes event, yawning event, looking left and right event, looking down distraction event, glancing down distraction event, making a phone call event, smoking event, eating event, drinking water event, and driver moving significantly event.

[0026] Environmental events include at least one of the following: forward collision, close following, pedestrian collision, lane departure, and lane keeping abnormality.

[0027] Motion events include at least one of the following: acceleration events, smooth deceleration events, rapid deceleration events, smooth deceleration events, sharp turning events, smooth turning events, collision events, and rollover events.

[0028] A second aspect of this application provides a driving state recognition device, the device comprising:

[0029] The event acquisition unit is used to acquire the first event during the vehicle's driving process. The first event includes first event attributes and first event entities used as nodes in the graph.

[0030] The relation label determination unit is used to add the first event entity of the first event to the first entity sequence in the graph according to the first event attribute, and determine the first relation label between the first event and the first entity sequence. The graph includes the first entity sequence determined by the event set before the first event. The first entity sequence is characterized by the relationship between two adjacent event entities in the first entity sequence through the second relation label.

[0031] The driving state determination unit is used to update the second relation label of the first entity sequence according to the first relation label, and to identify the driving state according to the first relation label and the second relation label.

[0032] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the electronic device to implement the method as described in any of the first aspects.

[0033] A fourth aspect of this application provides a computer program product that, when run on a computer, causes the computer to execute the methods described in the first aspect or its various implementations.

[0034] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods in the first aspect.

[0035] A sixth aspect of this application provides a chip for implementing the methods in the various implementations of the first aspect described above. Specifically, the chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the methods as described in the first aspect or its various implementations. Beneficial effects

[0036] The beneficial effects of this application embodiment compared to the prior art are as follows: When a first event is collected during driving, this application embodiment adds the first event entity of the first event to a pre-determined first entity sequence in a knowledge graph based on the first event attribute of the first event, determines the first relationship label between the first event and the first entity sequence, updates the second relationship label of the first entity sequence based on the first relationship label, and identifies the driving state based on the first and second relationship labels. Because this method can update the second relationship label of the first entity sequence based on the first relationship label between the newly generated first event and the previously generated first entity sequence, the accuracy of the second relationship label is higher. Therefore, a more accurate driving state can be identified based on the first relationship label and the updated second relationship label. Furthermore, by using a knowledge graph search method, the efficiency of driving state identification can be effectively improved. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 is a schematic diagram of an implementation scenario of a driving state recognition method provided in an embodiment of this application;

[0039] Figure 2 is a schematic diagram of the implementation process of a driving state recognition method provided in an embodiment of this application;

[0040] Figure 3 is a schematic diagram of the implementation process for determining relationship tags provided in an embodiment of this application;

[0041] Figure 4 is a schematic diagram of an entity sequence for determining a first relation label based on a first entity sequence and a first event, provided by an embodiment of this application;

[0042] Figure 5 is a schematic diagram of updating a second relation label based on a first relation label according to an embodiment of this application;

[0043] Figure 6 is a schematic diagram of an entity sequence based on a first entity sequence and a first event to determine a first relationship label, according to an embodiment of this application.

[0044] Figure 7 is a schematic diagram of updating a second relation label based on a first relation label according to an embodiment of this application;

[0045] Figure 8 is a schematic diagram of a driving state recognition device provided in an embodiment of this application;

[0046] Figure 9 is a schematic diagram of an electronic device provided in an embodiment of this application. Embodiments of the present invention

[0047] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0048] To illustrate the technical solution of this application, specific embodiments are described below.

[0049] Driving condition recognition technology uses multiple sensors and algorithms to monitor drivers' behavioral patterns, physiological responses, and the driving environment in real time, thereby assessing whether the driver is in a condition suitable for safe driving. This technology can identify whether a driver exhibits fatigue, inattention, intoxication, or other conditions that may impair their driving skills. As a key component of intelligent driving systems, driving condition recognition technology plays a crucial role in enhancing road safety and reducing accident rates.

[0050] In current intelligent driving systems, driving status is often identified by analyzing a single event or several events over a period of time. However, judging driving status based solely on a single event can be affected by the randomness and individual differences in driver behavior and physiological responses. For example, an emergency deceleration does not always indicate that the driver is in a dangerous situation; it may simply be part of their personal driving habits. Similarly, several events over a period of time may also reflect the driver's personal driving habits, which can pose a challenge to improving the accuracy of driving status identification.

[0051] Figure 1 is a schematic diagram of an implementation scenario of a driving state recognition method provided in this application. As shown in Figure 1, the implementation scenario includes a vehicle 1 and a server 2. The vehicle 1 includes a driver monitoring system (DMS) 10, an advanced driver assistance system (ADAS) 11, a motion sensor 12, and a driving state recognition system 13. The driver monitoring system 10 can collect relevant information about the driver through the DMS camera to determine the driver's action events, including at least one of the following: closing eyes, yawning, looking left and right, looking down (distracted), glancing down (distracted), making a phone call, smoking, eating, drinking, and significant driver movement. The advanced driver assistance system 11 can collect environmental events during vehicle driving through the ADAS camera, including at least one of the following: forward collision, following distance too close, pedestrian collision, lane departure, and abnormal lane keeping. The motion sensor 12 can be used to collect vehicle motion information and determine vehicle motion events, including at least one of acceleration events, smooth deceleration events, rapid deceleration events, smooth deceleration events, sharp turning events, smooth turning events, collision events, and rollover events. The vehicle or server can generate a graph including entity sequences based on the collected events and their attributes, using relational tags to characterize the relationship between two adjacent events in the entity sequence.

[0052] For any newly acquired first event, the driving state recognition system 13 or service 2 can construct and update a knowledge graph. The knowledge graph can determine the first relation label between the first event and the first entity sequence, and the second relation label of the previously obtained first entity sequence can be updated based on the first relation label. The driving state can then be identified based on the first and second relation labels. By searching through entity sequences, the computational burden on the vehicle terminal can be effectively reduced.

[0053] Figure 2 is a schematic diagram of the implementation flow of a driving state recognition method provided in an embodiment of this application, which is described in detail below:

[0054] In S201, the first event during the vehicle's driving process is collected.

[0055] The first event includes a first event entity and a first event attribute, where the first event entity is used as a node in the graph. Any event can include both an event entity and an event attribute. When an event entity is used as a node in the graph, the name of the event entity can be used as the name of the node. For example, the name of the event entity can be determined based on the event name. Event entity names can include at least one of the following: closed eyes entity, yawning entity, looking left and right entity, head down distracted entity, glance downwards distracted entity, making a phone call entity, smoking entity, eating entity, drinking entity, forward collision entity, too close following entity, pedestrian collision entity, lane departure entity, lane keeping error entity, rapid acceleration entity, rapid deceleration entity, sharp turn entity, collision entity, and rollover entity.

[0056] When collecting the first event, image data can be collected through vehicle 1, including DMS camera and ADAS camera, or motion data of the vehicle can be collected through motion sensor. Based on the collected image data or motion data, the event during the driving process can be determined, and the event attribute of the event can be determined.

[0057] Specifically, the DMS camera can collect driver image data during the driving process, and based on the collected driver image data, determine the driver's action events. The determined action events may include at least one of the following: driver closing eyes, yawning, looking left and right, looking down in distraction, glancing down in distraction, making a phone call, smoking, eating, drinking water, and significant driver movement.

[0058] Image data of the vehicle's surrounding environment collected by ADAS cameras is used to determine environmental events during vehicle driving. The determined environmental events may include at least one of the following: forward collision event, close following event, pedestrian collision event, lane departure event, and lane keeping abnormality event.

[0059] Motion data of the vehicle is collected using motion sensors, including motion state detection devices such as accelerometers or gyroscopes. Based on the vehicle's motion data, motion events of the vehicle are determined. These motion events can include at least one of the following: acceleration events, smooth deceleration events, rapid deceleration events, smooth deceleration events, sharp turning events, smooth turning events, collision events, and rollover events.

[0060] The attributes of any event may include one or more of the following: event start time, event end time, speed at the start of the event, speed at the end of the event, driving time at the start of the event, and vehicle position at the start of the event.

[0061] When determining event attributes, the duration of the detected state can be considered to determine whether the event is valid. For example, when detecting an eye-closing event, the business process might include:

[0062] 1. The timer starts when either eye is detected to be closed.

[0063] 2. Once the timing starts, if it is detected that both eyes are not closed and the duration is longer than the first duration, such as more than 2 frames (i.e., two consecutive frames of images showing the eyes closed can be detected; the duration of 2 frames is related to the frame rate. After determining the frame rate, the frame interval duration can be determined based on the frame rate, and the number of frame intervals can be determined based on the number of frames. The actual duration corresponding to a specific frame duration can be determined based on the number of frame intervals. The first duration can be set as needed), then the timing for this event ends.

[0064] 3. If the duration of eye closure is less than the second duration, such as less than 200ms (which can be adjusted as needed), the event timed in this instance is invalid and discarded.

[0065] 4. If both eyes are not closed and the duration is less than the first duration, such as less than 2 frames (which can be set as needed), and then the driver is detected to be in a closed-eye state, then the closed-eye event is considered not to have ended.

[0066] 5. When the previous eye-closing event ends, if the left or right eye is detected to be in a closed state again, the timer starts. If the left or right eye is detected to be in a non-closed state and the duration is longer than the first duration, such as more than 2 frames (configurable), the timer ends and the timer ends is determined as the timer end time.

[0067] 6. When both eyes are detected to be closed, the timer starts. When both eyes are detected to be open (both of the driver's eyes are open) and the duration is longer than the first timer, such as more than 2 frames (which can be set as needed), the timer ends and the duration of both eyes being closed is calculated.

[0068] In S202, the first event entity of the first event is added to the first entity sequence in the graph according to the first event attribute, and the first relationship label between the first event and the first entity sequence is determined.

[0069] The graph includes a first entity sequence determined by the event set preceding the first event. The first entity sequence is characterized by a second relation label representing the relationship between two adjacent event entities in the first entity sequence.

[0070] The first entity sequence is determined based on events collected during vehicle driving; "first" distinguishes it from other entity sequences. Generally, the time interval between events can be used to determine if the first entity sequence in the map belongs to the same entity sequence. For example, if the end time of the first event is a first moment, the end time of the second event is a second moment, and the interval between the first and second moments is longer than a predetermined third time interval, such as greater than 100 seconds, then the first and second event entities can be determined to belong to different entity sequences. For subsequent event entities, new entity sequences can be created for identifying driving states.

[0071] In one possible implementation, the event entities in the first entity sequence can be dynamically updated based on a pre-defined number of event entities. For example, when a new first event is detected, the newly detected first event entity is added to the first entity sequence, and the event entity in the first entity sequence that is furthest from the end time of the first event is deleted, so that the set number of event entities in the first entity sequence is dynamically maintained.

[0072] When an entity sequence (including the first entity sequence) contains two or more events, the connection relationship between the entity sequences can be determined based on the end times of the events. That is, based on the end time of an event, one or two events most closely associated with that end time can be found. For example, if the event set in the event sequence includes a first event, a second event, and a third event, with corresponding end times t1, t2, and t3, and corresponding event entities of the first event entity, the second event entity, and the third event entity, where t1 is earlier than t3 and t3 is earlier than t2, then the connection order of the event entities is the first event entity, the third event entity, and the second event entity.

[0073] In this embodiment of the application, the relationships between event entities in the graph are represented by relationship labels. Relationship labels may include, for example, vehicle hijacking, severe driver fatigue, severe driver fatigue leading to abnormal behavior, severe driver distraction, mild driver fatigue, mild driver distraction, aggressive driving, conservative driving, smooth driving, following too closely, personalized driver habits, abnormal driver operation, and normal driver behavior.

[0074] In this embodiment, when adding the first event to the first entity sequence in the graph, if the vehicle is cold-started and the first entity sequence is empty, the first event can be directly used as the first node of the first entity sequence in the graph. In this case, it is not necessary to determine the relationship label between the first event and the first entity sequence. Alternatively, if the interval between the end time of the first event and the end time of the previous event is too long, and the first entity sequence and the first event entity belong to different entity sequences, it is not necessary to determine the relationship label between the first event and the first entity sequence.

[0075] If the first entity sequence is not empty, and the duration between the end time of the first event and the end time of the second event (i.e., the last event in the first entity sequence) is less than the third duration, then an association between the first event entity and the second event entity can be established, and the first relationship label between the first event and the first entity sequence can be determined. Specifically, as shown in Figure 3, this includes:

[0076] In S301, a second event associated with the first event in the first entity sequence is determined.

[0077] In general, the second event in the first entity sequence that is related to the first event is the event corresponding to the event entity at the end of the first entity sequence. The end time of the second event is usually before the end time of the first event.

[0078] In S302, conditional events are determined based on the first event and the event set in the first entity sequence. Based on the conditional events, the probability that the relationship between the first event and the second event is a different relation label is obtained.

[0079] When determining the relationship between the first event and the second event, it is necessary to determine the number of events in the first entity sequence.

[0080] When the event entity in the first entity sequence is 1, the second event and the first event can be directly used as condition events. Based on the second event, the probability of the relationship between the first event and the second event being different relation labels can be obtained, and the probability of all relation labels on the condition events can be calculated.

[0081] When the number of event entities in the first entity sequence is greater than 1, an event set of more than two events can be determined based on the first entity sequence. A conditional event is then determined based on the first event and the event set. The probability of generating different relational labels is then determined based on the conditional event.

[0082] When determining the first relationship label between the first event and the second event based on the conditional event, the probability of the current conditional event relative to different relationship labels can be found based on the knowledge graph determined by the driver's historical statistical data. Alternatively, the first event and the first entity sequence can be input into a pre-trained probability prediction model to determine the probability of the relationship between the first event and the second event being different relationship labels, and the relationship label with the highest probability can be selected to determine the association between the first event and the second event.

[0083] When determining the probability of different relationship labels based on historical statistical data, if a driver has been in the fleet system for less than a predetermined period, such as less than one year, the amount of historical statistical data for that driver is relatively small. In order to improve the accuracy of historical statistical data, the probability of different relationship labels based on conditional events can be determined based on the knowledge graph determined by the fleet's historical statistical data.

[0084] If the duration a driver spends in the fleet is greater than or equal to the predetermined duration, the probability of different related labels can be determined directly using the knowledge graph based on the driver's historical statistical data, based on the event conditions.

[0085] In S303, the first relationship label between the first event and the second event is determined based on probability.

[0086] After determining the first relationship label between the second event and the first event, embodiments of this application further update the second relationship label between events in the first entity sequence based on the first relationship label. The second relationship label may include one or more relationship labels.

[0087] In S203, the second relation label of the first entity sequence is updated according to the first relation label, and the driving state is identified according to the first relation label and the second relation label.

[0088] Since a new first relation label has been determined between the first event entities and the second event entities corresponding to the first event and the second event, the second relation labels between each event entity in the first entity sequence can be updated based on the first relation label, making the second relation labels between each event entity in the first entity sequence more accurate. Based on the accurate first and second relation labels, the driver's driving state can be identified according to the preset label relationship, the correspondence between the entity sequence and the driving state, or it can be identified based on the trained driving state recognition model.

[0089] After determining the relationship label between the first event and the second event, the second event can be updated to the first entity sequence to obtain the updated first entity sequence.

[0090] The entity sequence for determining the first relation label based on the first entity sequence and the first event can be shown in Figure 4. The first entity sequence in the graph includes event entity 1, event entity 2, and event entity 3, which are respectively the closed-eye entity 1, closed-eye entity 2, and balanced deceleration entity 3. The first event detected is event entity 4, with the entity name "vehicle distance too close entity". The event attributes of each event are described below:

[0091] A1, 12.00.10 generates entity 1 with closed eyes, whose event attribute start time is 12.00.00, end time is 12.00.10, and the duration of this entity is 10s.

[0092] A2, at 12:00:20, generates entity 2 with closed eyes. Its event attribute start time is 12:00:15, end time is 12:00:20, and the duration of this entity is 5 seconds. Based on the logical relationship (which can be determined by historical statistical information or by a relationship label prediction model), the relationship label between entity 1 and entity 2 is "driver is severely fatigued".

[0093] In instance A3, version 12.00.23, a smooth deceleration entity 3 is generated. Its event attribute start time is 12.00.18, end time is 12.00.28, duration is 10 seconds, and speed decreases from 40 km / h to 0 km / h, with a deceleration of 1.1 m / s². Based on the logical sequence of event entities 1, 2, and 3, the relationship between event entities 2 and 3 remains that the driver is severely fatigued.

[0094] A4, at 12:00:25, generated Entity 4, indicating a close following distance. Its event attribute starts at 12:00:20 and ends at 12:00:30. Based on the conditional events of Entities 1, 2, 3, and 4, the logical relationship between Entities 3 and 4 indicates that this sequence of actions was caused by normal driver behavior, not severe driver fatigue. Therefore, the relationship between Entities 3 and 4 is determined to be that the driver was driving smoothly.

[0095] Based on the relationship labels of event entities 3 and 4, and according to pre-statistical correspondences or a pre-trained driving state recognition model, the relationship label between event entities 2 and 3 can be determined to be "driver's personalized eye-closing habit," and the relationship label between entity 1 and entity 2 can also be "driver's personalized eye-closing." The knowledge graph at this point can be as shown in Figure 5.

[0096] In yet another embodiment, as shown in Figure 6:

[0097] B1 generates a driver lane change entity (event entity 1) in 12.00.05. Its event attributes are: lane change start time 12.00.02, start speed 70km / h, lane change end time 12.00.05, and end speed 80km / h.

[0098] B2 generates a driver response movement entity (Event Entity 2) in 12.00.08. Its event attributes are: start time 12.00.00, start speed 60km / h, end time 12.00.08, and end speed 85km / h. Based on the logical relationship between Event Entity 1 and Event Entity 2, it can be inferred that the relationship label between Event Entity 1 and Event Entity 2 is caused by abnormal driver operation.

[0099] B3 generates an entity (Event Entity 3) for the vehicle ahead in time at 12:00:20. Its event attributes are: start time 12:00:18 and end time 12:00:20. Based on the logical relationship between Event Entity 1, Event Entity 2, and Event Entity 3, the relationship label at this time is predicted to be abnormal driver control.

[0100] B4, 12.00.22 generates a driver sudden deceleration entity (event entity 4), with the following event attributes: start time 12.00.17, end time 12.00.22, and deceleration 23.6 m / s^2. Based on the logical relationship between event entities 1, 2, 3, and 4, it can be inferred that the relationship tag between event entities 3 and 4 is abnormal driver control.

[0101] B5, at 12:00:30, generates the driver disembarking entity (Event Entity 5), with event attributes of start time 12:00:30 and end time 12:00:30. Based on the logical dependencies of Event Entities 1, 2, 3, 4, and 5 (calculating the probability of relationship labels using Event Entities 1-5 as conditional events), it can be inferred that the relationship label between Event Entities 5 and Event Entities 4 is a vehicle hijacking relationship. The knowledge graph composed of Event Entities 1 to 5 is shown in Figure 6.

[0102] B6. From the graph linking relationships of event entities 1, 2, 3, 4, and 5, we know that the relationship label between event entities 4 and 3 is "vehicle hijacking," the relationship label between event entities 2 and 3 is "vehicle hijacking," and the relationship label between event entities 1 and 2 is "vehicle hijacking." The knowledge graph composed of event entities 1 to 5, with updated second relationship labels, is shown in Figure 7.

[0103] As can be seen from the two examples above, by updating the second relation label in the first entity sequence with the new first relation label, the driver's driving status can be determined more accurately using the updated first and second relation labels.

[0104] In addition, the embodiments of this application construct entity sequences through knowledge graphs, and construct entity sequences of complex events through triples of event entities, event attributes and relation tags, which is conducive to quickly retrieving and querying relevant information without having to perform tedious event searches in massive amounts of data.

[0105] Entity sequences can link disparate risk events together, and the association of adjacent event entity pairs can trace back the complete risk process, helping to obtain sufficient information to complete risk identification.

[0106] After the knowledge graph is constructed, new relationship tags can be quickly constructed and confirmed based on the constructed knowledge graph.

[0107] By calculating relationship tags and recognizing driving status on the server side, the device side only needs to collect events, which can greatly reduce the complexity of the device and reduce equipment costs.

[0108] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0109] Figure 8 is a schematic diagram of a driving state recognition device provided in an embodiment of this application. The device includes:

[0110] The event acquisition unit 801 is used to acquire the first event during the driving process of the vehicle. The first event includes the first event attributes and the first event entity used as a node in the graph.

[0111] The relation label determination unit 802 is used to add the first event entity of the first event to the first entity sequence in the graph according to the first event attribute, and determine the first relation label between the first event and the first entity sequence. The graph includes the first entity sequence determined by the event set before the first event. The first entity sequence is characterized by the relationship between two adjacent event entities in the first entity sequence through the second relation label.

[0112] The driving state determination unit 803 is used to update the second relation label of the first entity sequence according to the first relation label, and to identify the driving state according to the first relation label and the second relation label.

[0113] The driving state recognition device shown in Figure 8 corresponds to the driving state recognition method shown in Figure 2.

[0114] Figure 9 is a schematic diagram of an electronic device provided in an embodiment of this application. As shown in Figure 9, the electronic device 9 of this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90, such as a driving state recognition program. When the processor 90 executes the computer program 92, it implements the steps in the above-described embodiments of the driving state recognition methods. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module / unit in the above-described device embodiments.

[0115] For example, computer program 92 may be divided into one or more modules / units, one or more of which are stored in memory 91 and executed by processor 90 to complete this application. One or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 92 in electronic device 9.

[0116] Electronic device 9 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The electronic device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will understand that Figure 9 is merely an example of an electronic device 9 and does not constitute a limitation on the electronic device 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0117] The processor 90 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0118] The memory 91 can be an internal storage unit of the electronic device 9, such as a hard disk or RAM. The memory 91 can also be an external storage device of the electronic device 9, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 91 can include both internal and external storage units of the electronic device 9. The memory 91 is used to store computer programs and other programs and data required by the electronic device. The memory 91 can also be used to temporarily store data that has been output or will be output.

[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0120] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0121] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0122] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0125] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0126] In addition, this application also provides a computer program product that, when run on a computer, causes the computer to execute the methods in the above-described implementations.

[0127] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A driving state recognition method, characterized in that, The methods include: The first event during the vehicle's driving process is collected. The first event includes first event attributes and first event entities used as nodes in the graph. Based on the attributes of the first event, the first event entity of the first event is added to the first entity sequence in the graph, and the first relation label between the first event and the first entity sequence is determined. The graph includes the first entity sequence determined by the event set before the first event. The first entity sequence is characterized by the relationship between two adjacent event entities in the first entity sequence through the second relation label. Update the second relation label of the first entity sequence according to the first relation label, and identify the driving state according to the first relation label and the second relation label.

2. The method according to claim 1, characterized in that, Determine the first relation label between the first event and the first entity sequence in the graph, including: Identify the second event in the first entity sequence that is associated with the first event; Based on the event set in the first event and the first entity sequence, determine the condition event, and based on the condition event, obtain the probability that the relationship between the first event and the second event is a different relationship label; The first relationship label between the first event and the second event is determined based on probability.

3. The method according to claim 2, characterized in that, Based on the conditional events, the probabilities that the relationship between the first event and the second event has different relationship labels are obtained, including: Based on historical statistical data, determine the probability that the relationship between the first event and the second event is labeled differently, based on the conditional events; Alternatively, the first event and the first entity sequence can be input into a pre-trained probability prediction model to determine the probability that the relationship between the first event and the second event is a different relationship label.

4. The method according to claim 1, characterized in that, Based on historical statistical data, determine the probability of obtaining different relationship labels for the relationship between the first event and the second event, based on the conditional events, including: When the duration of a vehicle driver's time in the fleet system exceeds the predetermined duration, the probability of obtaining different relationship labels between the first event and the second event based on the knowledge graph determined by the fleet's historical statistical data is determined, based on the conditional events. When the driver of a vehicle has been in the fleet system for less than or equal to the predetermined duration, the probability of obtaining different relational labels between the first event and the second event is determined based on the knowledge graph determined by the driver's historical statistical data, on the basis of the conditional events.

5. The method according to claim 1, characterized in that, The method further includes adding the first event entity of the first event to the first entity sequence in the graph based on the first event attribute: The connection order of each entity in the first entity sequence is determined based on the event end time in the first event attribute.

6. The method according to claim 1, characterized in that, Collect the first event during vehicle driving, including at least one of the following: The driver's action events are acquired through a driver monitoring system; Acquire environmental events during vehicle driving through advanced driver assistance systems; Vehicle motion events are acquired through the vehicle's motion sensors.

7. The method according to claim 6, characterized in that, Action events include at least one of the following: closing eyes, yawning, looking left and right, looking down and being distracted, glancing down and being distracted, making a phone call, smoking, eating, drinking, and significant driver movement. Environmental events include at least one of the following: forward collision, close following, pedestrian collision, lane departure, and lane keeping abnormality. Motion events include at least one of the following: acceleration events, smooth deceleration events, rapid deceleration events, smooth deceleration events, sharp turning events, smooth turning events, collision events, and rollover events.

8. A driving state recognition device, characterized in that, The device includes: The event acquisition unit is used to acquire the first event during the vehicle's driving process. The first event includes first event attributes and first event entities used as nodes in the graph. The relation label determination unit is used to add the first event entity of the first event to the first entity sequence in the graph according to the first event attribute, and determine the first relation label between the first event and the first entity sequence. The graph includes the first entity sequence determined by the event set before the first event. The first entity sequence is characterized by the relationship between two adjacent event entities in the first entity sequence through the second relation label. The driving state determination unit is used to update the second relation label of the first entity sequence according to the first relation label, and to identify the driving state according to the first relation label and the second relation label.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes a computer program, it causes the electronic device to perform the method as claimed in any one of claims 1-7.

10. A computer program product comprising computer program instructions, characterized in that, When a computer program is run, the method of any one of claims 1-7 is performed.

Images