Driving early warning method and device, electronic equipment and program product

Abstract

The invention relates to a driving early warning method and device, electronic equipment and a program product, and the method comprises the steps: obtaining the current driving data of a target vehicle and the current video data of a driver in the driving process of the target vehicle; determining driving behavior information according to the current driving data and the current video data; and determining current driving risk information according to the driving behavior information, and sending out an early warning prompt under the condition that the current driving risk information meets a preset condition. On the basis, the driving risk can be analyzed from different dimensions, and the accuracy of driving risk judgment is improved, so that the safety of a driver in the driving process can be effectively guaranteed, and serious life and property loss is avoided.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a driving warning method, device, electronic equipment and program product. Background Art

[0002] In recent years, with the rapid development of logistics business, large vehicles transporting heavy goods have gradually become frequent visitors on expressways, secondary roads and urban roads.

[0003] In the related technology, an infrared camera can be used to monitor the driver's state during driving, and when the driver's behavior is abnormal, a warning is issued through the corresponding warning device. However, in actual applications, when the driver's state is normal but the current speed of the vehicle is too fast, it may also cause a traffic accident. Therefore, it is difficult to make an accurate warning judgment and avoid the occurrence of traffic accidents by only analyzing the driver's state. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a driving warning method, device, electronic device and program product to solve the problems existing in the related art.

[0005] According to a first aspect of an embodiment of the present disclosure, a driving warning method is provided, which includes: during the driving process of the target vehicle, obtaining current driving data of the target vehicle and current video data of the driver; determining driving behavior information based on the current driving data and the current video data; determining current driving risk information based on the driving behavior information, and issuing a warning prompt when the current driving risk information meets preset conditions.

[0006] A second aspect of an embodiment of the present disclosure provides a driving warning device, which is applied to the driving warning method of the first aspect, and the device includes: an acquisition module, which is used to acquire the current driving data of the target vehicle and the current video data of the driver during the driving of the target vehicle; a generation module, which is used to determine the driving behavior information based on the current driving data and the current video data; a prompt module, which is used to determine the current driving risk information based on the driving behavior information, and issue a warning prompt when the current driving risk information meets the preset conditions.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising at least one processor; and a memory for storing instructions executable by at least one processor; wherein the at least one processor is used to execute instructions to implement the steps of the above method.

[0008] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the steps of the above method.

[0009] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0010] At least one of the above technical solutions adopted in the embodiments of the present disclosure can achieve the following beneficial effects: by obtaining the current driving data of the target vehicle and the current video data of the driver during the driving process of the target vehicle; determining the driving behavior information based on the current driving data and the current video data; determining the current driving risk information based on the driving behavior information, and issuing a warning prompt when the current driving risk information meets the preset conditions. Based on this, the current driving data of the vehicle and the current video data of the driver can be analyzed respectively to determine the driving behavior information, and then the driving risk information generated by the driver during the driving process can be determined based on the driving behavior information of the driver, and a warning prompt can be issued in time when the driving risk information meets the preset conditions. It can be seen that the embodiments of the present disclosure can analyze driving risks from different dimensions and improve the accuracy of driving risk judgment, thereby effectively ensuring the safety of the driver during driving and avoiding serious loss of life and property. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A flowchart of a driving warning method provided by an exemplary embodiment of the present disclosure;

[0013] Figure 2 A schematic diagram of a flow chart of a driving timeline provided for an exemplary embodiment of the present disclosure;

[0014] Figure 3 A schematic diagram of a new message processing flow provided for an exemplary embodiment of the present disclosure;

[0015] Figure 4 A schematic diagram of the structure of a driving warning device provided by an exemplary embodiment of the present disclosure;

[0016] Figure 5 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present disclosure;

[0017] Figure 6 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0020] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] In recent years, with the rapid development of logistics business, large vehicles transporting heavy goods have gradually become frequent visitors on highways, secondary roads and urban roads. As the transportation business develops, the amount of goods that need to be transported increases, which makes the risks of transportation vehicles surge during the transportation process. In order to ensure the driving safety of drivers, a large number of early warning systems for road transport vehicles have appeared across the country, as well as on-board terminals, infrared cameras, pulse sensors, respiratory rate sensors and other sensor devices for monitoring the driver's status. At the same time, technical specifications for on-board terminals based on the Beidou satellite positioning system have been issued, while Jiangsu and Guangdong have issued technical specifications based on video monitoring.

[0024] And with the rapid development of logistics business, there are endless warning products for drivers on the market, but most of them rely on various sensor devices to monitor and directly warn. However, most of the warning products are not professional medical testing equipment. Therefore, the monitoring accuracy of the monitoring equipment is not high, resulting in inaccurate warning timing, and may cause multiple warnings and invalid alarms, which in turn interferes with the driver's driving process. Some drivers even choose to turn off or remove the sensor equipment, which makes the warning product lose its meaning of warning.

[0025] In the related technology, a new multifunctional driver monitoring and safety protection system is disclosed, which can be combined with the existing new energy vehicle wire control chassis technology. When the infrared camera detects that the driver has violated the rules, the signal is transmitted to the alarm device for alarm, and the data is uploaded to the monitoring cloud module to record the driver's illegal driving; at the same time, when the infrared camera detects that the passenger has violated the rules, that is, when the driver's normal driving behavior is obstructed, the control unit sends a brake signal and automatically sends a control signal to keep the car's angle at 0°, cut off the angle input from the steering wheel, force the vehicle to return the steering wheel to the center and actively brake, so as to reduce the potential traffic risk of passengers grabbing the steering wheel; the monitoring cloud immediately determines the specific location of the vehicle and calls the police nearby, using the convenience of the information age to quickly deal with the relevant personnel who violate the law and discipline, and maintain the personal safety of other passengers on the car. Based on this, the infrared camera can be used to monitor the behavior of the driver and passengers, and the detected violations of the driver and passengers will be fed back to the control unit in real time, and then the control unit will transmit the signal to the alarm device; the monitoring cloud module is used to receive the signal data of the driver and passenger violations sent by the control unit, upload and record it, and record the voice warning of the violation and search the vehicle location for the nearest alarm.

[0026] It can be seen that the above method mainly uses infrared cameras to monitor the behavior of drivers and passengers. When the driver or passenger violates the rules, the signal is transmitted to the alarm device to alarm, and the data is uploaded to the monitoring cloud module to record the driver's illegal driving. The infrared camera in this system is only used to identify people's illegal behaviors, that is, DMS detection. It lacks ADAS and BSD detection of vehicle driving information, driving environment information, and driving roads, resulting in an imperfect detection system. In addition, this method only uploads the violation data to the cloud for recording, and does not perform secondary identification and screening in the cloud to filter out invalid alarm information, which is very prone to duplicate and invalid alarms.

[0027] In the related art, a school bus driver monitoring system is also disclosed. It can collect facial images of school bus drivers and use pulse sensors and respiratory rate sensors to determine whether the driver is in a fatigued state. To a certain extent, it can effectively ensure that school bus drivers will not cause traffic accidents due to sudden illness or fatigue driving. However, this monitoring system can only detect driver fatigue driving and dangerous driving behaviors in poor physical condition. It cannot monitor some illegal behaviors of drivers such as playing with mobile phones, drinking water, and making phone calls, resulting in a limited monitoring range of the system. In addition, the use of pulse sensors and respiratory rate sensors will affect the driver's driving, and on the other hand, it will increase the driving risk factor.

[0028] In response to the above problems, the embodiments of the present disclosure provide a driving warning method, device, electronic device and program product, which can collect the current driving data of the vehicle during driving, and analyze the driving state of the vehicle based on the current driving data. At the same time, when the vehicle is in driving state, the current video data corresponding to the driver is obtained, and the driver's driving behavior is analyzed based on the current video data. Then, the driver's driving state is determined based on the analysis results of the current driving data and the current video data, and finally, it can be determined whether to issue a warning prompt based on the driver's driving state. Based on this, driving risks can be analyzed from different dimensions to improve the accuracy of driving risk judgment, thereby effectively ensuring the safety of the driver during driving and avoiding serious loss of life and property.

[0029] The driving warning method provided by the embodiment of the present disclosure may be executed by a terminal or by a chip applied to the terminal.

[0030] Exemplarily, the above-mentioned terminals may include one or more of mobile phones, tablet computers, wearable devices, vehicle-mounted devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and wearable devices based on augmented reality (AR) and / or virtual reality (VR) technology, and may also include but not be limited to remote control devices, wearable devices, street lamps, home appliances and other smart terminals, which are not specifically limited in the embodiments of the present disclosure.

[0031] Figure 1 The following is a flow chart of a driving warning method provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the method specifically includes:

[0032] S101: During the driving process of the target vehicle, the current driving data of the target vehicle and the current video data of the driver are obtained. It should be understood that the current driving data may include information such as the current driving speed, the current driving position and the current driving road.

[0033] In some embodiments, a plurality of different types of sensing devices are generally pre-loaded on the vehicle to obtain information such as the current driving speed, current driving position, and current driving environment of the vehicle in real time. At the same time, an image acquisition device can also be set in the interior of the vehicle to collect the current video data of the driver during driving, wherein the image acquisition range of the image acquisition device includes but is not limited to the driver's facial video data and the driver's arm behavior video data, which is mainly used to analyze the driver's driving behavior and the driver's own state, and the image acquisition device can be a high-definition camera, a high-speed camera, etc., and multiple image acquisition devices can be set in a vehicle, so that the driver's driving process of the target vehicle can be monitored in an all-round manner to obtain more comprehensive data. It should be understood that the current driving environment can be directly determined according to the regional weather conditions corresponding to the current driving position.

[0034] S102: Determine driving behavior information according to current driving data and current video data.

[0035] In some embodiments, there may be an interval speed limit for a certain road section, but the driver fails to obtain the speed limit information in time. This will result in an increase in driving risk even if the current video data confirms that the driver's driving behavior is normal, because the driver's vehicle is driving too fast, that is, when the speed information corresponding to the interval speed limit is exceeded. Therefore, the disclosed embodiment not only analyzes the current video data, but also analyzes the current driving data of the vehicle, so that the driving risk can be analyzed from different dimensions to improve the accuracy of driving risk judgment.

[0036] S103: Determine current driving risk information based on the driving behavior information, and issue a warning prompt if the current driving risk information meets a preset condition.

[0037] In some embodiments, a warning device may be installed in the vehicle interior, and the warning device may be used to issue a warning prompt, which may be a voice broadcast or a warning signal continuously issued for a period of time. Based on this, when the driving risk score is greater than or equal to the preset warning value, the warning device may issue a warning prompt to the driver, so that the driver can quickly adjust the current driving behavior according to the warning prompt, thereby reducing the risks that may exist during driving.

[0038] It can be seen that the disclosed embodiment can obtain the current driving data of the target vehicle and the current video data of the driver during the driving process of the target vehicle; determine the driving behavior information based on the current driving data and the current video data; determine the current driving risk information based on the driving behavior information, and issue a warning prompt when the current driving risk information meets the preset conditions. Based on this, the current driving data of the vehicle and the current video data of the driver can be analyzed respectively to determine the driving behavior information, and then the driving risk information generated by the driver during the driving process can be determined based on the driving behavior information of the driver, and a warning prompt can be issued in time when the driving risk information meets the preset conditions. It can be seen that the disclosed embodiment can analyze driving risks from different dimensions and improve the accuracy of driving risk judgment, thereby effectively ensuring the safety of the driver during driving and avoiding serious loss of life and property.

[0039] In some embodiments, the embodiments of the present disclosure can also determine the abnormal moment according to the driving behavior information when the driving behavior information is abnormal driving behavior information; taking the abnormal moment as the starting point, obtain the video frames corresponding to the driver at different moments within a first preset time length, and the driving data corresponding to the different moments; determine the current driving risk information corresponding to the first preset time length according to the video frames and the driving data.

[0040] Specifically, the driving behavior information may include the driver's behavior during driving, such as information corresponding to dozing off, drinking water, and sneezing, and may also include information such as the vehicle's current speed and current driving environment. Based on this, the driving behavior information may be determined to be abnormal driving behavior information when the driver is dozing off, drinking water, and sneezing; or when the vehicle's current speed exceeds the maximum speed corresponding to the current road, the driving behavior information may be determined to be abnormal driving behavior information.

[0041] In actual applications, the occurrence of the above behaviors is accidental, that is, the driver generally will not drink water or sneeze continuously during the entire driving process. Therefore, in order to save costs, the disclosed embodiment can design a time window, wherein the time window can be used to describe a time period, which is usually used to describe the completion time of a given task, and the requirement for a specific task to be completed within a given time period. Based on this, the time window can be opened when it is determined that the driving behavior information is abnormal driving behavior information, and then the abnormal driving behavior is monitored in real time within the first preset time period corresponding to the time window, and the driving risk score corresponding to the first preset time period is determined. Based on this, not only can the abnormal driving behavior be monitored in real time, but also the monitoring cost can be reduced.

[0042] For example, the first preset duration corresponding to the time window can be set to 5 minutes. At this time, when it is determined that the driver has dozed off according to the video frames corresponding to different moments, the driving behavior information at this time is confirmed to be abnormal driving behavior information, and the moment when the dozing event occurred is determined to be the abnormal moment. At this time, the abnormal moment can be used as the time starting point to open the time window, and the video frames at different moments within the first preset duration and the driving data corresponding to the vehicle at different moments can be obtained, and then the current driving risk information corresponding to the first preset duration can be determined based on the video frames and driving data. It should be understood that the embodiment of the present disclosure can obtain video frames and driving data at different moments in real time, and then the current driving risk information corresponding to different moments can be determined based on the video frames and driving data at different moments.

[0043] In practical applications, the number of times the driver dozed off in multiple video frames can be obtained, and then the current driving risk score can be determined based on the number combined with a pre-built driving risk comparison table. The driving risk comparison table can be used to obtain driving risk scores corresponding to different driving behavior information.

[0044] Specifically, for the drowsiness behavior in the driving risk comparison table, the preset number of warnings corresponding to the drowsiness can be set to 2, that is, when the number of times the driver drowses within the first preset time period is greater than or equal to 2, a warning prompt is issued. For example, assuming that within the time window, the driver dozes off once, and the driving risk score is determined to be 1 according to the driving risk comparison table, whether to issue a warning prompt at this time is no, and no warning prompt is issued; and when the driver dozes off twice, and the driving risk score is determined to be 5 according to the driving risk comparison table, whether to issue a warning prompt at this time is yes, and a warning prompt is issued. Among them, Table 1 is a driving style comparison table, which can be specifically:

[0045] Table 1 Driving risk comparison table

[0046] Driving Driving risk score Is an early warning issued? Dozed off 1 time 1 no Dozed off 2 times 5 hair Dozing off 3 times or more 10 hair

[0047] In some embodiments, the disclosed embodiments can filter the video frames corresponding to the driver at different times according to preset filtering rules to obtain target video frames; and determine the current driving risk information corresponding to the first preset duration based on the target video frames.

[0048] Specifically, there are multiple corresponding video frames at different moments within the first preset time length. If each video frame is analyzed, a large amount of calculation needs to be designed, which is costly. In addition, there may be useless video frames among the multiple video frames. Therefore, the multiple video frames can be preliminarily screened according to preset screening rules to obtain one or more target video frames, and then the driving risk score is determined according to the target video frames.

[0049] For example, when screening video frames of dozing behavior, the preset screening rule may be to screen out video frames where dozing behavior occurs, determine the video frames where dozing behavior occurs as target video frames, and then determine the number of times the driver's dozing behavior occurs based on the target video frames. It should be understood that dozing behavior is a series of actions that may occur in a period of time, so multiple video frames may correspond to one dozing behavior.

[0050] In some embodiments, the embodiments of the present disclosure can also obtain historical driving risk information, and when the current driving risk information does not meet the preset conditions, determine the target driving risk information based on the historical driving risk information and the current driving risk information; when the target driving risk information meets the preset conditions, issue a warning prompt.

[0051] Specifically, the duration of the dozing behavior may exceed the first preset duration corresponding to the time window. At this time, it is difficult to accurately characterize the driver's status by only analyzing the driving risk score corresponding to a single time window. Therefore, it is necessary to analyze multiple time windows within the duration of the dozing behavior and add up the driving risk information generated by multiple time windows to accurately analyze the driver's status. For example, the driver has dozing behavior in multiple time windows. At this time, the number of dozing behaviors in multiple time windows can be accumulated to obtain the total number of dozing in multiple time windows, and then the target driving risk information can be determined based on the total number of times.

[0052] In actual application, in the first time window, it is assumed that the number of times the driver dozes off is 1. At this time, the driving risk score is determined to be 1 by comparing with the driving risk comparison table 1. Since the driving risk score corresponding to the first time window is less than the preset warning value, no warning prompt is issued. At the same time, the number of dozes off and the corresponding driving risk score corresponding to the first time window can be recorded. However, as the vehicle travels, dozing off may still occur. Therefore, after the first time window, the abnormal moment corresponding to the next doze off can be used as the starting point to open the second time window, and the number of times the driver dozes off can be obtained in the second time window.

[0053] Specifically, since the first time window and the second time window are adjacent time windows, the dozing behavior generated in the adjacent time windows is a continuous behavior. At this time, the number of dozing behaviors in the first time window and the number of dozing behaviors in the second time window can be accumulated to obtain the number of dozing behaviors in a period of time. At this time, through accumulation, it is found that the total number of dozing behaviors in the first time window and the second time window is 2 times. The driving risk score is determined to be 5 by comparing with the driving risk comparison table 1. At this time, according to Table 1, when the dozing behavior is 2 times, whether to issue a warning prompt is yes. At this time, a warning prompt needs to be issued. Based on this, it can be avoided that multiple prompts affect the driver's mood, thereby affecting the driver's driving.

[0054] For example, if the driver has dozed off once in the first time window, and if the driver dozed off again in the second time window, then the dozing off behavior corresponding to the first time window and the second time window has occurred 2 times in total. Combining Table 1, it can be determined that the driving risk score is 5. At this time, according to Table 1, when the dozing off behavior occurs 2 times, whether to issue a warning prompt is yes, and issue a warning prompt. At the same time, the second time window can be closed after the warning prompt is issued, and the historical driving risk information can be cleared at the same time. This can avoid the continuous issuance of warning prompts and avoid affecting the driver's driving process.

[0055] It should be understood that the embodiments of the present disclosure only take dozing behavior as an example. In real life, the driver may have multiple behaviors occurring at the same time. For example, dozing and rubbing eyes occur at the same time. At this time, both dozing and rubbing eyes will cause driving risks during driving. Therefore, when multiple behaviors occur at the same time, as long as one driving behavior reaches the corresponding preset warning times, an early warning prompt will be issued.

[0056] In some embodiments, the embodiments of the present disclosure may also determine a driving risk score based on the driving risk information, and issue a warning prompt when the driving risk score is greater than or equal to a preset warning value.

[0057] Specifically, the driving risk information can be scored so that the probability of risk occurrence can be more clearly characterized. That is, the greater the driving risk score, the greater the probability of an accident occurring during driving, and the smaller the driving risk score, the smaller the probability of an accident occurring during driving. At this time, a preset warning value can be set. When the driving risk score is greater than the preset warning value, an early warning prompt is issued.

[0058] For example, as shown in Table 1, the driving risk scores corresponding to different numbers of dozing off can be set in advance, and then the driving risk score can be determined directly from Table 1 according to the number of dozing off. Since the preset number of warnings corresponding to dozing off in Table 1 is 2, and the corresponding preset warning value is 5, at this time, when the driving risk score is determined to be greater than or equal to 5 according to Table 1, a warning prompt is issued.

[0059] In some embodiments, the embodiments of the present disclosure may also stop issuing warning prompts when it is detected that the number of warning prompts reaches a preset number within a second preset time period, wherein the second preset time period is greater than the first preset time period.

[0060] Specifically, the second preset duration can be the duration of the drowsiness behavior. At this time, if the warning prompts are issued multiple times within a time period, it may affect the driver's driving mood. For example, multiple warning prompts cause the driver to become irritable. At this time, the driver may turn off the device corresponding to the warning prompt, resulting in the inability to issue warning prompts to the driver subsequently. Therefore, a preset number of times can be set. When the warning prompt reaches the preset number of times, the warning prompt will stop being issued, thereby avoiding affecting the driver's driving process.

[0061] In some embodiments, the disclosed embodiments may also clear historical driving risk data after issuing a warning prompt.

[0062] Specifically, when a warning prompt is issued, it can be assumed that the driver has been prompted that the current driving state is abnormal and needs to be adjusted. At this time, the historical driving risk data can be cleared, thereby reducing data storage costs, speeding up data processing efficiency, and avoiding historical data.

[0063] In some embodiments, the driving warning method of the embodiment of the present disclosure can be applied to a driving warning system. Specifically, during the driving process of the target vehicle, the current driving data of the target vehicle and the current video data of the driver can be obtained through the vehicle-mounted terminal, wherein the current video data is the video data generated when the driver is monitored by an image sensor.

[0064] In practical applications, the current driving data and the current video data can be pushed to a message queue (MQ), and the corresponding algorithm program in the driving warning system will extract the current driving data and the current video data from MQ and then process the data.

[0065] Specifically, since the current driving data includes the current driving speed and current driving position of the vehicle, which can be directly obtained through the corresponding sensor equipment, the embodiment of the present disclosure mainly processes the current video data. First, it is necessary to extract the current video data from MQ, and convert the unstructured data corresponding to the current video data into structured data, and push the converted structured data to MQ. Among them, the data conversion process is mainly to facilitate the subsequent system to process the data.

[0066] In some embodiments, the disclosed embodiments first need to preprocess the processed structured data and the original current video data, wherein the preprocessing includes data screening and data cleaning, wherein the data screening mainly removes video frames with occlusions and video frames without driver information, and the data cleaning mainly removes duplicate data, fills in missing values, processes abnormal values, and converts data formats, etc., based on which the accuracy and reliability of the data can be improved. Then, the preprocessed data can be further identified, invalid warnings can be removed, and real, effective and timely data can be screened out, and driving behavior information can be determined based on the data, and the driving behavior information can be pushed back to MQ.

[0067] For example, when the driving behavior information contains a driver dozing off behavior, the driving behavior information may be "at 12:00 am, the driver dozed off once", and the driving behavior information needs to be pushed back to the MQ after it is generated. It should be understood that the driving behavior information may also include weather information corresponding to the current driving location, and in this case, the driving behavior information may be "at 12:00 am, the weather was sunny, and the driver dozed off once".

[0068] In some embodiments, the disclosed embodiments can also monitor the entire driving process. When the driving warning system obtains a new message pushed from MQ, a time window with a time value of T will be opened. The time window will record the driving behavior information during this period, wherein the time window plays a role in summarizing the driving behavior information during this period. It should be understood that the specific value of T can be determined according to actual conditions.

[0069] Figure 2 A schematic diagram of a driving timeline flow chart provided by an exemplary embodiment of the present disclosure. Figure 2 As shown, when the system detects a new message in MQ "12:00 am, sunny, the driver dozed off once", it determines that 12:00 am is the first abnormal moment t1. At this time, starting from t1, the first time window 201 is opened, and the current driving data of the target vehicle and the current video data of the driver are obtained in real time within the first time window 201.

[0070] However, as the vehicle is driving, dozing off may still occur. Therefore, after experiencing the first time window 201, the second time window 202 can be opened with the abnormal moment t2 corresponding to the next dozing off as the starting point, and the number of times the driver dozed off can be determined within the second time window 202. When the number of times the driver dozed off within the second time window 202 is also 1, at this time, it is necessary to determine the final driving risk score based on the number of times the driver dozed off within the first time window 201 and the second time window 202, and determine whether to issue a warning prompt based on the final driving risk score.

[0071] In some embodiments, within the time window, if the driving risk score within the time window does not reach the preset warning value, the current driving risk score will be accumulated with the historical driving risk score. At the same time, if the driving risk score corresponding to the new message has reached the preset warning value, an early warning prompt will be issued immediately and the time window will be ended in advance.

[0072] For example, assuming that the number of dozes corresponding to the first time window is 1, on this basis, as long as the number of dozes in the second time window is 1, the sum of the number of dozes in the first time window and the second time window reaches 2, that is, the preset conditions are met, and an early warning prompt can be issued. At this time, the time window can be closed while the early warning prompt is issued. It should be understood that when the driving behavior information and driving risk information corresponding to a certain moment in the first time window meet the preset conditions, an early warning prompt can be directly issued and the first time window can be closed.

[0073] Figure 3 A schematic diagram of a new message processing flow provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, specifically including:

[0074] S301, start.

[0075] S302, determine whether there is a new message in MQ. If there is a new message in MQ, execute S303.

[0076] S303, obtain new messages from MQ.

[0077] Specifically, when a new message is detected in the MQ, the new message is obtained, wherein the new message may be a message containing driving behavior information.

[0078] S304, whether there is a time window.

[0079] Specifically, if there is a time window, S305 is executed, and if there is no time window, S307 is executed.

[0080] S305, whether there is a historical time window.

[0081] Specifically, if there is a historical time window, S306 is executed.

[0082] S306, determining time creation data according to historical time window data and current time window data.

[0083] Specifically, the historical time window data corresponding to the historical time window can be obtained respectively, that is, the historical driving data and the historical video data of the driver generated by the target vehicle in the historical time window, and the current driving data and the current video data of the driver generated in this time window, and then the time window data can be determined based on the historical time window data and the current time window data.

[0084] S307: Determine driving behavior information according to the time window data.

[0085] Specifically, the driving behavior information is determined based on the driving data and video data included in the time window data. For example, the number of times the driver dozed off or drank water can be determined based on the video data.

[0086] S308: Determine whether the driving behavior information meets a preset condition.

[0087] Specifically, when the driving behavior information meets the preset condition, S309 is executed, and when the driving behavior information does not meet the preset condition, S302 is executed.

[0088] S309, issue a warning.

[0089] Based on this, the disclosed embodiments can judge the driver's driving behavior based on automatic real-time monitoring of the time window, process structured data and unstructured data at the same time, and judge whether the driver's driving process is dangerous in multiple dimensions, thereby issuing early warning instructions, improving the accuracy of monitoring and early warning, and effectively ensuring the driver's safety during driving by promptly warning the driver, avoiding serious loss of life and property.

[0090] The above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of the server. It is understandable that, in order to implement the above functions, the server includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0091] The embodiments of the present disclosure may divide the server into functional units according to the above method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one management module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0092] In the case of dividing each functional module according to each function, an exemplary embodiment of the present disclosure provides a driving warning device, which may be a server or a chip applied to a server. Figure 4 The structure diagram of a driving warning device provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the driving warning device 400 includes:

[0093] The acquisition module 401 is used to acquire the current driving data of the target vehicle and the current video data of the driver during the driving process of the target vehicle;

[0094] A generating module 402, configured to determine driving behavior information according to the current driving data and the current video data;

[0095] The prompt module 403 is used to determine the current driving risk information according to the driving behavior information, and issue a warning prompt when the current driving risk information meets a preset condition.

[0096] In an optional manner, the prompt module 403 is also used to determine the abnormal moment according to the driving behavior information when the driving behavior information is abnormal driving behavior information; taking the abnormal moment as the starting point, obtain the video frames corresponding to the driver at different moments within a first preset time length, and the driving data corresponding to the different moments; determine the current driving risk information within the first preset time length according to the video frames and the driving data.

[0097] In an optional manner, the prompt module 403 is also used to filter the video frames corresponding to the driver at different times according to preset filtering rules to obtain target video frames; and determine the current driving risk information within the first preset time length based on the target video frames.

[0098] In an optional manner, the prompt module 403 is also used to obtain historical driving risk information, and when the current driving risk information does not meet the preset conditions, determine the target driving risk information based on the historical driving risk information and the current driving risk information; when the target driving risk information meets the preset conditions, issue a warning prompt.

[0099] In an optional manner, the prompt module 403 is further configured to determine a driving risk score according to the driving risk information, and to issue a warning prompt when the driving risk score is greater than or equal to a preset warning value.

[0100] In an optional manner, the driving warning device 400 also includes a stop module 404, which is used to stop issuing warning prompts when it is detected that the number of warning prompts reaches a preset number within a second preset time period, wherein the second preset time period is greater than the first preset time period.

[0101] An embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; a memory for storing instructions executable by at least one processor; wherein the at least one processor is used to execute instructions to implement the steps of the above method disclosed in the embodiment of the present disclosure.

[0102] Figure 5 The structure diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 5 As shown, the electronic device 500 includes at least one processor 501 and a memory 502 coupled to the processor 501 , and the processor 501 can execute corresponding steps in the above method disclosed in the embodiment of the present disclosure.

[0103] The processor 501 may also be referred to as a central processing unit (CPU), which may be an integrated circuit chip having signal processing capabilities. Each step in the method disclosed in the embodiment of the present disclosure may be completed by an integrated logic circuit of hardware in the processor 501 or by instructions in the form of software. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present disclosure may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a memory 502, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The processor 501 reads the information in the memory 502 and completes the steps of the method in combination with its hardware.

[0104] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, they can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, for example, Figure 6 The computer system 600 shown installs the programs constituting the software, and when the various programs are installed, the computer system can execute various functions, including the functions described above. Figure 6 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure.

[0105] Computer system 600 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0106] like Figure 6 As shown, the computer system 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0107] A plurality of components in the computer system 600 are connected to the I / O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 can be any type of device capable of inputting information to the computer system 600, and the input unit 606 can receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 607 can be any type of device capable of presenting information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 608 can include but is not limited to a disk, an optical disk. The communication unit 609 allows the computer system 600 to exchange information / data with other devices over a network such as the Internet, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, for example, a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0108] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the above methods disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 602 and / or a communication unit 609. In some embodiments, the computing unit 601 may be configured to perform the above methods disclosed in the embodiments of the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0109] The embodiment of the present disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.

[0110] The computer-readable storage medium in the disclosed embodiments may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specifically, the computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0111] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0112] The embodiments of the present disclosure also provide a computer program product, including a computer program, wherein the computer program implements the above method disclosed in the embodiments of the present disclosure when executed by a processor.

[0113] In embodiments of the present disclosure, computer program codes for performing the operations of the present disclosure may be written in one or more programming languages ​​or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The modules, components or units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the modules, components or units do not, in some cases, limit the modules, components or units themselves.

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] The above descriptions are only some embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, a technical solution formed by replacing the above features with the technical features with similar functions disclosed in the present disclosure (but not limited to).

[0118] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A driving warning method, characterized in that: include: During the driving process of the target vehicle, obtaining current driving data of the target vehicle and current video data of the driver; determining driving behavior information according to the current driving data and the current video data; The current driving risk information is determined according to the driving behavior information, and a warning prompt is issued when the current driving risk information meets a preset condition.

2. The method according to claim 1, characterized in that The determining the current driving risk information according to the driving behavior information includes: In a case where the driving behavior information is abnormal driving behavior information, determining an abnormal time according to the driving behavior information; Taking the abnormal moment as the starting point, obtaining video frames corresponding to the driver at different moments within a first preset time period, and driving data corresponding to different moments; Current driving risk information within the first preset time period is determined according to the video frame and the driving data.

3. The method according to claim 2, characterized in that The method further comprises: Filter the video frames corresponding to the driver at different times according to the preset filtering rules to obtain the target video frame; The current driving risk information within the first preset time length is determined according to the target video frame.

4. The method according to claim 1, characterized in that The method further comprises: Acquire historical driving risk information, and if the current driving risk information does not meet a preset condition, determine target driving risk information according to the historical driving risk information and the current driving risk information; When the target driving risk information meets the preset conditions, a warning prompt is issued.

5. The method according to claim 4, characterized in that Determining the current driving risk information according to the driving behavior information, and issuing a warning prompt when the current driving risk information meets a preset condition, includes: A driving risk score is determined based on the driving risk information, and a warning prompt is issued when the driving risk score is greater than or equal to a preset warning value.

6. The method according to claim 1, characterized in that The method further comprises: When it is detected that the number of warning prompts reaches a preset number within a second preset time period, the warning prompt is stopped, wherein the second preset time period is greater than the first preset time period.

7. A driving warning device, characterized in that: include: An acquisition module, used for acquiring current driving data of the target vehicle and current video data of the driver during the driving process of the target vehicle; A generating module, configured to determine driving behavior information according to the current driving data and the current video data; The prompt module is used to determine the current driving risk information according to the driving behavior information, and issue a warning prompt when the current driving risk information meets the preset conditions.

8. An electronic device, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is used to execute the instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images