Vehicle Alarm Control Method, Device, Equipment and Machine-readable Storage Medium

Abstract

The present application provides a vehicle alarm control method, device, equipment and machine-readable storage medium. The method includes: detecting a first type of alarm event for the image data provided by a first type of camera according to the currently used detection conditions of the first type of alarm event; adjusting the currently used detection conditions of a second type of alarm event according to the detection result of the first type of alarm event. This method can improve the flexibility and rationality of determining the driving safety state of a vehicle.

Description

Technical Field

[0001] The present application relates to the fields of computers and automotive electronics, and in particular to a vehicle alarm control method, device, equipment, and machine-readable storage medium. Background Art

[0002] Advanced driver assistance systems use a variety of sensors installed on the vehicle to sense the surrounding environment at any time while the vehicle is driving, collect data, and identify static and dynamic objects, etc., so that the driver can be aware of possible dangers in advance, effectively increasing the comfort and safety of vehicle driving.

[0003] Currently, the technical solutions used by advanced driver assistance systems on the market are as follows: an embedded device is deployed in a motor vehicle. This embedded device can be connected to two cameras. One camera (ADAS (Advanced Driving Assistance System) camera) is installed in the center of the windshield, with the imaging direction facing directly in front of the cockpit, and is used to detect ADAS alarms; the other camera (DMS (Driver Monitoring System) camera) is installed inside the cockpit, facing the driver's face, and is used to detect DMS alarms. The two cameras provide video data to the embedded controller, which runs a convolutional neural network to perceive the video data and make alarm judgments. When the alarm is triggered, operations such as sound and light alarms, image capture, and short video recording are immediately performed; finally, the data is transmitted to the corresponding platform via the mobile network.

[0004] Practice has found that in traditional solutions, the alarm event detection conditions are all fixed by manual settings and cannot be adjusted according to actual conditions, resulting in poor flexibility and rationality. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a vehicle alarm control method, apparatus, device, and machine-readable storage medium to at least solve the problem of poor flexibility and rationality in setting alarm event detection conditions in traditional solutions.

[0006] Specifically, this application is implemented through the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a vehicle alarm control method is provided, comprising:

[0008] Performing first-type alarm event detection on image data provided by the first-type camera according to the currently used first-type alarm event detection condition;

[0009] Adjust the detection conditions of the second type of alarm event currently in use according to the detection result of the first type of alarm event;

[0010] Wherein, when the first type of alarm event is an ADAS alarm event, the first type of camera is an ADAS camera, and the second type of alarm event is a DMS alarm event; or,

[0011] When the first type of alarm event is a DMS alarm event, the first type of camera is a DMS camera, and the second type of alarm event is an ADAS alarm event.

[0012] According to a second aspect of the embodiments of the present application, there is provided a vehicle alarm control device, including:

[0013] A detection unit for detecting the first type of alarm event on the image data provided by the first type of camera according to the detection conditions of the first type of alarm event currently in use;

[0014] An adjustment unit for adjusting the detection conditions of the second type of alarm event currently in use according to the detection result of the first type of alarm event;

[0015] Wherein, when the first type of alarm event is an ADAS alarm event, the first type of camera is an ADAS camera, and the second type of alarm event is a DMS alarm event; or,

[0016] When the first type of alarm event is a DMS alarm event, the first type of camera is a DMS camera, and the second type of alarm event is an ADAS alarm event.

[0017] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including a processor and a machine-readable storage medium, the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the vehicle alarm control method provided in the first aspect above.

[0018] According to a fourth aspect of the embodiments of the present application, there is provided a machine-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the vehicle alarm control method provided in the first aspect.

[0019] According to a fifth aspect of the embodiments of the present application, there is provided a computer program product, which stores a computer program, and when the processor executes the computer program, it causes the processor to execute the vehicle alarm control method provided in the first aspect.

[0020] The vehicle alarm control method according to the embodiments of the present application takes into account the relevance of the impact of ADAS alarm events and DMS alarm events on vehicle driving safety, fuses the detection of ADAS alarm events and DMS alarm detection, improves the flexibility and rationality of ADAS alarm event detection and DMS alarm detection, and thus can improve the flexibility and rationality of vehicle driving safety status determination. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic flowchart of a vehicle alarm control method shown in an exemplary embodiment of the present application;

[0022] Figure 2 It is a schematic architecture diagram for implementing vehicle alarm control provided by the embodiments of the present application;

[0023] Figure 3 It is a schematic diagram of a confidence - false alarm rate curve provided by the embodiments of the present application;

[0024] Figure 4 It is a schematic flowchart of another vehicle alarm control method shown in an exemplary embodiment of the present application;

[0025] Figure 5 It is a schematic structural diagram of a vehicle alarm control device shown in an exemplary embodiment of the present application;

[0026] Figure 6 It is a schematic structural diagram of another vehicle alarm control device shown in another exemplary embodiment of the present application;

[0027] Figure 7 It is a schematic hardware structure diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0030] To enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, some technical terms related to the embodiments of the present application will be briefly described below.

[0031] 1. Convolutional Neural Network: A network structure commonly used in object detection algorithms based on deep learning. This network structure is suitable for object detection.

[0032] 2. ADAS Alarm: Refers to alarms related to the driving posture of a vehicle, mainly including but not limited to: FCW (Forward Collision Warning), LDW (Lane Departure Warning), PCW (Pedestrian Collision Warning), HMW (Headway Monitoring Warning), TSR (Traffic Sign Recognition) and so on.

[0033] 3. DMS Alarm: Refers to alarms related to abnormal driver behavior states (or called driving states). Driving states mainly include but not limited to: yawning, closing eyes, smoking, making phone calls, not wearing seat belts, wearing sunglasses, distracted driving and other abnormal states.

[0034] 4. Sensitivity: Characterizes the ease of alarm triggering. The higher the value of sensitivity, the easier the alarm is to be triggered.

[0035] 5. Confidence: The object detection confidence threshold of the deep learning algorithm based on the convolutional neural network. That is, when the confidence of object detection is greater than this value, the object is considered to be correctly detected.

[0036] 6. Alarm Interval: The minimum time interval between two identical alarms. For example, if the alarm interval is 2 minutes, if a yawning alarm is triggered at a certain moment, the next yawning alarm will be triggered at least 2 minutes later.

[0037] 7. Platform: Generally refers to the front-end network platform, which can receive the alarm information uploaded by the embedded device.

[0038] 8. Alarm Linkage: Refers to a series of alarm processing related operations after the alarm is successfully triggered, including operations such as screenshot capture, short video capture, audible and visual alarms, and platform upload.

[0039] In order to make the above objects, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] Please refer to Figure 1, which is a schematic flowchart of a vehicle alarm control method provided by an embodiment of the present application. This method can be executed by an electronic device (also referred to as an in-vehicle terminal or in-vehicle device) deployed on the vehicle, such as an embedded device deployed on the vehicle. The electronic device is communicatively connected to an ADAS camera and a DMS camera respectively. Exemplarily, the ADAS camera can be installed at the center of the vehicle's windshield, with the imaging direction facing directly in front of the cockpit, used to detect ADAS alarms; the DMS camera can be installed inside the cockpit, facing directly at the driver's face, used to detect DMS alarms; the electronic device detects and identifies the image data provided by the ADAS camera and the DMS camera by running a target detection algorithm, and makes an alarm judgment.

[0041] As Figure 1 shown, the vehicle alarm control method may include the following steps:

[0042] Step S100, detect a first type of alarm event for the image data provided by the first type of camera according to the currently used detection conditions for the first type of alarm event.

[0043] Exemplarily, the first type of alarm event may include an ADAS alarm event or a DMS alarm event, that is, both the ADAS alarm event detection and the DMS alarm event detection can be processed using the solution provided by the embodiment of the present application.

[0044] When the first type of alarm event is an ADAS alarm event, the first type of camera is the ADAS camera.

[0045] Or, when the first type of alarm event is a DMS alarm event, the first type of camera is the DMS camera.

[0046] Exemplarily, the ADAS alarm event includes alarm events related to the vehicle driving posture, such as a front vehicle collision alarm event, a lane departure alarm event, a pedestrian collision alarm event, a too-close vehicle distance alarm event, a traffic sign alarm event, etc.

[0047] Exemplarily, the DMS alarm event includes alarm events related to the driver's driving state, such as a yawning alarm event, a closing eyes alarm event, a smoking alarm event, a making a call alarm event, a not wearing seat belt alarm event, a wearing sunglasses alarm event, a distracted driving alarm event, etc.

[0048] In the embodiment of the present application, a first type of alarm event can be detected for the image data provided by the first type of camera according to the currently used detection conditions for the first type of alarm event.

[0049] Exemplarily, the image data provided by the first type of camera may include video frame data, captured image data, etc.

[0050] Exemplarily, when the first type of alarm event is an ADAS alarm event, the video data provided by the ADAS camera can be detected for ADAS alarm events according to the ADAS alarm event detection conditions.

[0051] Exemplarily, when the first type of alarm event is a DMS alarm event, the video data provided by the DMS camera can be detected for DMS alarm events according to the DMS alarm event detection conditions.

[0052] Step S110: Adjust the detection conditions of the second type of alarm event currently in use according to the detection result of the first type of alarm event.

[0053] In the embodiments of the present application, the detection result of the alarm event may include detecting an alarm event or not detecting an alarm event.

[0054] Correspondingly, the detection result of the first type of alarm event may include detecting the first type of alarm event or not detecting the first type of alarm event.

[0055] Exemplarily, for the case of detecting an alarm event, the detection result of the alarm event may further include the frequency of detecting the alarm event, such as the number of times of detecting the alarm event within a preset unit time.

[0056] For the case of not detecting an alarm event, the detection result of the alarm event may further include the duration of continuously not detecting the alarm event.

[0057] In the embodiments of the present application, it is considered that the detection result of the ADAS alarm event may affect the driving safety in the case of the DMS alarm event occurring, and the detection result of the DMS alarm event may also affect the driving safety in the case of the ADAS alarm event occurring.

[0058] For example, assume that an ADAS alarm event is detected at a certain moment, such as a forward collision warning (FCW) event. In this case, if a DMS alarm event occurs, such as a distracted driving alarm event, the impact on the driving safety of the vehicle will increase. That is, in this case, when the driver is distracted, the probability of a vehicle accident will increase.

[0059] Another example, assume that a DMS alarm event is detected at a certain moment, such as a yawning (indicating that the driver may be fatigued) alarm event. In this case, if an ADAS alarm event occurs, such as a headway monitoring warning (HMW) event, the impact on the driving safety of the vehicle will increase. That is, in this case, when the headway is too close, the probability of a vehicle accident will increase.

[0060] Based on the above considerations, in the embodiments of the present application, the detection of ADAS alarm events and the detection of DMS alarms can be fused, and according to the detection result of one type of alarm event, the detection of the other type of alarm event can be adjusted.

[0061] Correspondingly, in the embodiments of the present application, the detection conditions of the second type of alarm event currently in use can be adjusted according to the detection result of the first type of alarm event.

[0062] Exemplarily, when the first type of alarm event is an ADAS alarm event, the second type of alarm event is a DMS alarm event.

[0063] Exemplarily, when the first type of alarm event is a DMS alarm event, the second type of alarm event is an ADAS alarm event.

[0064] It can be seen that in Figure 1 In the shown method flow, considering the relevance of the impact of ADAS alarm events and DMS alarm events on vehicle driving safety, the detection of ADAS alarm events and the detection of DMS alarms are fused to improve the flexibility and rationality of the detection of ADAS alarm events and DMS alarms. Furthermore, the flexibility and rationality of determining the vehicle driving safety state can be improved.

[0065] For the convenience of description and understanding, in the following, an example is given where the first type of alarm event is an ADAS alarm event and the second type of alarm event is a DMS alarm event. The same applies to the case where the first type of alarm event is a DMS alarm event and the second type of alarm event is an ADAS alarm event.

[0066] In some embodiments, the alarm event detection conditions include: the confidence level of alarm event detection and alarm configuration parameters. The alarm configuration parameters include sensitivity and / or alarm interval. The sensitivity is used to characterize the ease of alarm triggering, and the higher the sensitivity, the easier the alarm is to trigger.

[0067] Exemplarily, in order to control the false alarm rate and / or missed alarm rate of the alarm and improve the controllability of the alarm event detection, the alarm event detection conditions may include, but are not limited to, the confidence level of alarm event detection and alarm configuration parameters.

[0068] Exemplarily, the alarm configuration parameters include sensitivity and / or alarm interval.

[0069] Among them, the sensitivity is used to characterize the ease of alarm triggering, and the higher the sensitivity, the easier the alarm is to trigger.

[0070] For example, increasing the confidence level of alarm event detection can reduce the false alarm rate of alarm events, but the missed alarm rate may also increase accordingly; decreasing the confidence level of alarm event detection can reduce the missed alarm rate of alarm events, but the false alarm rate may also increase accordingly.

[0071] Increasing the sensitivity can reduce the missed alarm rate of alarm events, but the false alarm rate may also increase accordingly; decreasing the sensitivity can reduce the false alarm rate of alarm events, but the missed alarm rate may also increase accordingly.

[0072] In one example, the alarm event detection confidence level and the alarm configuration parameters include the data input by the user through the configuration interface of the alarm event detection conditions.

[0073] Exemplarily, the user can set the alarm event detection confidence level and the alarm configuration parameters through the configuration interface of the alarm event detection conditions.

[0074] As an example, the configuration interface may include the interface displayed on the vehicle-mounted terminal, for example, the interface displayed on the vehicle-mounted central control screen, so that the setting of the alarm event detection conditions can be realized without the need to rely on external devices.

[0075] As another example, the configuration interface may include the interface displayed on the user terminal, for example, the interface displayed on the user's mobile terminal device (such as a smart phone or a tablet computer). The user can access the vehicle-mounted electronic device through the user terminal to realize the setting of the alarm event detection conditions and improve the operation flexibility of setting the alarm event detection conditions.

[0076] In some embodiments, the second type of alarm event detection conditions may include the detection conditions respectively corresponding to the second type of working mode at the first level and the second type of working mode at the second level;

[0077] Step S110, adjusting the currently used second type of alarm event detection conditions according to the detection result of the first type of alarm event, including:

[0078] When it is detected that the first type of event alarm event occurs and the level of the current second type of working mode is at the first level, adjusting the level of the current second type of working mode to the second level;

[0079] Among them, for the second type of working mode, the alarm event detection confidence level at the first level is higher than that at the second level; and / or,

[0080] For the second type of working mode, the alarm interval at the first level is higher than that at the second level, and / or the sensitivity at the first level is lower than that at the second level.

[0081] Exemplarily, for alarm event detection (including first alarm event detection or second alarm event detection), multiple different levels of working modes can be preset. The alarm event detection conditions under different levels of working modes are different. Thus, when it is necessary to adjust the alarm event detection conditions, the adjustment can be achieved by adjusting the level of the working mode, simplifying the operation of adjusting the alarm event detection conditions and improving the adjustment efficiency of the alarm event detection conditions.

[0082] Exemplarily, the device working mode corresponding to the first type of alarm event detection can be called the first type of working mode, and the device working mode corresponding to the second type of alarm event detection can be called the second type of working mode.

[0083] Exemplarily, the second type of alarm event detection conditions can include the detection conditions respectively corresponding to two levels of the second type of working mode (which can be respectively called the first-level second type of working mode and the second-level second type of working mode).

[0084] Exemplarily, the first type of alarm event detection conditions can include the detection conditions respectively corresponding to two levels of the first type of working mode (which can be respectively called the first-level first type of working mode and the second-level first type of working mode).

[0085] Exemplarily, for the first type of working mode or the second type of working mode, the alarm event detection confidence level at the first level is higher than that at the second level; and / or, for the second type of working mode, the alarm interval at the first level is higher than that at the second level, and / or, the sensitivity at the first level is lower than that at the second level.

[0086] Exemplarily, in the case of detecting an ADAS alarm event, for example, detecting any one of the FCW, LDW, PCW, HMW, and TSR alarm events, the level of the current second type of working mode can be determined. When the level of the current second type of working mode is the first level, the level of the current second type of working mode can be adjusted to the second level to reduce the difficulty of meeting the DMS alarm event detection conditions, that is, making it easier for the electronic device to trigger an alarm for the DMS alarm event, so as to improve the detection rate of the DMS alarm event, reduce the missed detection rate of the DMS alarm event, and improve the driving safety of the vehicle.

[0087] It should be noted that in the case of detecting an ADAS alarm event, if the level of the current second type of working mode is the second level, the level of the second type of working mode can be maintained at the second level.

[0088] In addition, in the embodiments of the present application, when an ADAS alarm event is detected, the detection rate of DMS alarm event detection can also be directly increased and the false negative rate can be reduced by adjusting the DMS alarm event detection conditions, without the need to adjust the level of the second type of working mode.

[0089] For example, the detection confidence of the alarm events included in the currently used DMS alarm event detection conditions can be reduced, and / or the sensitivity can be increased and / or the alarm interval can be reduced.

[0090] Exemplarily, different DMS alarm events can be set with different detection confidences of alarm events. When the DMS alarm event detection conditions need to be adjusted, different DMS alarm events can be adjusted separately.

[0091] For example, when it is necessary to reduce the detection confidence of the alarm events included in the currently used DMS alarm event detection conditions, the detection confidences of DMS alarm event detections such as yawning event detection, eyes-closed event detection, random inspection event detection,..., distracted driving event detection can be reduced separately.

[0092] Similarly, different DMS alarm events can be set with different sensitivities and / or alarm intervals. When the sensitivity and / or alarm interval need to be adjusted, the sensitivities and / or alarm intervals of different DMS alarm events can be adjusted separately to improve the flexibility of adjusting the alarm event detection conditions and refine the granularity of adjusting the alarm event detection conditions.

[0093] In one example, the vehicle alarm control method provided by the embodiments of the present application may further include:

[0094] When the level of the current second type of working mode is the second level and no second type of alarm event is detected within a preset detection period, the level of the current second type of working mode is adjusted to the first level.

[0095] Exemplarily, considering that when the level of the second type of working mode is the second level, the false alarm rate of DMS alarm events is relatively high. To avoid an excessive false alarm rate of DMS alarms when the vehicle is driving relatively safely, when the level of the current second type of working mode is the second level and no second type of alarm event is detected within a preset detection period, the level of the current second type of working mode can be adjusted to the first level to reduce the false alarm rate of DMS alarms and save device resources.

[0096] It should be noted that when the level of the current second type of working mode is the second level, if a DMS alarm event is detected within the preset detection period, the preset detection period can be reset and the level of the second type of working mode can be maintained at the second level.

[0097] For example, assuming that the preset detection period is 5 minutes, when the level of the second type of working mode is adjusted to the second level, a timer with a timing duration of 5 minutes can be started. If no DMS alarm event is detected before the timer times out, the level of the second type of working mode can be restored to the first level; if a DMS alarm event is detected before the timer times out, the timing duration of the timer can be reset.

[0098] In one example, when a first type of alarm event is detected, it may further include:

[0099] Associating and storing the first type of alarm event and the image data collected by the first type of camera.

[0100] Exemplarily, when an ADAS alarm event is detected, the ADAS alarm event and the image data collected by the ADAS camera can also be associated and stored, so that in the case of a vehicle driving accident, the driving conditions of the vehicle before and after the accident can be better analyzed, and at the same time, the effect of retaining evidence can also be achieved.

[0101] Exemplarily, considering that the level of the working mode for alarm event detection is the second level, which usually means a higher probability of a vehicle driving accident. To better ensure vehicle driving safety and to better analyze the driving conditions of the vehicle before and after a vehicle driving accident, when the level of the working mode for alarm event detection is the second level, the frequency of alarm linkage can be increased.

[0102] Exemplarily, increasing the frequency of alarm linkage can include: increasing the frequency of capturing images and / or short video clips to save more image data during vehicle driving; and / or, increasing the frequency of audible and visual alarms to better remind the driver to drive safely and improve vehicle driving safety.

[0103] In some embodiments, for the second type of working mode, the alarm event detection confidence at the first level is the first alarm event detection confidence, and the alarm event detection confidence at the second level is the second alarm event detection confidence;

[0104] Wherein, the first alarm event detection confidence is the confidence when the slope of the target confidence - false alarm rate curve converges successfully, and the second alarm event detection confidence is the confidence corresponding to the inflection point of the slope of the target confidence - false alarm rate curve;

[0105] The confidence - false alarm rate curve is used to characterize the variation relationship between the alarm event detection confidence and the false alarm rate.

[0106] Exemplarily, in order to improve the rationality of the setting of the alarm event detection confidence, the alarm event detection confidence in different levels of working modes can be set according to the variation relationship between the alarm event detection confidence and the false detection rate.

[0107] Exemplarily, according to the variation relationship between the alarm event detection confidence and the false detection rate, a curve representing this variation relationship (referred to as the target confidence-false detection rate curve in this article) can be statistically obtained, and the alarm event detection confidence in different levels of working modes can be set according to this target confidence-false detection rate curve. For example, the target confidence-false detection rate curve can have the confidence as the horizontal axis and the false detection rate as the vertical axis.

[0108] Exemplarily, considering the second type of working mode, after the level adjustment based on the detection results of the first type of alarm events, when in the first level, the probability of danger in the corresponding scenario is relatively small. By setting an appropriate alarm event detection confidence, the false alarm rate can be made lower, reducing false alarms and saving device resources; while when in the second level, the probability of danger in the corresponding scenario is relatively high. At this time, it is necessary to reduce the missed alarm rate, and the tolerance for the false alarm rate can be increased.

[0109] Correspondingly, for the second type of working mode, the confidence when the slope of the target confidence-false detection rate curve converges successfully can be set as the alarm event detection confidence in the first level (referred to as the first alarm event detection confidence in this article). In this setting, the alarm event detection confidence is relatively high and the false alarm rate is relatively low; the confidence corresponding to the inflection point of the slope of the target confidence-false detection rate curve can be set as the alarm event detection confidence in the second level (referred to as the first alarm event detection confidence in this article). In this setting, the alarm event detection confidence is relatively low and the false alarm rate is relatively high, but it can effectively reduce missed alarms and improve the driving safety of the vehicle. Its specific implementation will be described in combination with specific examples below.

[0110] In one example, the alarm event detection confidence and the false detection rate are related to the target convolutional neural network model, which is used for alarm event detection. The specific structure of the target convolutional neural network model is not specifically limited in the embodiments of the present application.

[0111] Exemplarily, in order to improve the accuracy and rationality of the alarm event detection confidence setting, the target confidence-false detection rate curve can be determined according to the convolutional neural network model applied to alarm event detection (referred to as the target convolutional neural network model in this article). For example, by using the validation set to verify the target convolutional neural network model, the variation relationship between the alarm event detection confidence and the false detection rate is obtained, and then the above-mentioned target confidence-false detection rate curve is obtained.

[0112] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of the present application, the technical solutions provided in the embodiments of the present application will be described below with reference to specific examples.

[0113] Please refer to Figure 2 , which is a schematic diagram of an architecture for implementing vehicle alarm control provided in the embodiments of the present application. As Figure 2 shown, the architecture may include an embedded controller (which may be referred to as the main controller), an ADAS camera, a DMS camera, a storage medium, an audible and visual alarm, a host computer / mobile phone APP, and a 4G / 5G communication module; where:

[0114] The ADAS camera and the DMS camera serve as data sources for alarm judgment.

[0115] The main controller is respectively connected to the ADAS camera and the DMS camera. A deep learning algorithm may be deployed on the main controller to perform target recognition and detection on the image data provided by the two cameras (the ADAS camera and the DMS camera) through a convolutional neural network, and further analyze the results to complete the judgment of alarm events.

[0116] The host computer / mobile phone APP serves as a human-machine interaction interface and can configure ADAS alarm and DMS alarm related configuration parameters into the main controller; when the main controller determines that an alarm is triggered, it can store the alarm event video recording and captured images in the storage medium; and can upload the alarm information to the platform side (such as the server side) through the 4G / 5G communication module, and the platform side can also configure the parameters to the main controller. That is, the ADAS alarm and DMS alarm related configuration parameters can be configured through human-machine interaction or through the platform side.

[0117] In the embodiments of the present application, there are two different levels of working modes for the alarm judgment mode. One is the normal mode (i.e., corresponding to the above first level), and the other is the sensitive mode (i.e., corresponding to the above second level).

[0118] In the embodiments of the present application, considering that the resources of embedded devices are limited and complex convolutional neural network related algorithms cannot be run. And when the vehicle is running, it will be affected by complex external information such as light, weather, and temperature at different times, and the algorithm often has false alarms and missed alarms in some extreme cases. If a higher confidence level for alarm event detection is used to reduce false detections and false alarms, it will lead to the inability to output the alarm in a timely manner in some cases, and the purpose of timely prompting the driver cannot be achieved.

[0119] Based on the above considerations, in the technical solution provided in the embodiments of the present application, at the level of target detection algorithm deployment, a deep learning algorithm can be adopted for target detection based on a convolutional neural network. Before actual deployment, a large amount of sample data in the case of distracted drivers can be used to train the target detection algorithm model based on the convolutional neural network (i.e., the above-mentioned target convolutional neural network model for alarm event detection). After the target detection algorithm model is fully trained, a large amount of validation set (also called test set) data is used to statistically analyze the detection effect of the target convolutional neural network model.

[0120] As Figure 3 shown, a two-dimensional relationship graph (i.e., the above-mentioned target confidence - false detection rate curve) can be plotted on a rectangular coordinate system for the false detection rates obtained using different confidence levels. Since the detection rate decreases as the detection threshold increases, in order to ensure the detection rate, the confidence level with a successfully converged slope is selected as the confidence level for the normal mode (i.e., the above-mentioned first alarm event detection confidence level), and the confidence level at the inflection point where the slope no longer decreases significantly is selected as the confidence level for the sensitive mode (i.e., the above-mentioned second alarm event detection confidence level). The confidence level for the sensitive mode is relatively low, which can ensure that as many abnormalities as possible are detected in the sensitive mode to prompt the driver; the confidence level for the normal mode is relatively high, which can reduce false detection situations when the device is operating normally.

[0121] In addition, in the embodiments of the present application, considering that a series of linkage operations after normal alarm triggering are also executed when false detection occurs, this will result in more waste of the device's storage resources and affect the user experience on the platform side. If the scheme of extending the alarm interval is adopted to reduce false alarms, then in the case where the alarm is in the suppression time (i.e., the interval between two adjacent alarm times), it will lead to the situation that the device cannot prompt the driver in time when a behavior that truly threatens vehicle driving safety is triggered, resulting in critical missed alarms.

[0122] Based on the above considerations, in the technical solution provided in the embodiments of the present application, at the level of alarm parameter configuration, there are also differences in the two levels of working modes (i.e., normal mode and sensitive mode). For example, the default alarm configuration parameters for the normal mode can be stored in the FLASH (i.e., FLASH memory) of the main controller, and each ADAS alarm event and each DMS alarm event can correspond to separate sensitivities and alarm intervals.

[0123] Exemplarily, in the normal mode, the sensitivity and alarm interval of each alarm event can be configured on the host computer / mobile phone APP (i.e., application program) and / or the platform side according to the user's usage habits, or according to the alarm monitoring requirements. Exemplarily, after the electronic device is started, it can first enter the normal mode, including the ADAS normal mode and the DMS normal mode.

[0124] Similarly, for the sensitive mode, a set of default alarm configuration parameters for the sensitive mode will also be stored in the FLASH of the main controller. Each alarm event can also have a separate sensitivity and alarm interval. The alarm interval and sensitivity of the sensitive mode can also be configured separately according to the user's usage habits or according to the alarm monitoring requirements.

[0125] Exemplarily, under normal conditions, the sensitivity in the sensitive mode is higher than that in the normal mode (i.e., it is easier to trigger an alarm in the sensitive mode), and the alarm interval in the sensitive mode is relatively shorter than that in the normal mode.

[0126] The following is an exemplary description of the operation process of the electronic device.

[0127] When the electronic device is powered on and started for the first time, it is in the normal mode. If an ADAS alarm occurs during the operation of the electronic device, it is considered that if the driver's behavior is abnormal in the next period of time, the probability of causing an accident will increase. The electronic device can switch the DMS alarm event detection confidence level and alarm configuration parameters to the corresponding parameters in the pre-set sensitive mode. In the sensitive mode, DMS alarm events are more easily detected, and the low alarm interval can ensure that the driver is prompted to drive attentively more frequently in relatively dangerous situations.

[0128] If no DMS alarm is triggered within a period of time (i.e., the above-mentioned preset detection period) after entering the DMS sensitive mode, the DMS sensitive mode can be exited and switched to the normal mode.

[0129] Similarly, if a DMS alarm occurs during the operation of the electronic device, it can be considered that if the vehicle attitude is abnormal in the next period of time, the probability of causing an accident will increase. The target detection confidence level and alarm configuration parameters of ADAS can be switched to the corresponding parameters in the pre-set sensitive mode. In the sensitive mode, ADAS alarm events can be detected more easily, and the low alarm interval can prompt the driver more frequently.

[0130] If no ADAS alarm is triggered within a period of time after entering the ADAS sensitive mode, the ADAS sensitive mode is exited and switched to the normal mode.

[0131] Exemplarily, if an ADAS alarm is triggered after entering the ADAS sensitive mode, the DMS sensitive mode can also be entered; or, if a DMS alarm is triggered after entering the DMS sensitive mode, the ADAS sensitive mode can also be entered. If no ADAS alarm and DMS alarm are triggered within a period of time after entering the two sensitive modes, the electronic device can be switched to the ADAS normal mode and the DMS normal mode.

[0132] Exemplarily, the durations of the ADAS sensitive mode and the DMS sensitive mode (i.e., the durations of the above-mentioned preset detection periods) are configurable.

[0133] Figure 4 The flowchart of another vehicle alarm control method shown in an exemplary embodiment of the present application. As Figure 4 shown, the method may include:

[0134] Model (i.e., the above-mentioned target convolutional neural network model) training and threshold selection: The target detection algorithm model adopts a deep learning algorithm based on convolutional neural network. During the algorithm development stage, a large amount of training data is collected to train the target detection algorithm model, and two target detection confidence levels (which can also be called target detection thresholds) are selected through a large number of test sets. Its schematic diagram can be as Figure 3 shown.

[0135] Among them, the alarm event detection confidence level and alarm configuration parameters can be configured through human-machine interaction interfaces such as the host computer / mobile phone APP. The electronic device has two working modes: normal mode and sensitive mode. All parameters in the alarm event detection conditions can be configured in both levels of working modes.

[0136] Exemplarily, when the model training and threshold selection are completed, the trained model can be deployed to an embedded system (i.e., the electronic device deployed on the vehicle), and the embedded system performs alarm event detection according to the parameters in the FLASH (flash memory).

[0137] Exemplarily, after the embedded system is deployed and running normally, the embedded system continuously analyzes the video data collected by the ADAS camera and the DMS camera to determine whether an alarm event occurs.

[0138] Taking the detection of an ADAS alarm event as an example, it is considered that the occurrence of abnormal driver behavior (i.e., DMS alarm event) within a subsequent period of time will seriously threaten driving safety, and enter the DMS sensitive mode. At the same time, the detection time or alarm time of the current ADAS alarm event can be recorded as the start time of the DMS sensitive window period (i.e., the above-mentioned preset detection period). Among them, the duration of this window period can be configured through the host computer / mobile phone APP.

[0139] If a DMS alarm event occurs again within the DMS sensitive window period, the DMS sensitive window period will be refreshed to a new cycle (i.e., the detection time or alarm time of the DMS alarm event detected again is determined as the start time of the new cycle); if no DMS alarm event is detected within the DMS sensitive window period, the window period ends, and the electronic device switches to the normal mode, that is, exits the DMS sensitive mode.

[0140] Taking the detection of a DMS alarm event as an example, it is considered that the occurrence of vehicle driving posture-related alarm events (i.e., ADAS alarm events) within a subsequent period of time will seriously threaten driving safety, and the ADAS sensitive mode is entered. At the same time, the detection time or alarm time of the current DMS alarm event can be recorded as the start time of the ADAS sensitive window period (i.e., the above-mentioned preset detection period). Among them, the duration of this window period can also be configured through the host computer / mobile phone APP.

[0141] If an ADAS alarm event occurs again within the ADAS sensitive window period, the ADAS sensitive window period will be refreshed to a new cycle (i.e., the detection time or alarm time when the ADAS alarm event is detected again is determined as the start time of the new cycle); if no ADAS alarm event is detected within the ADAS sensitive window period, the window period ends, and the electronic device switches to the normal mode, that is, exits the ADAS sensitive mode.

[0142] Exemplarily, in the case of entering the ADAS sensitive mode and the DMS sensitive mode at the same time, as long as no alarm event is triggered within each sensitive window period (including ADAS alarm events or DMS alarm events), the sensitive mode can be exited; if an alarm event is detected within a certain window period, the ADAS sensitive mode and the DMS sensitive mode are maintained, and at the same time, the sensitive window period is refreshed to a new cycle.

[0143] The method provided by the embodiments of the present application has been described above. Next, the device provided by the embodiments of the present application will be described. For the content not explained in detail in the following embodiments, reference can be made to the description in the above embodiments.

[0144] Please refer to Figure 5 , which is a schematic structural diagram of a vehicle alarm control device provided by an embodiment of the present application. As Figure 5 shown, the vehicle alarm control device may include:

[0145] A detection unit 510, configured to detect a first type of alarm event for the image data provided by a first type of camera according to the first type of alarm event detection condition currently used;

[0146] An adjustment unit 520, configured to adjust the second type of alarm event detection condition currently used according to the detection result of the first type of alarm event;

[0147] Wherein, when the first type of alarm event is an advanced driver assistance system (ADAS) alarm event, the first type of camera is an ADAS camera, and the second type of alarm event is a driver monitoring system (DMS) alarm event; or,

[0148] When the first type of alarm event is a DMS alarm event, the first type of camera is a DMS camera, and the second type of alarm event is an ADAS alarm event.

[0149] In some embodiments, the alarm event detection conditions include: the alarm event detection confidence level and the alarm configuration parameters. The alarm configuration parameters include the sensitivity and / or the alarm interval. The sensitivity is used to characterize the ease of alarm triggering, and the higher the sensitivity, the easier the alarm is to be triggered.

[0150] In some embodiments, the second type of alarm event detection conditions include the detection conditions respectively corresponding to the second type of working mode at the first level and the second type of working mode at the second level;

[0151] The adjustment unit 520 is specifically configured to, when detecting the first type of alarm event and the level of the current second type of working mode is the first level, adjust the level of the current second type of working mode to the second level;

[0152] Wherein, for the second type of working mode, the alarm event detection confidence level at the first level is higher than the alarm event detection confidence level at the second level; and / or,

[0153] For the second type of working mode, the alarm interval at the first level is higher than the alarm interval at the second level, and / or, the sensitivity at the first level is lower than the sensitivity at the second level.

[0154] In some embodiments, the adjustment unit 520 is further configured to, when the level of the current second type of working mode is the second level and no second type of alarm event is detected within a preset detection period, adjust the level of the current second type of working mode to the first level.

[0155] In some embodiments, as Figure 6 shown, the vehicle alarm control device may further include:

[0156] The storage unit 530 is configured to, when the detection unit detects the first type of alarm event, associate and store the first type of alarm event and the image data collected by the first type of camera.

[0157] In some embodiments, for the second type of working mode, the alarm event detection confidence level at the first level is the first alarm event detection confidence level, and the alarm event detection confidence level at the second level is the second alarm event detection confidence level;

[0158] Among them, the first alarm event detection confidence level is the confidence level when the slope of the target confidence level - false alarm rate curve converges successfully, and the second alarm event detection confidence level is the confidence level corresponding to the inflection point of the slope of the target confidence level - false alarm rate curve;

[0159] The target confidence level - false alarm rate curve is used to characterize the change relationship between the alarm event detection confidence level and the false alarm rate.

[0160] In some embodiments, the alarm event detection confidence level and the false alarm rate are related to the target convolutional neural network model, and the target convolutional neural network model is used for alarm event detection.

[0161] In some embodiments, the ADAS alarm events include alarm events related to the vehicle driving posture, and the DMS alarm events include alarm events related to the driver's driving state.

[0162] In some embodiments, the alarm event detection confidence level and the alarm configuration parameters include data input by the user through the configuration interface of the alarm event detection conditions;

[0163] The configuration interface includes the interface displayed by the in-vehicle terminal or the interface displayed by the user terminal.

[0164] Please refer to Figure 7 , which is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. The electronic device may include a processor 701 and a machine-readable storage medium 702 storing a computer program. The processor 701 and the machine-readable storage medium 702 may communicate via a system bus 703. And by reading and executing the computer program corresponding to the vehicle alarm control logic in the machine-readable storage medium 702, the processor 701 may execute any vehicle alarm control method described above.

[0165] The machine-readable storage medium 702 mentioned in this article may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Random Access Memory, random access memory), volatile memory, non-volatile memory, flash memory, storage drive (such as a hard disk drive), solid-state drive, any type of storage disk (such as an optical disc, DVD, etc.), or a similar storage medium, or a combination thereof.

[0166] In some embodiments, a machine-readable storage medium is further provided. The machine-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle alarm control method described above is implemented. For example, the machine-readable storage medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0167] An embodiment of the present application further provides a computer program product, which stores a computer program, and when the processor executes the computer program, the processor is caused to execute any vehicle alarm control method described above.

[0168] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0169] The above are only the preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A vehicle alarm control method, characterized in that, Including: Detecting a first type of alarm event for the image data provided by a first type of camera according to the first type of alarm event detection condition currently in use; Adjusting the second type of alarm event detection condition currently in use according to the detection result of the first type of alarm event; Wherein, when the first type of alarm event is an Advanced Driver Assistance System (ADAS) alarm event, the first type of camera is an ADAS camera, and the second type of alarm event is a Driver Monitoring System (DMS) alarm event; or, When the first type of alarm event is a DMS alarm event, the first type of camera is a DMS camera, and the second type of alarm event is an ADAS alarm event; Wherein, the alarm event detection condition includes: an alarm event detection confidence level and an alarm configuration parameter, and the alarm configuration parameter includes a sensitivity and / or an alarm interval. The sensitivity is used to characterize the ease of triggering an alarm, and the higher the sensitivity, the easier it is to trigger an alarm; The second type of alarm event detection condition includes detection conditions respectively corresponding to a first-level second type of working mode and a second-level second type of working mode; The adjusting the second type of alarm event detection condition currently in use according to the detection result of the first type of alarm event includes: When the first type of alarm event is detected and the level of the current second type of working mode is the first level, adjusting the level of the current second type of working mode to the second level; Wherein, for the second type of working mode, the alarm event detection confidence level at the first level is higher than the alarm event detection confidence level at the second level; For the second type of working mode, the alarm event detection confidence level at the first level is the first alarm event detection confidence level, and the alarm event detection confidence level at the second level is the second alarm event detection confidence level; Wherein, the first alarm event detection confidence level is the confidence level when the slope of the target confidence level - false alarm rate curve converges successfully, and the second alarm event detection confidence level is the confidence level corresponding to the inflection point of the slope of the target confidence level - false alarm rate curve; The target confidence level - false alarm rate curve is used to characterize the variation relationship between the alarm event detection confidence level and the false alarm rate.

2. The method according to claim 1, wherein For the second type of working mode, the alarm interval at the first level is higher than the alarm interval at the second level, and / or, the sensitivity at the first level is lower than the sensitivity at the second level.

3. The method according to claim 1, wherein The method further includes: When the level of the current second type of working mode is the second level and no second type of alarm event is detected within a preset detection period, adjusting the level of the current second type of working mode to the first level.

4. The method according to claim 1, characterized in that When the first type of alarm event is detected, the method further includes: Associating and storing the first type of alarm event and the image data collected by the first type of camera.

5. The method according to claim 1, characterized in that, The alarm event detection confidence level and the false alarm rate are related to a target convolutional neural network model, and the target convolutional neural network model is used for alarm event detection.

6. The method according to claim 1, wherein The ADAS alarm events include alarm events related to the driving attitude of the vehicle, and the DMS alarm events include alarm events related to the driving state of the driver.

7. The method according to claim 1, characterized in that The confidence level of alarm event detection and the alarm configuration parameters include data input by the user through the configuration interface of the alarm event detection conditions; The configuration interface includes the interface displayed on the in-vehicle terminal or the interface displayed on the user terminal.

8. A vehicle alarm control device, characterized in that, It includes: A detection unit, configured to perform first-type alarm event detection on the image data provided by the first-type camera according to the first-type alarm event detection conditions currently in use; An adjustment unit, configured to adjust the second-type alarm event detection conditions currently in use according to the detection result of the first-type alarm event; Wherein, when the first-type alarm event is an Advanced Driver Assistance System (ADAS) alarm event, the first-type camera is an ADAS camera, and the second-type alarm event is a Driver Monitoring System (DMS) alarm event; or, When the first-type alarm event is a DMS alarm event, the first-type camera is a DMS camera, and the second-type alarm event is an ADAS alarm event; Wherein, the alarm event detection conditions include: the confidence level of alarm event detection and the alarm configuration parameters, and the alarm configuration parameters include sensitivity and / or alarm interval. The sensitivity is used to characterize the ease of alarm triggering, and the higher the sensitivity, the easier the alarm is triggered; Wherein, the second-type alarm event detection conditions include the detection conditions corresponding to the second-type working mode at the first level and the second-type working mode at the second level respectively; The adjustment unit is specifically configured to, when the first-type alarm event is detected and the level of the current second-type working mode is the first level, adjust the level of the current second-type working mode to the second level; Wherein, for the second-type working mode, the confidence level of alarm event detection at the first level is higher than the confidence level of alarm event detection at the second level; For the second-type working mode, the confidence level of alarm event detection at the first level is the first alarm event detection confidence level, and the confidence level of alarm event detection at the second level is the second alarm event detection confidence level; Wherein, the first alarm event detection confidence level is the confidence level when the slope of the target confidence level - false alarm rate curve converges successfully, and the second alarm event detection confidence level is the confidence level corresponding to the inflection point of the slope of the target confidence level - false alarm rate curve; The target confidence level - false alarm rate curve is used to characterize the change relationship between the confidence level of alarm event detection and the false alarm rate.

9. The device according to claim 8, wherein For the second-type working mode, the alarm interval at the first level is higher than the alarm interval at the second level, and / or, the sensitivity at the first level is lower than the sensitivity at the second level; Wherein, the adjustment unit is further configured to, when the level of the current second-type working mode is the second level and no second-type alarm event is detected within a preset detection period, adjust the level of the current second-type working mode to the first level; Wherein, the device further comprises: a storage unit, configured to, when the detection unit detects the first type of alarm event, associate and store the first type of alarm event and the image data collected by the first type of camera; Wherein, for the second type of working mode, the alarm event detection confidence at the first level is the first alarm event detection confidence, and the alarm event detection confidence at the second level is the second alarm event detection confidence; Wherein, the first alarm event detection confidence is the confidence when the slope of the target confidence - false alarm rate curve converges successfully, and the second alarm event detection confidence is the confidence corresponding to the inflection point of the slope of the target confidence - false alarm rate curve; The target confidence - false alarm rate curve is used to characterize the variation relationship between the alarm event detection confidence and the false alarm rate; Wherein, the alarm event detection confidence and the false alarm rate are related to the target convolutional neural network model, and the target convolutional neural network model is used for alarm event detection; Wherein, the ADAS alarm event includes an alarm event related to the vehicle driving posture, and the DMS alarm event includes an alarm event related to the driver driving state; Wherein, the alarm event detection confidence and the alarm configuration parameters include the data input by the user through the configuration interface of the alarm event detection conditions; The configuration interface includes the interface displayed by the in - vehicle terminal or the interface displayed by the user terminal.

10. An electronic device, characterized in that, Comprising a processor and a machine - readable storage medium, the machine - readable storage medium stores machine - executable instructions that can be executed by the processor, and the processor is configured to execute the machine - executable instructions to implement the method according to any one of claims 1 - 7.

11. A machine-readable storage medium, characterized in that, The machine - readable storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 - 7 is implemented.

Images