Intelligent drive control method and device, vehicle, electronic device, and storage medium

Abstract

The present disclosure relates to an intelligent drive control method and device, a vehicle, an electronic device, and a storage medium, which includes: acquiring driving state information of a vehicle; acquiring video information of a driving area of ​​the vehicle; determining a driving state of a driver of the vehicle based on the driving state information of the vehicle and the video information of the driving area; and performing intelligent drive control in response to detecting that the driver of the vehicle is in a predetermined dangerous driving state.

Description

[Technical field]

[0001] The present disclosure relates to the field of computer technology, and in particular to an intelligent drive control method and apparatus, a vehicle, an electronic device, and a storage medium. [Background technology]

[0002] When a driver is driving a vehicle, if the driver is in a dangerous driving state, it will pose a risk to safe driving and will easily lead to a traffic accident. Therefore, how to accurately detect the driver's dangerous driving state and take measures in a timely manner is very important for safe driving. Summary of the Invention

[0003] The present disclosure provides a technical means for intelligent drive control.

[0004] According to one aspect of the present disclosure, An intelligent drive control method is provided, which includes: acquiring driving state information of a vehicle; acquiring image information of a driving area of ​​the vehicle; determining a driving state of a driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area; and performing intelligent drive control in response to detection that the driver of the vehicle is in a predetermined dangerous driving state.

[0005] In one possible embodiment, the determining of the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: detecting a behavior of a driver of the vehicle based on image information of the driving area; Determining a driving state of the driver based on information about a running state of the vehicle and a result of detecting the behavior of the driver.

[0006] In one possible embodiment, the performing of behavior detection for a driver of the vehicle includes: determining a danger level corresponding to the driver's behavior detection result; The above-mentioned determining the driving state of the driver based on the vehicle running state information and the driver behavior detection result, In response to the determination that the vehicle is in an abnormal driving state based on the driving state information of the vehicle, a danger level of the driver's behavior detection result is increased to a first danger level; In response to determining that the first danger level has reached a preset alarm level, determining that the driving state of the driver is in a preset dangerous driving state.

[0007] In one possible embodiment, the determining of the driving state of the driver based on the vehicle running state information and the driver's behavior detection result includes: This includes determining that the driver is in a predetermined dangerous driving state in response to a determination that both the vehicle's driving state and the driver's behavior state are abnormal based on the vehicle's driving state information and the driver's behavior detection result.

[0008] In one possible embodiment, the determining of the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: In response to detection that the vehicle is in an abnormal driving state based on the driving state information of the vehicle, detecting a behavior of the driver based on image information of the driving area. The method includes determining that the driver is in a predetermined dangerous driving state in response to the driver's behavior detection result being determined to be inattentive driving or fatigued driving.

[0009] In one possible embodiment, the abnormal operating condition of the vehicle is The detection signal includes at least one of the following: the number of times the vehicle steps on a line within a first set time period reaches a predetermined threshold number of times the vehicle steps on a line; the width of the vehicle's left and right sway reaches a predetermined width threshold; the vehicle's passage does not follow the instructions of road signs or traffic signals; and the vehicle's speed exceeds a predetermined speed threshold.

[0010] In one possible embodiment, performing the intelligent drive control includes: information on the running state of the vehicle; A behavior detection result of the driver of the vehicle based on image information of the driving area; and a driving state of the driver.

[0011] In one possible embodiment, performing the intelligent drive control includes: The method includes controlling the alarm device to continue outputting alarm information until both the vehicle running state and the driver's driving state return to normal.

[0012] In one possible embodiment, performing the intelligent drive control includes: In response to detecting that the driver is in a predetermined dangerous driving state for a duration exceeding a predetermined time, control is performed to activate an assisted driving / automated driving function and / or control the vehicle to decelerate and stop.

[0013] In one possible embodiment, The vehicle further includes acquiring environmental information about a surrounding of the vehicle; The above-mentioned determining a driving state of a driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: The method includes determining a driving state of a driver of the vehicle based on environmental information around the vehicle, driving state information of the vehicle, and image information of the driving area.

[0014] In one possible embodiment, the environmental information comprises road traffic information, The acquiring of environmental information about the surroundings of the vehicle includes acquiring road traffic information about a road on which the vehicle is traveling, The above-mentioned determining a driving state of a driver of the vehicle based on environmental information around the vehicle, driving state information of the vehicle, and image information of the driving area includes: The method includes determining that the driver is in a predetermined dangerous driving state when the vehicle's driving state information and the road traffic information do not satisfy a predetermined correspondence relationship and when a behavior detection result of the driver of the vehicle based on image information of the driving area indicates that the driver's behavior state is abnormal.

[0015] In one possible embodiment, the obtaining of the vehicle driving state information includes: Acquiring driving sensing data including at least one of video information collected by an advanced driving assistance system of the vehicle, video information collected by a drive recorder of the vehicle, and speed information detected by a speed sensor of the vehicle; Determining driving state information of the vehicle based on the driving sensing data.

[0016] In one possible embodiment, the determining of the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: The method includes inputting the acquired driving state information of the vehicle and image information of the driving area into a trained neural network, and determining the driving state of the driver of the vehicle through the neural network.

[0017] According to one aspect of the present disclosure, A first acquisition module for acquiring vehicle running state information; a second acquisition module for acquiring image information of a driving area of ​​the vehicle; A driving state determination module for determining a driving state of a driver of the vehicle based on driving state information of the vehicle and image information of the driving area; and a control module for performing intelligent drive control in response to detection that a driver of the vehicle is in a predetermined dangerous driving state.

[0018] In one possible embodiment, the driving state determination module includes a first driving state determination sub-module and a second driving state determination sub-module; The first driving state determination submodule detects a behavior of a driver of the vehicle based on image information of the driving area; The second driving state determination submodule determines the driving state of the driver based on the vehicle running state information and the driver behavior detection result.

[0019] In one possible embodiment, the first driving state determination submodule determines a danger level corresponding to the driver's behavior detection result; The second driving state determination submodule: raising a danger level of the driver's behavior detection result to a first danger level in response to the vehicle being determined to be in an abnormal driving state based on the vehicle driving state information; In response to the determination that the first danger level has reached a preset alarm level, it is determined that the driving state of the driver is in a preset dangerous driving state.

[0020] In one possible embodiment, the second driving state determination submodule determines that the driver is in a predetermined dangerous driving state in response to determining that both the vehicle's driving state and the driver's behavior state are abnormal based on the vehicle's driving state information and the driver's behavior detection result.

[0021] In one possible embodiment, the driving state determination module includes a third driving state determination sub-module and a fourth driving state determination sub-module; The third driving state determination submodule performs a behavior detection of the driver based on the image information of the driving area in response to the detection that the vehicle is in an abnormal driving state based on the driving state information of the vehicle; The fourth driving state determination submodule determines that the driver is in a predetermined dangerous driving state in response to the driver's behavior detection result being determined to be inattentive driving or fatigued driving.

[0022] In one possible embodiment, the abnormal operating condition of the vehicle is The detection signal includes at least one of the following: the number of times the vehicle steps on a line within a first set time period reaches a predetermined threshold number of times the vehicle steps on a line; the width of the vehicle's left and right sway reaches a predetermined width threshold; the vehicle's passage does not follow the instructions of road signs or traffic signals; and the vehicle's speed exceeds a predetermined speed threshold.

[0023] In one possible embodiment, the control module comprises: information on the running state of the vehicle; A behavior detection result of the driver of the vehicle based on image information of the driving area; and a driving state of the driver.

[0024] In one possible embodiment, the control module controls the alarm device to continue outputting alarm information until both the vehicle running state and the driver's driving state return to normal.

[0025] In one possible embodiment, the control module controls an assisted / automated driving function to be activated and / or controls the vehicle to slow down and stop in response to detecting that the driver is in a pre-defined unsafe driving condition for a duration exceeding a pre-defined time.

[0026] In one possible embodiment, the device comprises: Further comprising a third acquisition module for acquiring environmental information around the vehicle; The driving state determination module determines a driving state of a driver of the vehicle based on environmental information about the surroundings of the vehicle, driving state information about the vehicle, and image information about the driving area.

[0027] In one possible embodiment, the environmental information comprises road traffic information, The third acquisition module acquires road traffic information of a road on which the vehicle is traveling; The driving state determination module determines that the driver is in a predetermined dangerous driving state when the vehicle's driving state information and the road traffic information do not satisfy a predetermined correspondence relationship and the behavior detection result of the driver of the vehicle based on the image information of the driving area indicates that the driver's behavior state is abnormal.

[0028] In one possible embodiment, the first acquisition module includes a sensing data acquisition submodule and a driving state information determination submodule; The sensing data acquisition submodule acquires driving sensing data including at least one of video information collected by an advanced driving assistance system of the vehicle, video information collected by a drive recorder of the vehicle, and speed information detected by a speed sensor of the vehicle; The driving state information determination submodule determines driving state information of the vehicle based on the driving sensing data.

[0029] In one possible embodiment, the driving state determination module inputs the acquired driving state information of the vehicle and image information of the driving area into a trained neural network, and determines the driving state of the driver of the vehicle through the neural network.

[0030] A first sensor for collecting image information of a driving area of ​​the vehicle; A controller for acquiring vehicle driving state information, judging a driving state of a driver of the vehicle based on the vehicle driving state information and image information of the driving area, and performing intelligent drive control in response to detection that the driver of the vehicle is in a predetermined dangerous driving state.

[0031] In one possible embodiment, the vehicle further includes a second sensor for collecting driving sensing data; The controller acquires information about the driving state of the vehicle based on the driving sensing data.

[0032] According to one aspect of the present disclosure, there is provided an electronic device including a processor and a memory for storing instructions executable by the processor, the processor being configured to invoke the instructions stored in the memory to execute the above-described intelligent drive control method.

[0033] According to one aspect of the present disclosure, there is provided a computer readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a processor, implementing the above-described intelligent drive control method.

[0034] According to one aspect of the present disclosure, there is provided a computer program product including computer readable code, the computer readable code, when executed in an electronic device, causing a processor of the electronic device to execute instructions for implementing the above-described intelligent drive control method.

[0035] In the embodiment of the present disclosure, the vehicle driving state information and the image information of the driving area of ​​the vehicle are acquired, the driving state of the driver of the vehicle is judged based on the driving state information of the vehicle and the image information of the driving area, and the intelligent drive control is performed in response to the detection that the driver of the vehicle is in a pre-set dangerous driving state, so that the driving state of the driver of the vehicle can be judged by combining the driving state of the vehicle and the image information of the driving area, and the accuracy of the judged driving state of the driver is improved. In addition, the safety of vehicle driving is further improved by performing the intelligent drive control in response to the detection that the driver of the vehicle is in a pre-set dangerous driving state.

[0036] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and are not intended to limit the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments taken in conjunction with the drawings. [Brief description of the drawings]

[0037] The drawings herein are incorporated as part of the specification, show embodiments consistent with the present disclosure, and are used together with the specification to explain the technical means of the present disclosure. [Figure 1] 2 shows a flowchart of an intelligent drive control method according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 shows a block diagram of an intelligent drive controller according to an embodiment of the present disclosure. [Diagram 3] 8 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. [Figure 4] 19 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] Various exemplary embodiments, features, and aspects of the present disclosure are described in detail below with reference to the drawings, in which like reference numerals represent functionally identical or similar elements, and in which various aspects of the embodiments are illustrated, unless otherwise noted, the drawings are not necessarily drawn to scale.

[0039] The word "exemplary" herein means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or superior over other embodiments.

[0040] The term "and / or" in this specification simply describes a relational relationship with a related object and indicates that three relations can exist, for example, A and / or B may indicate three cases: only A is present, both A and B are present, and only B is present. Also, the term "at least one" in this specification indicates any one of a plurality, or any combination of at least two of a plurality, for example, including at least one of A, B, and C may indicate including any one or more elements selected from the set consisting of A, B, and C.

[0041] In addition, in order to more effectively describe the present disclosure, various specific details are provided in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can be similarly implemented without any specific details. In some embodiments, detailed descriptions of methods, means, elements and circuits familiar to those skilled in the art are not provided in order to emphasize the gist of the present disclosure.

[0042] The present disclosure provides an intelligent drive control method, and an execution subject of the method may be an intelligent drive control device installed in a vehicle. In one possible embodiment, the method may be executed by a terminal device, a server, or other processing device. Here, the terminal device may be an in-vehicle device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a wearable device, etc. The in-vehicle device may be an in-vehicle infotainment or domain controller in a vehicle cabin, or may be a device host for executing the intelligent drive control method in an ADAS (Advanced Driving Assistance System), an OMS (Occupant Monitoring System), or a DMS (Driver Monitoring System), etc. In some possible embodiments, the intelligent drive control method may be realized by a processor calling computer-readable instructions stored in a memory.

[0043] 1 shows a flowchart of an intelligent drive control method according to an embodiment of the present disclosure. As shown in FIG 1, the intelligent drive control method includes steps S11 to S14.

[0044] In step S11, vehicle running state information is acquired.

[0045] The vehicle herein may be at least one of a private car, a car-sharing, an online car-hailing, a taxi, a truck, etc., and the present disclosure does not limit the specific type of the vehicle.

[0046] The driving state information represents a driving state of the vehicle. In one possible embodiment, the driving state can be divided into a normal driving state and an abnormal driving state. The normal driving state may be that the vehicle drives according to rules such as a predetermined speed, direction, route, and road traffic law. The abnormal driving state may be that the vehicle drives without following rules such as a predetermined speed, direction, route, and road traffic law, corresponding to the normal driving state.

[0047] The driving condition information may include at least one of information representing the normal driving condition or abnormal driving condition of the vehicle, such as the vehicle's speed, direction, acceleration, information on the road on which the vehicle is located, line stepping information, vehicle body swaying information, lane change information, speed change information, braking information, braking information, route information, speeding situation, and information on the degree of correspondence between the driving condition and road signs.

[0048] In one possible embodiment, the abnormal driving condition of the vehicle includes at least one of the following: the number of times the vehicle crosses a line within a first set time period reaches a predetermined number of times a line crossing threshold; the width of the vehicle's left and right sway reaches a predetermined width threshold; the vehicle's traffic does not follow the instructions of road signs or traffic signals; and the vehicle's speed exceeds a predetermined speed threshold.

[0049] Here, the fact that the number of times the vehicle steps on the line within the first set time has reached a predetermined threshold value for the number of times the vehicle steps on the line indicates that the vehicle steps on the line continuously within a short time. In such a case, it can be determined that the vehicle is in an abnormal driving state. The first set time can be set according to the actual situation, and may be, for example, one minute or 15 seconds. The predetermined threshold value for the number of times the vehicle steps on the line can also be set according to the actual situation, and may be, for example, two times, and is not particularly limited in the present disclosure.

[0050] When the width of the vehicle's left and right sway reaches a predetermined width threshold, it indicates that the width of the vehicle body's left and right sway is relatively large. In such a case, it can be determined that the vehicle is in an abnormal driving state. The predetermined width threshold can be set according to the actual situation such as the type of vehicle, and is not particularly limited in the present disclosure.

[0051] In addition, failure of a vehicle to follow road signs or traffic signals and a vehicle speed exceeding a predetermined speed threshold also indicate that the vehicle is in an abnormal driving state, and a detailed explanation of this will be omitted here.

[0052] The vehicle's driving condition information may be determined based on information such as the vehicle's driving speed, driving direction, position, and the environment around the vehicle, and this information can be obtained using a sensor for detecting environmental information outside the passenger compartment and / or a sensor for detecting the vehicle's operating condition.

[0053] In one optional embodiment, acquiring the driving condition information of the vehicle includes acquiring sensing data including at least one of video information collected by an advanced driver assistance system (ADAS) of the vehicle, video information collected by a drive recorder of the vehicle, and speed information detected by a speed sensor of the vehicle, and determining the driving condition information of the vehicle based on the driving sensing data.

[0054] For example, an image collecting device, millimeter wave / laser radar, sonar sensor, etc. in the vehicle body acquires information for monitoring vehicle conditions such as images, temperature, and pressure outside the vehicle cabin. These sensors can be placed in the front and rear bumpers, side mirrors, inside the steering column, or on the windshield of the vehicle to collect sensing data outside the vehicle cabin. These sensing data can detect the environment around the vehicle, including pedestrians, objects such as vehicles, and information on lanes, traffic signals, road sign lights, etc. around the vehicle. In addition, information such as vehicle speed, acceleration, position, and attitude can be identified by speed / acceleration sensors, attitude sensors, positioning devices, etc.

[0055] In step S12, image information of the driving area of ​​the vehicle is obtained.

[0056] The driving area here may be an area in the vehicle cabin where the driver is located, or any area including the area where the driver is located, and the area is generally the area of ​​the driver's seat.

[0057] Therefore, the video information of the driving area may be video information of the area in the vehicle where the driver is located. The video information may be collected by an on-board video collection device arranged inside or outside the vehicle, and the on-board video collection device may be an on-board camera or a video collection device with a camera. The camera may be a camera for collecting video information inside the vehicle, or a camera for collecting video information outside the vehicle.

[0058] For example, the camera may include a camera in the DMS and / or a camera in the OMS, and these cameras may collect image information inside the vehicle. The camera may include a camera in the ADAS, and may collect image information outside the vehicle. Of course, the in-vehicle image collection device may be a camera in another system or a camera that is separately arranged, and the embodiment of the present disclosure does not specifically limit the in-vehicle image collection device.

[0059] The carrier of the image information here may be a two-dimensional image or video. For example, the image information may be specifically a visible light image / video, or an infrared light image / video, or a three-dimensional image formed by a point cloud by radar scanning, etc., which can be determined according to the actual application scene, and the present disclosure is not particularly limited.

[0060] The collected video information can be obtained through a communication connection established with the vehicle-mounted video collection device. In one example, the vehicle-mounted video collection device can transmit the collected video information to the vehicle-mounted controller or the remote server in real time via a bus or a wireless communication channel, and the vehicle-mounted controller or the remote server can receive the real-time video information via the bus or the wireless communication channel.

[0061] In step S13, the driving state of the driver of the vehicle is judged based on the driving state information of the vehicle and the image information of the driving area.

[0062] The driving state of the driver may be a state when the driver drives the vehicle. The driving state may be divided into a normal driving state and a dangerous driving state. The dangerous driving state may be a state in which the driver is performing a pre-set non-normative driving behavior, such as making a phone call, looking at a mobile phone, taking hands off the steering wheel, or driving while fatigued.

[0063] In the embodiment of the present disclosure, when there is a non-normative driving behavior by the driver, it is often accompanied by a change in the vehicle driving state, and for example, when the driver is in a dangerous driving state such as inattentive driving or overtired driving, it is considered that an abnormality occurs in the driving state of the vehicle, such as an abnormal driving state such as an unstable vehicle speed or continuous line stepping. Therefore, in the embodiment of the present disclosure, the driving state of the driver can be determined based on the image information of the driving area and the driving state information of the vehicle in order to increase the reliability of the determined driving state of the driver.

[0064] As described above, the image information of the driving area is image information of the area in the vehicle where the driver is located. Therefore, the image information of the driving area can be analyzed and processed using image processing technology to detect the driver's characteristics, such as physical characteristics and facial characteristics, and the driver's behavior can be analyzed based on the detected driver's characteristics to determine whether the driver will make a phone call, drink water, take his / her hands off the steering wheel, or close his / her eyes.

[0065] When the vehicle driving state is already abnormal, the probability that the driver is in a pre-defined dangerous driving state is relatively high. Therefore, the vehicle driving state information and the image information of the driving area can be combined to determine the driver's driving state. For example, if the reliability of determining that the driver is in an overfatigue driving state is 0.5 by analyzing the image information of the driving area and the vehicle is in a continuous stepping on the line at this time, the reliability of the driver being in an overfatigue driving state can be increased to 0.8. Also, for example, if the image information of the driving area is analyzed to determine that the driver is in a mild overfatigue driving state and the vehicle is in a continuous stepping on the line at this time, the mild overfatigue driving state can be increased to a moderate overfatigue driving state or a severe overfatigue driving state.

[0066] In step S14, in response to detection that the driver of the vehicle is in a preset dangerous driving state, intelligent drive control is performed.

[0067] The predefined dangerous driving state is a predefined driving state that is a risk to safe driving, and may include one or more predefined driver states, such as a moderate overworked driving state, a severe fatigued state, a severe distracted driving state, etc. The predefined dangerous driving state may be a state defined by combining a predefined driver behavior with a predefined vehicle state, for example, a state in which the driver makes a phone call and the vehicle steps on the line two or more times consecutively can be defined as a predefined dangerous driving state. Intelligent drive control can be performed for the predefined dangerous driving state. The intelligent drive control here may be an intervention in driving according to the road conditions on which the vehicle is traveling by a vehicle control center or a remote control terminal. A specific intelligent drive control method may be, for example, a measure such as transmission of alarm information, activation control of assisted driving / automatic driving function, and deceleration control of the vehicle. The following will be specifically described in detail in combination with possible embodiments of the present disclosure, and detailed description will be omitted here.

[0068] In some alternative embodiments, the risk level can be determined based on the driver's dangerous driving state and the vehicle's running state, and a corresponding intelligent drive control method can be selected according to the risk level. For example, when the driver is in a dangerous driving state such as making a call, looking at a mobile phone, or taking his / her hands off the steering wheel, and the vehicle's continuous stepping time does not exceed a preset time (e.g., 5 seconds), the intelligent drive control can be performed by sending alarm information, and when the driver is in a dangerous driving state such as making a call, looking at a mobile phone, or taking his / her hands off the steering wheel, and the vehicle is in a state where the vehicle is overspeeding by 50%, the vehicle's continuous stepping time is more dangerous than the state where the vehicle's continuous stepping time does not exceed a preset time (e.g., 5 seconds), the intelligent drive control can be performed by controlling the vehicle to decelerate, thereby realizing flexible safe driving control.

[0069] In an embodiment of the present disclosure, vehicle driving state information and image information of the driving area of ​​the vehicle are acquired, the driving state of the driver of the vehicle is determined based on the driving state information of the vehicle and the image information of the driving area, and intelligent drive control is performed in response to detection that the driver of the vehicle is in a pre-set dangerous driving state. This allows the driving state of the driver of the vehicle to be determined by combining the driving state of the vehicle and the image information of the driving area, and improves the accuracy of the determined driving state of the driver. In addition, the safety of vehicle driving is further improved by performing intelligent drive control in response to detection that the driver of the vehicle is in a pre-set dangerous driving state.

[0070] In some possible embodiments, the image information of the driving area may be image information of the inside of the vehicle cabin, and the driving state information of the vehicle may be information determined based on sensing data outside the vehicle cabin. That is, in the present disclosure, the driving state of the driver can be determined by combining the information inside the vehicle cabin and the information outside the vehicle cabin, and the accuracy of the determined driving state of the driver is improved. In addition, the driving state of the driver is determined by combining the information inside the vehicle cabin and the information outside the vehicle cabin, and intelligent drive control is performed on the vehicle, thereby improving the safety of vehicle driving.

[0071] In one possible embodiment, determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes performing behavior detection for the driver of the vehicle based on the image information of the driving area, and determining the driving state of the driver based on the driving state information of the vehicle and the driver's behavior detection result.

[0072] In this embodiment, behavior detection of the vehicle driver is performed based on image information of the driving area to obtain the behavior detection result, and then the vehicle's driving state information and the behavior detection result are combined to determine the driver's driving state, thereby improving the accuracy of the obtained driver's driving state.

[0073] The image information of the driving area is analyzed by image processing technology to perform behavior detection for the driver, and a behavior detection result can be obtained. In the embodiment of the present disclosure, the preset dangerous driving state may be a state in which the driver performs a preset non-normative driving behavior. Therefore, the behavior detection for the driver of the vehicle based on the image information of the driving area may be detection of the non-normative driving behavior of the driver. For example, when it is detected that the mobile phone is held in the driver's hand and near the ear, the behavior detection result can be determined to be a behavior in which the driver makes a call while driving. When it is detected that the time during which the driver's hands are not on the steering wheel has reached a first time, the behavior detection result can be determined to be a behavior in which the driver takes his / her hands off the steering wheel. When it is detected that the time during which the driver's eyes are closed has reached a second time, the behavior detection result can be determined to be a behavior in which the driver performs mild overfatigue driving. When it is detected that the time during which the driver's eyes are closed has reached a third time, the behavior detection result can be determined to be a behavior in which the driver performs moderate overfatigue driving.

[0074] In some optional embodiments, if the detection result is that no unsafe driving behavior by the driver is detected, and the vehicle driving state information indicates that the vehicle driving state is normal, it can be determined that the driver's driving state is normal.

[0075] In some optional embodiments, when the detection result indicates that an unsafe driving behavior by the driver is detected, if the vehicle driving state information indicates that the vehicle driving state is normal, the driver's driving state can be determined based on the detected non-normative driving behavior. For example, when the behavior detection result indicates that the frequency of the driver closing his / her eyes or the number of consecutive times that the driver closes his / her eyes is within an interval corresponding to mild fatigue driving, even if the vehicle driving state information is normal, the driver's driving state is still determined to be mild fatigue driving.

[0076] In some alternative embodiments, when the detection result indicates that an unsafe driving behavior by the driver is detected, if the vehicle driving state information indicates that the vehicle driving state is abnormal, the driver's behavior detection result can be further modified based on the vehicle driving state information to obtain the driver's driving state in order to improve the reliability of the obtained behavior detection result. For example, since the abnormal driving state of the vehicle is often caused by the driver's non-normative driving behavior, the reliability of the driver's behavior being unsafe in the behavior detection result can be increased to improve the reliability of the obtained dangerous driving state. Also, since the vehicle is already in an abnormal driving state, the danger level of the detection result can be increased to take a higher level of response measures.

[0077] In one possible embodiment, performing behavior detection of the driver of the vehicle includes determining a danger level corresponding to the driver's behavior detection result, and determining the driver's driving state based on the vehicle driving state information and the driver's behavior detection result includes raising the danger level of the driver's behavior detection result to a first danger level in response to determining that the vehicle is in an abnormal driving state based on the vehicle driving state information, and determining that the driver's driving state is in a predetermined dangerous driving state in response to determining that the first danger level has reached a predetermined alarm level.

[0078] The driver's behavior detection result may correspond to a risk level, and the risk level represents the degree of risk of the driver's dangerous driving state. For example, if it is detected that the driver has his / her eyes closed for a second time, the behavior detection result may be determined to be that the driver has a behavior of driving with mild fatigue. If it is detected that the driver has his / her eyes closed for a third time, the behavior detection result may be determined to be that the driver has a behavior of driving with moderate fatigue. Here, the third time is longer than the second time. It is clear that the longer the driver has his / her eyes closed, the higher the degree of risk and the higher the probability of a vehicle accident occurring.

[0079] Since the vehicle is already in an abnormal driving state, the danger level of the detection result can be increased to take a higher level of countermeasures. Here, for convenience of explanation, the increased danger level is referred to as a first danger level. For the first danger level, it is determined whether the first danger level has reached a preset alarm level, and if it is determined that the first danger level has reached the preset alarm level, an alarm can be issued or other smart control operations can be performed.

[0080] For example, it is not necessary to issue an alarm for driving behavior that indicates mild fatigue, but an alarm will be issued for driving behavior that indicates severe fatigue, since this corresponds to a pre-set dangerous driving state.

[0081] In this embodiment, by determining the danger level corresponding to the driver's behavior detection result, when the vehicle is in an abnormal driving state, the danger level of the driver's behavior detection result is raised to a first danger level, and when the first danger level reaches a preset alarm level, the driver's driving state is determined to be in a preset dangerous driving state. In this way, when the vehicle is already in an abnormal driving state, the danger level of the detection result is raised and countermeasures are taken in a timely and accurate manner, thereby improving the safety of vehicle driving.

[0082] In one possible embodiment, determining the driving state of the driver based on the driving state information of the vehicle and the behavior detection result of the driver includes determining that the driver is in a predetermined dangerous driving state in response to the driving state of the vehicle and the behavior detection result of the driver determining that both the driving state of the vehicle and the behavior state of the driver are abnormal.

[0083] In this embodiment, if the vehicle's driving state is abnormal, it may cause a traffic accident, and if the driver's behavior state is also abnormal at this time, it is considered that the driver is not driving safely and the safety risk is high. Therefore, if the vehicle's driving state and the driver's behavior state are both abnormal, it can be determined that the driver is in a preset dangerous driving state. In this way, intelligent drive control is performed in response to the detection that the driver of the vehicle is in a preset dangerous driving state, thereby improving the safety of the vehicle driving.

[0084] In one possible embodiment, determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes performing behavior detection of the driver based on the image information of the driving area in response to detection that the vehicle is in an abnormal driving state based on the driving state information of the vehicle, and determining that the driver is in a predetermined dangerous driving state in response to the driver's behavior detection result being determined to be inattentive driving or fatigued driving.

[0085] In this case, distracted driving by the driver may be behavior that distracts attention from driving, such as taking your hands off the steering wheel, making a phone call while driving, drinking water while driving, or using a mobile phone while driving, while fatigued driving may be behavior such as the driver closing their eyes for a long period of time or yawning.

[0086] In this embodiment, if the vehicle's driving state is abnormal, there is a possibility of causing a traffic accident, and if the driver's behavior detection result at this time is that the driver is inattentive driving or overtired driving, it can be determined that the vehicle is in an unsafe state due to the driver's inattentive driving or overtired driving. Therefore, if the vehicle's driving state is abnormal and the driver's behavior detection result is that the driver is inattentive driving or overtired driving, it can be determined that the driver is in a pre-set dangerous driving state. This allows the intelligent drive control to intervene in a timely manner, thereby improving safety.

[0087] In one possible embodiment, performing the intelligent drive control includes generating alarm information based on at least one of driving state information of the vehicle, behavior detection results of the driver of the vehicle based on image information of the driving area, and the driving state of the driver.

[0088] In this embodiment, in order to achieve a better alarm effect, at least one of the vehicle driving state information, the driver's behavior detection result, and the driver's driving state is notified to the driver, so that the driver can recognize the severity of the current dangerous driving state. For example, if the vehicle driving state information is "the vehicle has stepped on the line three times in a row" and the vehicle driver's behavior detection result based on the video information of the driving area is "the driver is making a phone call", the generated alarm information may be "the vehicle has stepped on the line three times in a row, please do not make a phone call while driving". The specific form of issuing the alarm information may be audio or video, but the present disclosure is not particularly limited.

[0089] In this embodiment, alarm information is generated based on at least one of the vehicle driving state information, the driver's behavior detection result, and the driver's driving state, so that the driver can know the unsafe state of the vehicle and the severity of the current dangerous driving state, thereby achieving a better alarm effect.

[0090] In one possible embodiment, performing the intelligent drive control includes controlling an alarm device to continue outputting alarm information until both the vehicle running state and the driver's driving state return to normal. By continuing to output the alarm information to continuously alarm the driver, the alarm effect can be further improved, continuous monitoring and alarming of the driver's state can be realized, and driving safety can be further improved. When both the vehicle running state and the driver's driving state return to normal, the alarm can be stopped, thereby reducing the interference of unnecessary alarms with the driver.

[0091] In one possible embodiment, performing the intelligent drive control includes controlling an assisted driving / automated driving function to be activated and / or controlling the vehicle to slow down and stop in response to detecting that the driver is in a pre-defined dangerous driving state for a duration exceeding a pre-defined time.

[0092] When the driver is in a pre-set dangerous driving state for a duration exceeding a pre-set time, it indicates that the driver is unable to operate the vehicle normally for a long period of time. This has a significant impact on the driving safety of the vehicle and poses a very large safety risk. Therefore, in such a case, the assisted driving / automated driving function may be controlled to be activated or the vehicle may be controlled to decelerate and stop. Controlling the vehicle to decelerate and stop may be realized by the assisted driving / automated driving function.

[0093] Assisted / autonomous driving functions can adapt the vehicle to changing traffic conditions by controlling acceleration, braking and direction based on driving sensing data collected by the vehicle's sensors, and can realize driving functions such as adaptive cruise control, keeping the vehicle within its lane, and controlling the following distance between the vehicle and the vehicle ahead.

[0094] In this embodiment, when the driver is in a pre-set dangerous driving state for a duration exceeding a pre-set time, the assisted driving / automatic driving function is controlled to be activated and / or the vehicle is controlled to decelerate and stop, thereby controlling the vehicle's driving state when the driver is unable to operate the vehicle normally for a long period of time, thereby improving the driving safety of the vehicle.

[0095] In one possible embodiment, the method further comprises obtaining environmental information about the vehicle; As described above, determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes determining the driving state of the driver of the vehicle based on environmental information around the vehicle, the driving state information of the vehicle, and the image information of the driving area.

[0096] In an actual driving scene, the driving state of a vehicle may be affected by the environment around the vehicle. For example, the speed of the vehicle may increase or decrease due to traffic congestion, or the vehicle may sway left and right due to driving on a curved road. Therefore, in order to further improve the accuracy of the determined driving state of the driver of the vehicle, the driving state information of the vehicle may be determined based on the environmental information around the vehicle, the driving state information of the vehicle, and the image information of the driving area.

[0097] Here, the environmental information around the vehicle may include road traffic information of the road on which the vehicle is traveling, and the road traffic information represents the condition of the road on which the vehicle is traveling, and may include at least one of information indicating the degree of congestion of the road, the shape of the road, the type of the road, the speed limit of the road, etc. The road condition can be classified into a plurality of levels, such as very congested, congested, smooth, etc., based on the degree of congestion of the road, the shape of the road can be classified into shapes such as curves and straight lines, and the type of the road can be classified into types such as urban roads, rural roads, and expressways.

[0098] In one possible embodiment, acquiring environmental information about the surroundings of the vehicle includes acquiring road traffic information about the road on which the vehicle is traveling, and determining the driving state of the driver of the vehicle based on the environmental information about the surroundings of the vehicle, the driving state information of the vehicle, and the image information of the driving area includes determining that the driver is detected to be in a predetermined dangerous driving state when the driving state information of the vehicle and the road traffic information do not satisfy a predetermined correspondence relationship and the behavior detection result of the driver of the vehicle based on the image information of the driving area indicates that the behavior state of the driver is in an abnormal state.

[0099] The road traffic information can be obtained by a third party platform, such as a navigation service platform, based on the vehicle's current geographic location.

[0100] Based on environmental information around the vehicle and vehicle driving condition information, it is possible to determine whether the vehicle driving condition information and road traffic information satisfy a predetermined correspondence relationship, and if the vehicle driving condition information and road traffic information do not satisfy the predetermined correspondence relationship, it indicates that the vehicle is in an abnormal driving state.

[0101] The preset correspondence relationship is a correspondence relationship between road traffic information and driving state information of a vehicle that is driving safely in the road conditions indicated by the road traffic information. For example, the preset correspondence relationship may include at least one of a correspondence relationship between a "smooth" road condition and a vehicle traveling at a constant speed, a correspondence relationship between a straight road condition and a vehicle's left-right sway width not exceeding a predetermined width threshold, and a correspondence relationship between road speed limit information and traveling within a speed limit range.

[0102] Accordingly, the fact that the vehicle driving state information and the road traffic information do not satisfy the preset correspondence relationship may include, for example, when the vehicle's left / right sway width reaches a predetermined width threshold on a straight road section, when the vehicle drives too fast or too slow on a road section with smooth traffic flow, when the vehicle is speeding, etc. All of these cases indicate that the vehicle is in an abnormal driving state.

[0103] In this way, when the vehicle is in an abnormal driving state and the driver's behavior is in an abnormal state, it can be determined that the driver is in a preset dangerous driving state, thereby improving the accuracy of the determined driving state of the driver and further improving the safety of vehicle driving.

[0104] In an alternative embodiment, determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes inputting the acquired driving state information of the vehicle and the image information of the driving area into a trained neural network and determining the driving state of the driver of the vehicle through the neural network. In this embodiment, the accuracy and speed of determining the driving state can be improved by determining the driving state of the vehicle driver through the neural network. In addition, the neural network can be trained and configured in advance, so that it can quickly realize the detection of images or video streams with large data size, and can be adapted to intelligent drive control of real-time driving scenes.

[0105] An application scene according to an embodiment of the present disclosure will be described below. In this application scene, the DMS camera collects video information of the driving area, the processor acquires the video information of the driving area from the DMS camera, and performs behavior detection for the driver of the vehicle based on the video information of the driving area to obtain a behavior detection result. Also, the ADAS camera collects video information outside the vehicle cabin, the processor acquires the video information outside the vehicle cabin from the ADAS camera, and the vehicle's driving state information can be determined. If the determined vehicle's driving state information is "the vehicle has stepped on the line three times in a row" and the behavior detection result for the driver of the vehicle is "the driver is making a phone call," audio alarm information is generated and broadcast, saying "the vehicle has stepped on the line three times in a row, please do not make a phone call while driving."

[0106] It should be understood that the above method embodiments mentioned in this disclosure can be combined with each other to form an embodiment without violating the principles and logic. Due to the limited number of pages, detailed descriptions are omitted in this disclosure. Those skilled in the art should understand that in the above method according to a specific embodiment, the execution order of each step is specifically determined by its function and internal logic.

[0107] In addition, the present disclosure further provides an intelligent drive control device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to realize any one of the intelligent drive control methods of the present disclosure. The corresponding technical means and descriptions can be referred to the corresponding descriptions of the methods, and detailed descriptions are omitted.

[0108] FIG. 2 shows a block diagram of an intelligent drive control device according to an embodiment of the present disclosure. As shown in FIG. 2, the device includes: a first acquisition module 21 for acquiring vehicle running state information; a second acquisition module 22 for acquiring image information of the driving area of ​​the vehicle; A driving state determination module 23 for determining a driving state of a driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area; and a control module 24 for performing intelligent drive control in response to detection of a driver of the vehicle being in a pre-defined unsafe driving condition.

[0109] In one possible embodiment, the driving state determination module 23 includes a first driving state determination sub-module and a second driving state determination sub-module; The first driving state determination submodule detects a behavior of a driver of the vehicle based on image information of the driving area; The second driving state determination submodule determines the driving state of the driver based on the vehicle running state information and the driver behavior detection result.

[0110] In one possible embodiment, the first driving state determination submodule determines a danger level corresponding to the driver's behavior detection result; The second driving state determination submodule: raising a danger level of the driver's behavior detection result to a first danger level in response to the vehicle being determined to be in an abnormal driving state based on the vehicle driving state information; In response to the determination that the first danger level has reached a preset alarm level, it is determined that the driving state of the driver is in a preset dangerous driving state.

[0111] In one possible embodiment, the second driving state determination submodule determines that the driver is in a predetermined dangerous driving state in response to determining that both the vehicle's driving state and the driver's behavior state are abnormal based on the vehicle's driving state information and the driver's behavior detection result.

[0112] In one possible embodiment, the driving state determination module 23 includes a third driving state determination sub-module and a fourth driving state determination sub-module; The third driving state determination submodule performs a behavior detection of the driver based on the image information of the driving area in response to the detection that the vehicle is in an abnormal driving state based on the driving state information of the vehicle; The fourth driving state determination submodule determines that the driver is in a predetermined dangerous driving state in response to the driver's behavior detection result being determined to be inattentive driving or fatigued driving.

[0113] In one possible embodiment, the abnormal operating condition of the vehicle is The detection signal includes at least one of the following: the number of times the vehicle steps on a line within a first set time period reaches a predetermined threshold number of times the vehicle steps on a line; the width of the vehicle's left and right sway reaches a predetermined width threshold; the vehicle's passage does not follow the instructions of road signs or traffic signals; and the vehicle's speed exceeds a predetermined speed threshold.

[0114] In one possible embodiment, the control module 24 comprises: information on the running state of the vehicle; A behavior detection result of the driver of the vehicle based on image information of the driving area; and a driving state of the driver.

[0115] In one possible embodiment, the control module 24 controls the alarm device to continue outputting alarm information until both the vehicle running conditions and the driver's driving conditions return to normal.

[0116] In one possible embodiment, the control module 24 controls an assisted / automated driving function to be initiated and / or controls the vehicle to slow down and stop in response to detecting that the driver is in a pre-defined unsafe driving condition for a duration exceeding a pre-defined time.

[0117] In one possible embodiment, the device comprises: Further comprising a third acquisition module for acquiring environmental information around the vehicle; The driving state determination module 23 determines the driving state of the driver of the vehicle based on environmental information around the vehicle, driving state information of the vehicle, and image information of the driving area.

[0118] In one possible embodiment, the environmental information comprises road traffic information; The third acquisition module acquires road traffic information of a road on which the vehicle is traveling; The driving state determination module determines that the driver is in a predetermined dangerous driving state when the vehicle's driving state information and the road traffic information do not satisfy a predetermined correspondence relationship and the behavior detection result of the driver of the vehicle based on the image information of the driving area indicates that the driver's behavior state is abnormal.

[0119] In one possible embodiment, the first acquisition module 21 includes a sensing data acquisition submodule and a driving state information determination submodule, The sensing data acquisition submodule acquires driving sensing data including at least one of video information collected by an advanced driving assistance system of the vehicle, video information collected by a drive recorder of the vehicle, and speed information detected by a speed sensor of the vehicle; The driving state information determination submodule determines driving state information of the vehicle based on the driving sensing data.

[0120] In one possible embodiment, the driving state determination module 23 inputs the acquired driving state information of the vehicle and the image information of the driving area into a trained neural network, and determines the driving state of the driver of the vehicle through the neural network.

[0121] In some embodiments, the functions or modules included in the apparatus according to the embodiments of the present disclosure can be used to perform the methods described in the above-mentioned method embodiments, and for specific implementations thereof, reference may be made to the descriptions of the above-mentioned method embodiments, and detailed descriptions thereof will be omitted here for the sake of simplicity.

[0122] In the embodiment of the present disclosure, A first sensor for collecting image information of a driving area of ​​the vehicle; A vehicle is further provided that includes a controller for acquiring vehicle driving state information, determining a driving state of a driver of the vehicle based on the vehicle driving state information and image information of the driving area, and performing intelligent drive control in response to detection that the driver of the vehicle is in a predetermined dangerous driving state.

[0123] The vehicle of this embodiment judges the driving state of the vehicle driver by combining the running state of the vehicle with image information of the driving area, thereby improving the accuracy of the judged driving state of the driver.

[0124] In some possible embodiments, the vehicle further includes a second sensor for collecting driving sensing data, and the controller obtains driving state information of the vehicle based on the driving sensing data, thereby achieving accurate detection of the driving state by the sensor for detecting different data of the vehicle.

[0125] In an embodiment of the present disclosure, there is further provided a computer-readable storage medium having computer program instructions stored thereon, the computer-readable storage medium implementing the above-mentioned intelligent drive control method when executed by a processor. The computer-readable storage medium may be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.

[0126] In an embodiment of the present disclosure, there is further provided an electronic device including a processor and a memory for storing instructions executable by the processor, the processor being configured to invoke the instructions stored in the memory to execute the above-mentioned intelligent drive control method.

[0127] In an embodiment of the present disclosure, there is further provided a computer program product including computer readable code that, when executed in an apparatus, causes a processor of the apparatus to execute instructions for implementing an intelligent drive control method according to any of the embodiments described above.

[0128] In an embodiment of the present disclosure, there is further provided another computer program product having computer readable instructions stored thereon, the instructions, when executed, causing a computer to perform operations of the intelligent drive control method according to any of the embodiments described above.

[0129] The electronic device may be provided as a terminal, a server, or another form of device.

[0130] 3 illustrates a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0131] Referring to FIG. 3, electronic device 800 may include one or more of a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0132] The processing component 802 typically controls the overall operation of the electronic device 800, such as operations related to display, phone calls, data communication, camera operation, and recording operation. The processing component 802 may include one or more processors 820 that execute instructions to perform all or some of the steps of the above methods. The processing component 802 may also include one or more modules for interaction with other components. For example, the processing component 802 may include a multimedia module for interaction with the multimedia component 808.

[0133] The memory 804 is configured to store various types of data to support operation on the electronic device 800. These data include, by way of example, instructions for any application programs or methods operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by various types of volatile or non-volatile storage devices, such as, for example, static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks, or combinations thereof.

[0134] The power component 806 provides power to each component of the electronic device 800. The power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0135] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, it may be implemented as a touch screen that receives input signals from a user. The touch panel may include one or more touch sensors to detect touches, slides, and gestures on the touch panel. The touch sensors may detect not only the boundaries of a touch or slide motion, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operational mode, such as a photography mode or a capture mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may have a fixed optical lens system or a focal length and optical zoom capability.

[0136] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a record mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0137] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0138] The sensor component 814 includes one or more sensors for evaluating the state of each aspect of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components such as the display and keypad of the electronic device 800, and the sensor component 814 can further detect changes in the position of the electronic device 800 or certain components of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration or deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor component 814 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor component 814 may further include an optical sensor for use in imaging applications, such as a complementary metal oxide semiconductor (CMOS) or charge-coupled device (CCD) image sensor. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0139] The communication component 816 is configured to realize wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be realized by radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0140] In an exemplary embodiment, the electronic device 800 may be implemented with one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components and may be used to perform the methods described above.

[0141] In an exemplary embodiment, a non-volatile computer readable storage medium, such as a memory 804 containing computer program instructions, which, when executed by the processor 820 of the electronic device 800, can cause the above-described method to be performed, is further provided.

[0142] FIG. 4 illustrates a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 4, the electronic device 1900 includes a processing component 1922 including one or more processors, and a memory resource, represented by a memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. The processing component 1922 is configured to execute the above method by executing the instructions.

[0143] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as a Microsoft server operating system (Windows Server®), a graphical user interface operating system from Apple (Mac OS X®), a multi-user, multi-process computer operating system (Unix®), a free and open source Unix-like operating system (Linux®), an open source code Unix-like operating system (FreeBSD®), or the like.

[0144] In an exemplary embodiment, a non-volatile computer readable storage medium, such as a memory 1932 containing computer program instructions, which, when executed by the processing component 1922 of the electronic device 1900, can cause the above-described method to be performed, is further provided.

[0145] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of the present disclosure.

[0146] A computer readable storage medium may be a tangible device capable of storing and storing instructions for use in an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage media include portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical coding devices such as punch cards or slot-in projection structures on which instructions are stored, and any suitable combination of the above. A computer readable storage medium as used herein is not to be construed as a momentary signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated through a wave guide or other transmission medium (e.g., pulsed light passing through a fiber optic cable), or electrical signals transmitted through electrical wires.

[0147] The computer readable program instructions described herein may be downloaded from a computer readable storage medium into each computing / processing device, or may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in a computer readable storage medium within each computing / processing device.

[0148] Computer program instructions for carrying out the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or target code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and general procedural programming languages ​​such as "C" or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, state information of the computer readable program instructions may be used to personalize electronic circuitry, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), such that the electronic circuitry executes the computer readable program instructions to implement aspects of the present disclosure.

[0149] Although various aspects of the present disclosure have been described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure, it should be understood that each block of the flowcharts and / or block diagrams, and combinations of each block in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0150] These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that, when these instructions are executed by the processor of the computer or other programmable data processing apparatus, an apparatus is produced that performs the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored on a computer-readable storage medium to cause a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, a computer-readable storage medium having instructions stored thereon includes an article of manufacture having instructions that implement each aspect of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0151] The computer readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device, and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to produce a computer-implemented process, such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0152] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions, and operations of systems, methods, and computer program products according to several embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or part of instructions, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions depicted in the blocks may be implemented in a different order than that depicted in the drawings. For example, two consecutive blocks may be substantially executed in parallel, or may be executed in a reverse order depending on the function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts may be implemented by a dedicated system based on hardware that executes the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0153] The computer program product may be embodied in hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is embodied as a computer storage medium. In another alternative embodiment, the computer program product is embodied as a software product, such as a Software Development Kit (SDK).

[0154] Although the embodiments of the present disclosure have been described above, the above description is merely illustrative and not exhaustive, and is not limited to the embodiments disclosed. Various modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms selected in this specification are intended to appropriately interpret the principles, practical applications, or improvements to existing technology of the embodiments, or to allow other skilled in the art to understand the embodiments disclosed in the text.

Claims

1. Acquiring vehicle driving state information; acquiring image information of a driving area of ​​the vehicle; Determining a driving state of a driver of the vehicle based on driving state information of the vehicle and image information of the driving area; In response to detection of a driver of the vehicle being in a predetermined dangerous driving state, performing intelligent drive control; The above-mentioned determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: detecting a behavior of a driver of the vehicle based on image information of the driving area; determining a driving state of the driver based on driving state information of the vehicle and a behavior detection result of the driver, The above-mentioned detecting the behavior of the driver of the vehicle includes: determining a first danger level corresponding to the driver's behavior detection result; The above-mentioned determining the driving state of the driver based on the vehicle running state information and the driver behavior detection result, In response to the determination that the vehicle is in an abnormal driving state based on the driving state information of the vehicle and not on the behavior detection result, increasing a first danger level of the driver's behavior detection result to a second danger level having a higher danger level; and determining that the driver's driving state is in a predetermined dangerous driving state in response to determining that the second danger level has reached a predetermined alarm level.

2. The above-mentioned determining the driving state of the driver based on the vehicle running state information and the driver behavior detection result, determining that the driver is in a dangerous driving state set in advance in response to the determination that both the vehicle driving state and the driver behavior state are abnormal based on the vehicle driving state information and the driver behavior detection result; 2. The intelligent drive control method according to claim 1, further comprising at least one of: detecting a behavior of the driver based on the image information of the driving area in response to detection that the vehicle is in an abnormal driving state based on the driving state information of the vehicle; and determining that the driver is in a predetermined dangerous driving state in response to the driver's behavior detection result being determined to be inattentive driving or fatigue driving.

3. The abnormal driving condition of the vehicle is The intelligent drive control method according to claim 1 or 2, characterized in that the method includes at least one of the following: the number of times the vehicle steps on the line within a first set time reaches a predetermined number of times threshold; the width of the vehicle's left and right sway reaches a predetermined width threshold; the vehicle's traffic does not follow the instructions of road signs or traffic signals; and the vehicle's speed exceeds a predetermined speed threshold.

4. The above-mentioned intelligent drive control is performed information on the running state of the vehicle; A behavior detection result of the driver of the vehicle based on image information of the driving area; and a driving state of the driver. The above-mentioned intelligent drive control is performed Controlling the alarm device so as to continue outputting alarm information until both the vehicle running state and the driver's driving state return to normal; The intelligent drive control method according to any one of claims 1 to 3, characterized in that it includes at least one of controlling an assisted driving / automated driving function to be activated and / or controlling the vehicle to decelerate and stop in response to detection that the driver is in a predetermined dangerous driving state for a duration exceeding a predetermined time.

5. The method further includes acquiring environmental information about the vehicle; The above-mentioned determining a driving state of a driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: The intelligent drive control method according to any one of claims 1 to 4, further comprising: determining the driving state of the driver of the vehicle based on the environmental information around the vehicle, the driving state information of the vehicle, and the image information of the driving area.

6. The environmental information includes road traffic information, The acquiring of environmental information about the surroundings of the vehicle includes acquiring road traffic information about a road on which the vehicle is traveling, The above-mentioned determining a driving state of a driver of the vehicle based on environmental information around the vehicle, driving state information of the vehicle, and image information of the driving area includes: The intelligent drive control method according to claim 5, further comprising: determining that the driver is in a predetermined dangerous driving state according to the fact that the vehicle driving state information and the road traffic information do not satisfy a predetermined correspondence relationship and the behavior detection result of the driver of the vehicle based on the image information of the driving area indicates that the behavior state of the driver is abnormal.

7. The above-mentioned determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: The intelligent drive control method according to any one of claims 1 to 6, further comprising: inputting the acquired vehicle driving state information and driving area image information into a trained neural network, and judging the driving state of the driver of the vehicle through the neural network.

8. A first acquisition module for acquiring vehicle running state information; a second acquisition module for acquiring image information of a driving area of ​​the vehicle; A driving state determination module for determining a driving state of a driver of the vehicle based on driving state information of the vehicle and image information of the driving area; A control module for performing intelligent drive control in response to detection of a driver of the vehicle being in a predetermined dangerous driving state, The driving state determination module includes a first driving state determination sub-module and a second driving state determination sub-module; The first driving state determination submodule detects a behavior of a driver of the vehicle based on image information of the driving area; The second driving state determination submodule determines a driving state of the driver based on the vehicle running state information and the driver's behavior detection result, The first driving state determination submodule determines a first danger level corresponding to a driver's behavior detection result; The second driving state determination submodule: In response to the vehicle being determined to be in an abnormal driving state based on the vehicle driving state information rather than the behavior detection result, a first danger level of the driver's behavior detection result is increased to a second danger level which is a higher danger level; An intelligent drive control device characterized in that, in response to determining that the second danger level has reached a predetermined alarm level, it is determined that the driver's driving state is in a predetermined dangerous driving state.

9. a first sensor for collecting image information of a driving area of ​​the vehicle; A controller for acquiring vehicle driving state information, judging a driving state of a driver of the vehicle based on the vehicle driving state information and image information of the driving area, and performing intelligent drive control in response to detection that the driver of the vehicle is in a predetermined dangerous driving state, The above-mentioned determining the driving state of the driver of the vehicle based on the driving state information of the vehicle and the image information of the driving area includes: detecting a behavior of a driver of the vehicle based on image information of the driving area; determining a driving state of the driver based on driving state information of the vehicle and a behavior detection result of the driver, The above-mentioned detecting the behavior of the driver of the vehicle includes: determining a first danger level corresponding to the driver's behavior detection result; The above-mentioned determining the driving state of the driver based on the vehicle running state information and the driver behavior detection result, In response to the determination that the vehicle is in an abnormal driving state based on the driving state information of the vehicle and not on the behavior detection result, increasing a first danger level of the driver's behavior detection result to a second danger level having a higher danger level; determining that the driving state of the driver is in a predetermined dangerous driving state in response to determining that the second danger level has reached a predetermined alarm level.

10. a second sensor for collecting driving sensing data; The vehicle according to claim 9 , wherein the controller acquires information about a driving state of the vehicle based on the driving sensing data.

11. A processor; a memory for storing instructions executable by a processor; 8. An electronic device, characterized in that the processor is configured to execute the intelligent drive control method according to any one of claims 1 to 7 by calling instructions stored in the memory.

12. A computer-readable storage medium having computer program instructions stored thereon, comprising: A computer-readable storage medium, comprising the computer program instructions, when executed by a processor, for implementing the intelligent drive control method according to any one of claims 1 to 7.

13. A computer program comprising computer readable code, the computer readable code, when executed in an electronic device, causing a processor of the electronic device to execute instructions for implementing the intelligent drive control method according to any one of claims 1 to 7.

Images