Detection method, alarm method, device, and vehicle for using mobile phone while driving

Abstract

The present application discloses a detection method, an alarm method, a device, and a vehicle for using a mobile phone while driving, and belongs to the field of data processing. The method includes sorting the usage interval of each frame image acquired within a preset time period to obtain a spacing time series queue, and performing weighted calculation on the usage interval in the spacing time series queue to obtain the spacing average value of the spacing time series queue, and determining the distance threshold according to the face frame detection result and the mobile phone frame detection result corresponding to each frame image in the spacing time series queue; comparing and judging the spacing average value with the distance threshold, if the spacing average value is greater than the distance threshold, and the face orientation result is that the face is facing the mobile phone, then the driver's use of the mobile phone behavior is judged as playing with the mobile phone, and if the spacing average value is not greater than the distance threshold, then the driver's use of the mobile phone behavior is judged as making a phone call, and an alarm is issued. The present application improves the accuracy of the mobile phone position in the image, and improves the accuracy and real-time performance of the judgment.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a detection method, an alarm method, a device, a vehicle, a medium and equipment for the use of a mobile phone while driving. Background Art

[0002] DMS (Driver Monitor System) is the abbreviation of driver monitoring system, which refers to the information technology system that monitors the driver's dangerous driving behavior around the clock during driving. After discovering that the driver is making a phone call or playing with the phone while driving, the DMS system will analyze such behavior in a timely manner and issue a voice prompt to warn the driver and correct the wrong driving behavior.

[0003] How to apply DMS technology to vehicles to detect whether the driver is using the phone to make calls or play with the phone while driving is an urgent problem to be solved in this industry. Summary of the invention

[0004] In order to solve the above problems, the present application mainly provides a method, device, vehicle, medium and equipment for detecting the use of mobile phones while driving.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting the use of a mobile phone while driving, which comprises:

[0006] Performing perception detection on the driver in each frame of the continuous frame images, and obtaining the face orientation result, face frame detection result and mobile phone frame detection result of the driver in each frame;

[0007] According to the mobile phone frame detection result, the mobile phone frame position change of a preset number of frame images including the current frame image is calculated, and when the mobile phone frame position change is less than the position change threshold, the current mobile phone frame position of the current frame image is determined;

[0008] According to the face frame detection result and the current mobile phone frame position, the distance between the face frame and the mobile phone frame in the current frame image is calculated to obtain the usage spacing;

[0009] In a preset time period, the used spacing of each frame of the acquired image is sorted to obtain a spacing timing queue, and the used spacing in the spacing timing queue is weightedly calculated to obtain the spacing average of the spacing timing queue, and the distance threshold is determined according to the face frame detection result and the mobile phone frame detection result corresponding to each frame of the image in the spacing timing queue;

[0010] The average distance value is compared with the distance threshold. If the average distance value is greater than the distance threshold and the face orientation result is that the face is facing the mobile phone, the driver's mobile phone use behavior is judged as playing with the mobile phone. If the average distance value is not greater than the distance threshold, the driver's mobile phone use behavior is judged as making a phone call.

[0011] In a second aspect, an embodiment of the present application provides an alarm method for using a mobile phone while driving, which includes: issuing a warning to the driver when it is detected that the driver uses a mobile phone while driving through the method in scheme one.

[0012] In a third aspect, an embodiment of the present application provides a device for detecting mobile phone use while driving, comprising:

[0013] A perception information acquisition module, which is used to perform perception detection on the driver in each frame of the continuous frame images, and obtain a face orientation result, a face frame detection result, and a mobile phone frame detection result about the driver in each frame of the image;

[0014] A position determination module is used to calculate the position change of the mobile phone frame of a preset number of frame images including the current frame image according to the mobile phone frame detection result, and determine the current mobile phone frame position of the current frame image when the position change of the mobile phone frame is less than the position change threshold;

[0015] A spacing calculation module is used to calculate the distance between the face frame and the mobile phone frame in the current frame image according to the face frame detection result and the current mobile phone frame position to obtain the use spacing;

[0016] A data processing module, which is used to sort the used spacing of each frame of the acquired image within a preset time period to obtain a spacing timing queue, and perform weighted calculation on the used spacing in the spacing timing queue to obtain the spacing average of the spacing timing queue, and determine the distance threshold according to the face frame detection result and the mobile phone frame detection result corresponding to each frame of the image in the spacing timing queue;

[0017] The alarm judgment module is used to compare and judge the average value of the interval with the distance threshold. If the average value of the interval is greater than the distance threshold and the face orientation result is that the face is facing the mobile phone, the driver's use of the mobile phone behavior is judged as playing with the mobile phone and an alarm is issued. If the average value of the interval is not greater than the distance threshold, the driver's use of the mobile phone behavior is judged as making a phone call and an alarm is issued.

[0018] In a fourth aspect, an embodiment of the present application provides a vehicle, wherein the vehicle includes the device for detecting the behavior of using a mobile phone while driving in scheme three.

[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed, the computer executes the method for detecting the behavior of using a mobile phone while driving in scheme one.

[0020] In a sixth aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the method for detecting the behavior of using a mobile phone while driving in scheme one is implemented.

[0021] The technical solution of the embodiment of the present application screens out the available information of the solution according to the directly obtained perception detection results, and uses the available information to determine the position of the mobile phone frame in each frame of the available image, calculates the distance between the face frame and the mobile phone frame in each frame of the image, and sorts them within a preset time period, and judges whether the use of the mobile phone is to make a call or play with the mobile phone based on the comparison of the average spacing of the entire spacing time series queue within the preset time period and the distance threshold. This improves the accuracy of determining the position of the mobile phone in the image, and improves the accuracy and real-time performance of judging the type of mobile phone use behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings in the following description exemplarily show some embodiments of the present application.

[0023] Figure 1 It is a schematic diagram of a specific implementation method of a method for detecting mobile phone use behavior while driving in the present application;

[0024] Figure 2 It is a schematic diagram of a specific implementation of a device for detecting mobile phone use while driving according to the present application.

[0025] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present application are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present application can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present application.

[0027] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0028] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The specific embodiments described below can be combined with each other to form new embodiments. The same or similar ideas or processes described in one embodiment may not be repeated in other embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0029] Figure 1 A specific implementation of a method for detecting mobile phone use while driving according to the present application is shown.

[0030] exist Figure 1 In the specific implementation shown, a method for detecting the use of a mobile phone while driving mainly includes:

[0031] Step S101, performing perception detection on the driver in each frame of the continuous frame images, and obtaining the face orientation result, face frame detection result and mobile phone frame detection result of the driver in each frame.

[0032] In this implementation, the perception model outputs the perception result of the driver in each frame of the image. The face frame detection result may be that the face frame is included or not included. The face orientation result is related to the face frame detection result. When the face frame detection result is that the face frame is included, the face orientation result may be that the face is facing the phone or facing other directions. The phone frame detection result may be that the phone frame is included or not included. Obtaining the perception information directly from the perception model makes the recognition efficiency higher.

[0033] exist Figure 1 In the specific implementation shown, a method for detecting the use of a mobile phone while driving further includes:

[0034] Step S102, based on the mobile phone frame detection result, calculate the mobile phone frame position change of a preset number of frame images including the current frame image, and when the mobile phone frame position change is less than the position change threshold, determine the current mobile phone frame position of the current frame image.

[0035] In this embodiment, in a preset number of frame images, the change in the position of the mobile phone frame in every two frames is calculated to obtain the change in the position of the mobile phone frame. When the change in the position of the mobile phone frame is less than the position change threshold, it can be considered that the position of the mobile phone frame in the current frame image has not changed, thereby confirming the position of the mobile phone frame and laying the foundation for subsequent calculations.

[0036] In a specific embodiment of the present application, based on the face frame detection results and the mobile phone frame detection results, images with missing face frames, missing mobile phone frames, and mobile phone frames with a size greater than a size threshold are filtered out to obtain a current frame image containing a standard face frame and a standard mobile phone frame.

[0037] In this embodiment, each frame of image will have its corresponding face frame detection result and mobile phone frame detection result. Since not every frame of image is suitable for judging the use of mobile phone behavior, it is necessary to filter out the usable images. When the mobile phone is in front of the camera, the mobile phone will appear super large and the face will appear super small, and the frame of image will be filtered out, indicating that the detection result of the mobile phone frame in the frame of image is unreliable. For images with missing mobile phone frames or missing face frames, the frame of image is also unusable and needs to be filtered out.

[0038] It should be noted that although the present application scheme is described using a mobile phone as an example, it should be understood that the present application scheme is also applicable to other items that may distract the driver's attention and cause dangerous driving behavior, including but not limited to other mobile terminal devices with communication functions, such as tablets, computers, game consoles, watches, e-book readers, etc.; common items in the car with information provision functions or entertainment functions, such as magazines, books, newspapers, maps, etc.

[0039] In addition, it should be noted that the "playing with mobile phones" behavior mentioned in this application includes all other behaviors of using mobile phones except "making phone calls", such as using mobile phones to play games, send messages, browse web pages, operate interfaces, etc., which are all included in the protection scope of this application.

[0040] In a specific embodiment of the present application, the change in the position of a mobile phone frame of a preset number of frame images including the current frame image is calculated, and when the change in the position of the mobile phone frame is less than a position change threshold, the current mobile phone frame position of the current frame image is determined, including: according to the mobile phone frame position in the mobile phone frame detection results of the preset number of frame images, the position change of the mobile phone frame position of each two adjacent frame images is calculated to obtain the change in the position of the mobile phone frame; when the change in the position of the mobile phone frame is less than the position change threshold, the mobile phone frame position in the mobile phone frame detection result of the current frame image is used as the current mobile phone frame position.

[0041] In this embodiment, in a preset number of frame images, the change in the position of the mobile phone frame in each frame image is calculated by the covariance formula. If the obtained change in the mobile phone position is less than the position change threshold, it means that the change in the position of the mobile phone frame in the current frame image is within the allowable error range, and it can be considered that the mobile phone position is determined, that is, the accuracy of the mobile phone position is credible.

[0042] exist Figure 1 In the specific implementation shown, a method for detecting the use of a mobile phone while driving further includes:

[0043] Step S103, calculating the distance between the face frame and the mobile phone frame in the current frame image according to the face frame detection result and the current mobile phone frame position to obtain a use spacing.

[0044] In this embodiment, the face detection result of the current frame image includes a face frame, and the position of the face frame is obtained together with the current mobile phone frame position in the current frame image. According to the position of the face frame in the current frame image and the current mobile phone frame position, the distance between the face frame and the mobile phone frame can be obtained, that is, the usage spacing.

[0045] exist Figure 1 In the specific implementation shown, a method for detecting the use of a mobile phone while driving further includes:

[0046] Step S104, within a preset time period, sort the usage spacing of each frame of the acquired image to obtain a spacing timing queue, and perform weighted calculation on the usage spacing in the spacing timing queue to obtain the spacing average of the spacing timing queue, and determine the distance threshold based on the face frame detection result and the mobile phone frame detection result corresponding to each frame of the image in the spacing timing queue.

[0047] In this implementation, after unavailable images are filtered out, the usage intervals of each frame of available images are sorted in chronological order according to the timestamps in the images to obtain a corresponding interval timing queue; the interval timing queue within a preset time period makes the judgment of mobile phone usage behavior more accurate. To ensure the accuracy of the judgment of the type of mobile phone usage behavior, it is necessary to calculate the average usage interval of the interval timing queue, that is, the interval average value, to lay the foundation for subsequent judgment.

[0048] In a specific embodiment of the present application, a distance threshold is determined based on a face frame detection result and a mobile phone frame detection result corresponding to each frame image in a spacing timing queue, including: determining a distance threshold based on size information of a face frame and a corresponding mobile phone frame corresponding to each frame image in a spacing timing queue.

[0049] In this embodiment, each frame of the image has attributes that record the size information of the face frame in the image and the size information of the corresponding mobile phone frame, which facilitates the determination of the distance threshold and lays the foundation for subsequent calculations.

[0050] In a specific embodiment of the present application, a distance threshold is determined based on the size information of the face frame corresponding to each frame image in the spacing timing queue and the size information of the corresponding mobile phone frame, including: according to the size information of the face frame corresponding to each frame image in the spacing timing queue and the size information of the mobile phone frame, the width of the face frame and the diagonal of the mobile phone frame are taken with the same weight as the width distance corresponding to each frame image; the width distance of the spacing timing queue is weightedly calculated to obtain the distance threshold.

[0051] In this embodiment, each frame of image in the spacing timing queue records the length and width of the corresponding face frame, and the length and width of the mobile phone frame, from which the length of the diagonal of the mobile phone frame can be calculated, and the sum of the width of the face frame and the diagonal of the mobile phone frame is multiplied by 0.5 to obtain the width distance corresponding to each frame of image. Since the sizes of the face frame and the mobile phone frame in each frame of image are not necessarily exactly the same, the width distance obtained for each frame of image is not necessarily equal. Therefore, it is necessary to perform weighted averaging on each width distance in the spacing timing queue to obtain a distance threshold that can be used to measure the type of driver's mobile phone behavior. That is, the distance threshold can be used to determine whether the driver is making a phone call or playing with the phone.

[0052] In a specific embodiment of the present application, within a preset time period, the usage interval of each frame of the image acquired is sorted to obtain a spacing timing queue, including: subtracting the timestamp of the last frame of the image corresponding to the usage interval initially acquired in the image sorting from the timestamp of the latest frame of the image acquired, to obtain a time period result; comparing and judging the time period result with the preset time period, if the time period result is not equal to the preset time period, continue to count the next frame of the image, or discard the last frame of the image, until the time period result is equal to the preset time period, thereby obtaining a spacing timing queue within the preset time period.

[0053] In this embodiment, in the current sorting process, the timestamp of the latest frame of the image being sorted is subtracted from the timestamp of the last frame of the image that has been sorted. Based on the obtained time period result, it is checked whether the duration of the spacing timing queue is within the preset time period. If the duration of the spacing timing queue is less than or exceeds the preset time period, the next frame of the image is sorted or the last frame of the image is filtered out. Only when the duration of the timing queue is equal to the preset time period, the spacing timing queue within the preset time period is determined to facilitate improving accuracy.

[0054] exist Figure 1 In the specific implementation shown, a method for detecting the use of a mobile phone while driving further includes:

[0055] Step S105, compare and judge the average value of the spacing and the distance threshold. If the average value of the spacing is greater than the distance threshold, and the face orientation result is that the face is facing the mobile phone, the driver's use of the mobile phone behavior is judged as playing with the mobile phone, and an alarm is issued. If the average value of the spacing is not greater than the distance threshold, the driver's use of the mobile phone behavior is judged as making a phone call, and an alarm is issued.

[0056] In this embodiment, the distance threshold is used as a demarcation value to determine whether the driver's mobile phone use behavior is playing with the mobile phone or making a call. Using the distance threshold as a standard to independently determine the type of mobile phone use behavior can ensure the accuracy of the judgment.

[0057] In a specific embodiment of the present application, when the average value of the interval is greater than the distance threshold, if the face orientation result is not that the face is facing the mobile phone, the driver's other driving behaviors are judged, where the other driving behaviors do not include playing with the mobile phone and making calls.

[0058] In this embodiment, when the face is not facing the mobile phone, even if the average distance is greater than the distance threshold, the type of mobile phone use behavior will not be judged, and instead the driver's other driving behaviors will be alarmed, saving time for judging the driver's dangerous behavior.

[0059] In a specific embodiment of the present application, when the speed of the driver's vehicle is greater than a speed threshold, the driver's mobile phone usage behavior is re-alarmed at preset time intervals.

[0060] In this embodiment, the scheme is to distinguish the types of mobile phone usage behaviors of the driver during driving and give an alarm, so the speed of the driving vehicle only needs to be greater than 0. The different preset time intervals of the alarm make the distinction of the types of mobile phone usage behaviors and the alarm more timely and accurate.

[0061] The present application directly obtains the perception detection results of each frame of image through the perception model, which includes the face orientation results of the driver, the face frame detection results and the mobile phone frame detection results in each frame of image. These perception detection results are directly related to the judgment of the type of mobile phone use behavior. Direct acquisition can improve efficiency and lay the foundation for subsequent calculations; the directly obtained perception detection results are screened to filter out the images that can be used to judge the type of mobile phone use behavior, that is, each frame of image obtained contains a face frame, a mobile phone frame, and a ratio standard of the face frame and the mobile phone frame, which lays the foundation for subsequent calculations; the position of the mobile phone frame of the current frame image is determined by a preset number of frame images, thereby ensuring the accuracy of the mobile phone position and confirming that the current frame image contains the mobile phone frame; according to the current The face frame position in the previous frame image and the current mobile phone frame position can be used to obtain the distance between the face frame and the mobile phone frame, laying a solid foundation for subsequent judgment of whether the type of mobile phone usage behavior is making a call or playing with the phone; the usage interval of each frame image in the available images is sorted in chronological order according to the timestamp in the image to obtain the corresponding interval timing queue; the interval timing queue within the preset time period makes the judgment of mobile phone usage behavior more accurate. In order to ensure the accuracy of the judgment of the type of mobile phone usage behavior, it is necessary to calculate the average usage interval of the interval timing queue, that is, the interval average value, to lay the foundation for subsequent judgment; using the distance threshold as a standard to separately judge the type of mobile phone usage behavior can ensure the accuracy of the judgment of the type of mobile phone usage behavior.

[0062] In another specific embodiment of the present application, a method for warning of mobile phone use while driving includes: issuing a warning to the driver when the method for detecting mobile phone use while driving in any embodiment detects that the driver is using a mobile phone while driving.

[0063] In this embodiment, regardless of whether the driver uses the mobile phone to play with the mobile phone or to make a call, a warning reminder will be issued to the driver to inform the driver of the danger in a timely manner, and a voice alarm will be issued to indicate the type of behavior the driver is using the mobile phone, making this solution more accurate and timely.

[0064] Figure 2A specific implementation of a device for detecting mobile phone use while driving according to the present application is shown.

[0065] exist Figure 2 In the specific implementation shown, a device for detecting the use of a mobile phone while driving mainly includes:

[0066] A perception information acquisition module 201 is used to perform perception detection on the driver in each frame of the continuous frame images, and obtain a face orientation result, a face frame detection result, and a mobile phone frame detection result about the driver in each frame of the image;

[0067] A position determination module 202 is used to calculate the position change of the mobile phone frame of a preset number of frame images including the current frame image according to the mobile phone frame detection result, and determine the current mobile phone frame position of the current frame image when the position change of the mobile phone frame is less than the position change threshold;

[0068] The distance calculation module 203 is used to calculate the distance between the face frame and the mobile phone frame in the current frame image according to the face frame detection result and the current mobile phone frame position to obtain the use distance;

[0069] The data processing module 204 is used to sort the used spacing of each frame of the acquired image within a preset time period to obtain a spacing timing queue, and perform weighted calculation on the used spacing in the spacing timing queue to obtain the spacing average value of the spacing timing queue, and determine the distance threshold according to the face frame detection result and the mobile phone frame detection result corresponding to each frame of the image in the spacing timing queue;

[0070] The alarm judgment module 205 is used to compare and judge the average value of the interval with the distance threshold. If the average value of the interval is greater than the distance threshold, and the face orientation result is that the face is facing the mobile phone, the driver's use of the mobile phone behavior is judged as playing with the mobile phone, and an alarm is issued. If the average value of the interval is not greater than the distance threshold, the driver's use of the mobile phone behavior is judged as making a phone call, and an alarm is issued.

[0071] In this embodiment, the perception information acquisition module 201 directly obtains the perception detection results of the perception model on the image, improves efficiency, and lays the foundation for subsequent calculations; the position determination module 202 determines the position of the mobile phone frame of the current frame image through a preset number of frame images, ensures the accuracy of the mobile phone position, and confirms that the current frame image contains the mobile phone frame; the spacing calculation module 203 can obtain the distance between the face frame and the mobile phone frame according to the position of the face frame in the current frame image and the current mobile phone frame position, laying a solid foundation for the subsequent judgment of whether the type of mobile phone use behavior is to make a phone call or play with the phone; the data processing module 204 sorts the use spacing of each frame image in the available image in order according to the timestamp in the image to obtain the corresponding spacing time series queue; the spacing time series queue within the preset time period makes the judgment of the use of the mobile phone behavior more accurate. In order to ensure the accuracy of the judgment of the type of mobile phone use behavior, it is necessary to calculate the average value of the use spacing of the spacing time series queue, that is, the spacing average value, to lay the foundation for subsequent judgment; the alarm judgment module 205 uses the distance threshold as a standard to judge the type of mobile phone use behavior separately, which can ensure the accuracy of the judgment of the type of mobile phone use behavior.

[0072] The present application provides a device for detecting the use of a mobile phone while driving, which can be used to execute the method for detecting the use of a mobile phone while driving described in any of the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0073] In a specific embodiment of the present application, each functional module in the device for detecting mobile phone use while driving proposed in the present application may be directly in hardware, in a software module executed by a processor, or in a combination of the two.

[0074] The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from and write information to the storage medium.

[0075] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In an alternative, the storage medium may be integrated with the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0076] In another specific embodiment of the present application, a vehicle includes a device for detecting the use of a mobile phone while driving in any embodiment. Optionally, the vehicle includes a processor and a memory, the processor and the memory are coupled, and the vehicle is used to implement the attached description of the present application. Figure 1 A method for detecting the use of a mobile phone while driving in any of the embodiments shown.

[0077] In another specific embodiment of the present application, a computer-readable storage medium stores computer instructions, and when the computer instructions are executed, the computer executes the method for detecting the behavior of using a mobile phone while driving in any embodiment.

[0078] In another specific embodiment of the present application, a computer device includes a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the method for detecting the behavior of using a mobile phone while driving in any embodiment is implemented.

[0079] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0080] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0081] The above description is only an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structural transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting the use of a mobile phone while driving, characterized in that: include: Performing perception detection on the driver in each frame of the continuous frame images to obtain a face orientation result, a face frame detection result, and a mobile phone frame detection result of the driver in each frame of the image; According to the mobile phone frame detection result, the mobile phone frame position change of a preset number of frame images including the current frame image is calculated, and when the mobile phone frame position change is less than the position change threshold, the current mobile phone frame position of the current frame image is determined; According to the face frame detection result and the current mobile phone frame position, the distance between the face frame and the mobile phone frame in the current frame image is calculated to obtain a use spacing; Within a preset time period, the used spacing of each frame of the acquired image is sorted to obtain a spacing timing queue, and the used spacing in the spacing timing queue is weightedly calculated to obtain the spacing average value of the spacing timing queue, and the distance threshold is determined according to the face frame detection result and the mobile phone frame detection result corresponding to each frame of the image in the spacing timing queue; The average distance value is compared with the distance threshold. If the average distance value is greater than the distance threshold and the face orientation result is that the face is facing the mobile phone, the driver's use of the mobile phone behavior is determined to be playing with the mobile phone. If the average distance value is not greater than the distance threshold, the driver's use of the mobile phone behavior is determined to be making a phone call.

2. The method for detecting the use of a mobile phone while driving according to claim 1, characterized in that: According to the face frame detection results and the mobile phone frame detection results, images in which the face frame is missing, the mobile phone frame is missing, and the size of the mobile phone frame is greater than a size threshold are filtered out to obtain the current frame image containing a standard face frame and a standard mobile phone frame.

3. The method for detecting the use of a mobile phone while driving as claimed in claim 2, characterized in that: The determining of the distance threshold according to the face frame detection result and the mobile phone frame detection result corresponding to each frame image in the interval timing queue includes: The distance threshold is determined according to size information of a face frame corresponding to each frame of image in the interval timing queue and size information of a corresponding mobile phone frame.

4. The method for detecting the use of a mobile phone while driving as claimed in claim 3, characterized in that: The determining the distance threshold according to the size information of the face frame corresponding to each frame image in the interval timing queue and the size information of the corresponding mobile phone frame includes: According to the size information of the face frame corresponding to each frame of the image in the spacing timing queue and the size information of the corresponding mobile phone frame, the sum of the width of the face frame and the diagonal of the mobile phone frame with the same weight is taken as the width distance corresponding to each frame of the image; The distance threshold is obtained by performing weighted calculation on the width distance of the interval timing queue.

5. The method for detecting the use of a mobile phone while driving as claimed in claim 1, characterized in that: The method of calculating the position change of the mobile phone frame of a preset number of frame images including the current frame image according to the mobile phone frame detection result, and determining the current mobile phone frame position of the current frame image when the position change of the mobile phone frame is less than a position change threshold, includes: According to the mobile phone frame position in the mobile phone frame detection results of the preset number of frames of images, a position change calculation is performed on the mobile phone frame position between every two adjacent frames of images to obtain the mobile phone frame position change amount; When the change amount of the mobile phone frame position is less than the position change threshold, the mobile phone frame position in the mobile phone frame detection result of the current frame image is used as the current mobile phone frame position.

6. The method for detecting the use of a mobile phone while driving according to claim 1, characterized in that: Also includes: When the average value of the interval is greater than the distance threshold, if the face orientation result is not that the face is facing the mobile phone, the driver's other driving behaviors are judged, where the other driving behaviors do not include playing with the mobile phone and making phone calls.

7. The method for detecting the use of a mobile phone while driving as claimed in claim 1, characterized in that: Also includes: When the speed of the driver's vehicle is greater than a speed threshold, the driver's mobile phone usage behavior is re-alerted at preset time intervals.

8. The method for detecting the use of a mobile phone while driving as claimed in claim 1, characterized in that: The method of sorting the usage intervals of each frame of the acquired image within the preset time period to obtain an interval timing queue includes: The timestamp of the latest frame of image corresponding to the usage interval of each frame of image obtained is subtracted from the timestamp of the last frame of image corresponding to the usage interval obtained initially in the image sorting to obtain a time period result; The time period result is compared with the preset time period. If the time period result is not equal to the preset time period, continue to count the next frame of image, or discard the last frame of image, until the time period result is equal to the preset time period, and obtain the spacing timing queue within the preset time period.

9. A method for warning of mobile phone use while driving, characterized in that: include: When the method according to any one of claims 1 to 8 detects that the driver uses a mobile phone while driving, a warning is issued to the driver.

10. A device for detecting mobile phone use while driving, characterized in that: include: A perception information acquisition module, which is used to perform perception detection on the driver in each frame of the continuous frame images, and obtain a face orientation result, a face frame detection result, and a mobile phone frame detection result about the driver in each frame of the image; A position determination module, which is used to calculate the position change of the mobile phone frame of a preset number of frame images including the current frame image according to the mobile phone frame detection result, and determine the current mobile phone frame position of the current frame image when the position change of the mobile phone frame is less than the position change threshold; A spacing calculation module, which is used to calculate the distance between the face frame and the mobile phone frame in the current frame image according to the face frame detection result and the current mobile phone frame position to obtain a use spacing; A data processing module, which is used to sort the used spacing of each frame of the acquired image within a preset time period to obtain a spacing timing queue, and perform weighted calculation on the used spacing in the spacing timing queue to obtain a spacing average value of the spacing timing queue, and determine a distance threshold according to the face frame detection result and the mobile phone frame detection result corresponding to each frame of the image in the spacing timing queue; The alarm judgment module is used to compare and judge the average value of the interval with the distance threshold. If the average value of the interval is greater than the distance threshold and the face orientation result is that the face is facing the mobile phone, the driver's use of the mobile phone behavior is judged as playing with the mobile phone; if the average value of the interval is not greater than the distance threshold, the driver's use of the mobile phone behavior is judged as making a phone call.

11. A vehicle, characterized in that: The vehicle includes the device for detecting the behavior of using a mobile phone while driving as claimed in claim 10.

12. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed, the computer executes the method for detecting the behavior of using a mobile phone while driving as described in any one of claims 1-8.

13. A computer device comprising a processor and a memory, wherein the memory stores computer instructions, characterized in that: When the computer instructions are executed by the processor, a method for detecting the behavior of using a mobile phone while driving as described in any one of claims 1-8 is implemented.

Images