A driving fatigue detection method, device and system integrating multiple characteristic signals

Abstract

The application relates to the technical field of driving monitoring, and particularly discloses a driving fatigue detection method, device and system integrating various characteristic signals, which comprises the following steps: receiving a fatigue monitoring request sent by a user, and obtaining access permission of a cab collection device; obtaining operation information of the user within a preset time range at a fixed time, and determining an abnormal value of the user; when the abnormal value reaches a preset abnormal threshold, obtaining social data of the user; correcting the abnormal value according to the social data; when the corrected abnormal value is still not lower than the abnormal threshold, obtaining monitoring data of the user based on the collection device; and determining the fatigue degree of the user according to the monitoring data. The application receives a detection request, obtains a detection permission, obtains various characteristic signals at different levels based on various collection devices, and performs hierarchical detection on the driving fatigue degree of the user according to the characteristic signals, so that the detection result is comprehensive, and the utilization rate of computing resources is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of driving monitoring, and particularly relates to a driving fatigue detection method, device and system integrating multiple characteristic signals. BACKGROUND

[0002] Driving fatigue refers to a phenomenon that physiological functions and psychological functions of a driver are out of balance and driving skills are objectively reduced after long-time continuous driving. The driver is prone to fatigue due to poor or insufficient sleep quality and long-time driving. Driving fatigue affects the driver's attention, feeling, perception, thinking, judgment, will, decision and movement.

[0003] It can be imagined that the fatigue degree of the driver is increasing with the increase of driving time, and the fatigue degree is increased due to some troubles, thereby affecting driving safety. Based on this, how to detect the fatigue degree of the driver is a technical problem to be solved by the technical scheme of the present application. SUMMARY

[0004] The present application aims to provide a driving fatigue detection method, device and system integrating multiple characteristic signals to solve the problems in the background.

[0005] To achieve the above object, the present application provides the following technical scheme.

[0006] A driving fatigue detection method integrating multiple characteristic signals, the method comprising:

[0007] receiving a fatigue monitoring request sent by a user and obtaining access permission of a driving cab collection device; the collection device comprises an intelligent mobile terminal;

[0008] timely obtaining operation information of the user within a preset time range, comparing the operation information with preset historical behavior information, and determining an abnormal value of the user according to a comparison result; wherein the historical behavior information is a function of the operation information;

[0009] comparing the abnormal value with a preset abnormal threshold value, obtaining social data of the user when the abnormal value reaches the preset abnormal threshold value; wherein the social data comprises text information and voice information;

[0010] correcting the abnormal value according to the social data, and obtaining monitoring data of the user based on the collection device when the corrected abnormal value is still not lower than the abnormal threshold value;

[0011] determining a fatigue degree of the user according to the monitoring data.

[0012] As a further scheme of the present application: the step of acquiring the operation information of the user within the preset time range, comparing the operation information with the preset historical behavior information, and determining the abnormal value of the user according to the comparison result comprises:

[0013] According to the access permission, the operation information is collected in the preset time range according to a preset collection frequency, and an operation table is obtained; the operation table comprises an operation type item and an operation frequency item; the operation type at least comprises the number of times of stepping on the accelerator and the number of times of stepping on the brake;

[0014] The operation table is input into the trained reversible analysis conversion model, and behavior information indexed by the time range is obtained; the behavior information is text data;

[0015] The preset historical behavior information is traversed according to the behavior information, and a similarity is calculated in real time;

[0016] The historical behavior information with a similarity greater than a preset similarity threshold is extracted, and the extracted historical behavior information is converted into a target operation table according to the reversible analysis conversion model;

[0017] The abnormal value of the user is calculated according to the target operation table.

[0018] As a further scheme of the present application: the step of comparing the abnormal value with a preset abnormal threshold and acquiring social data of the user when the abnormal value reaches the preset abnormal threshold comprises:

[0019] The abnormal value of the user is read, and the abnormal value is compared with a preset abnormal threshold;

[0020] When the abnormal value reaches the preset abnormal threshold, a social class App in the intelligent mobile terminal is located based on the access permission;

[0021] Social data containing time information in the social class App is acquired, and the social data is filtered in real time according to the time information.

[0022] As a further scheme of the present application: the step of correcting the abnormal value according to the social data, and acquiring monitoring data of the user based on the collection device when the corrected abnormal value is still not lower than the abnormal threshold comprises:

[0023] The filtered social data is read, the participants of the social data are counted, and the social data is classified according to the participants;

[0024] The social features of the same type of social data are calculated, the social features comprise the order of data senders and the data volume of each piece of social data; when the social data is voice information, the voice information is converted into text information;

[0025] determine the social flatness of the social data according to the social features;

[0026] statistically determine the social flatness of all social data, and calculate the user flatness;

[0027] correct the abnormal value according to the user flatness, and when the corrected abnormal value is still not lower than the abnormal threshold, acquire monitoring data of the user based on the acquisition device.

[0028] As a further scheme of the present application, the step of determining the fatigue degree of the user according to the monitoring data comprises:

[0029] acquire a plurality of frames of face images, and sequentially extract local texture features and depth features of each frame of face image;

[0030] concatenate and fuse the local texture features and the depth features to obtain fused features, and input the fused features into a trained classifier to classify to obtain micro-expressions corresponding to the fused features;

[0031] determine the fatigue degree of the frame of face image according to the micro-expression;

[0032] statistically determine the fatigue degrees of the plurality of frames of face images, and calculate the fatigue degree of the user.

[0033] As a further scheme of the present application, the method further comprises:

[0034] read the screened social data, and extract social data sent by the user in the social data;

[0035] perform type identification on the social data sent by the user, and when the social data contains an image, perform amplification correction on the fatigue degree.

[0036] The technical scheme of the present application further provides a driving fatigue detection system integrating a plurality of feature signals, the system comprising:

[0037] an access permission acquisition module configured to receive a fatigue monitoring request sent by a user, and acquire access permission of a cab acquisition device; the acquisition device comprises an intelligent mobile terminal;

[0038] an abnormal value calculation module configured to acquire operation information of the user within a preset time range at regular time intervals, compare the operation information with preset historical behavior information, and determine an abnormal value of the user according to a comparison result; wherein the historical behavior information is a function of the operation information;

[0039] The social data acquisition module is configured to compare the abnormal value with a preset abnormal threshold, and acquire social data of the user when the abnormal value reaches the preset abnormal threshold; wherein the social data comprises text information and voice information.

[0040] The monitoring data acquisition module is configured to correct the abnormal value according to the social data, and acquire monitoring data of the user based on the collection device when the corrected abnormal value is still not lower than the abnormal threshold.

[0041] The fatigue degree determination module is configured to determine the fatigue degree of the user according to the monitoring data.

[0042] As a further scheme of the present application, the abnormal value calculation module comprises:

[0043] The operation table determination unit is configured to acquire operation information according to a preset collection frequency in a preset time range based on the access permission, and obtain an operation table; the operation table comprises an operation type item and an operation number item; the operation type at least comprises accelerator pedal stepping number and brake pedal stepping number.

[0044] The behavior information generation unit is configured to input the operation table into a trained reversible analysis conversion model, and obtain behavior information indexed by time range; wherein the behavior information is text data.

[0045] The similarity calculation unit is configured to traverse preset historical behavior information according to the behavior information, and calculate a similarity in real time.

[0046] The reverse conversion unit is configured to extract historical behavior information with a similarity greater than a preset similarity threshold, and convert the extracted historical behavior information into a target operation table according to the reversible analysis conversion model.

[0047] The processing execution unit is configured to calculate the abnormal value of the user according to the target operation table.

[0048] As a further scheme of the present application, the social data acquisition module comprises:

[0049] The abnormal comparison unit is configured to read the abnormal value of the user, and compare the abnormal value with a preset abnormal threshold.

[0050] The App positioning unit is configured to position a social class App in the intelligent mobile terminal based on the access permission when the abnormal value reaches the preset abnormal threshold.

[0051] The social data screening unit is configured to acquire social data containing time information in the social class App, and screen the social data in real time according to time information.

[0052] The technical scheme of the present application also provides a driving fatigue detection device integrating multiple characteristic signals, and the Internet of Things data processing device comprises one or more processors and one or more memories, at least one program code is stored in the one or more memories, the program code is loaded and executed by the one or more processors to realize the driving fatigue detection method integrating multiple characteristic signals.

[0053] Compared with the prior art, the present application has the beneficial effects that the present application receives a detection request, obtains a detection permission, obtains different levels of multiple characteristic signals based on various acquisition devices, and performs hierarchical detection on the driving fatigue degree of a user according to the characteristic signals, so that the detection result is comprehensive and the utilization rate of computing resources is high. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application.

[0055] Figure 1 The flowchart of the driving fatigue detection method integrating multiple characteristic signals.

[0056] Figure 2 The first sub-flowchart of the driving fatigue detection method integrating multiple characteristic signals.

[0057] Figure 3 The second sub-flowchart of the driving fatigue detection method integrating multiple characteristic signals.

[0058] Figure 4 The third sub-flowchart of the driving fatigue detection method integrating multiple characteristic signals.

[0059] Figure 5 The fourth sub-flowchart of the driving fatigue detection method integrating multiple characteristic signals.

[0060] Figure 6 The composition structure block diagram of the driving fatigue detection system integrating multiple characteristic signals.

[0061] Figure 7 The composition structure block diagram of the abnormal value calculation module in the driving fatigue detection system integrating multiple characteristic signals.

[0062] Figure 8 The composition structure block diagram of the social data acquisition module in the driving fatigue detection system integrating multiple characteristic signals. DETAILED DESCRIPTION

[0063] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clear and explicit, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0064] Embodiment 1

[0065] Figure 1 For the flowchart of the driving fatigue detection method integrating various feature signals, in the embodiment of the present application, a driving fatigue detection method integrating various feature signals, the method comprises steps S100 to S500:

[0066] Step S100: receiving a fatigue monitoring request sent by a user, and obtaining access permission of a cab collection device; the collection device comprises a smart mobile terminal;

[0067] In the fatigue detection process of the user, a lot of data related to the user will be collected, and the confidentiality of these data is high, and certain permission is required to obtain the data.

[0068] Step S200: acquiring operation information of the user within a preset time range at regular intervals, comparing the operation information with preset historical behavior information, and determining an abnormal value of the user according to a comparison result; wherein the historical behavior information is a function of the operation information;

[0069] There are many operation information, for example, how many times the accelerator is stepped in one minute, how many times the brake is stepped, how many degrees the steering wheel is turned or how many times the music is switched, etc.; all of these can reflect the mood state of the user, and these data are in numerical form. By operating these numerical values, an abnormal value can be obtained.

[0070] Step S300: comparing the abnormal value with a preset abnormal threshold value, and acquiring social data of the user when the abnormal value reaches the preset abnormal threshold value; wherein the social data comprises text information and voice information;

[0071] After the abnormal value is determined, the abnormal value can be used as a subsequent data acquisition condition. When the abnormal value reaches a certain degree, it is considered that the user is likely to be fatigued; whether the user is fatigued or not is further determined by the social data of the user.

[0072] Step S400: correcting the abnormal value according to the social data, and acquiring monitoring data of the user based on the collection device when the corrected abnormal value is still lower than the abnormal threshold value.

[0073] The social data can be optional, and the social data can more truly reflect the user state, and the default is not changed if there is no social data; in addition, the correction process is mainly to reduce the abnormal value, if it is not changed, the abnormal value is still higher than the abnormal threshold, and then the monitoring data of the user is obtained.

[0074] Step S500: determining the fatigue degree of the user according to the monitoring data;

[0075] The monitoring data of the user is the data that can most reflect the user state, but it has high privacy, and more computing resources are required in the acquisition and processing process. Therefore, the technical scheme of the present application sets some conditions determined by other parameters before the acquisition process of the monitoring data.

[0076] Figure 2 The first sub-flow block diagram of the driving fatigue detection method integrating a plurality of feature signals is provided, and the steps of determining the abnormal value of the user according to the comparison result of the operation information and the preset historical behavior information include steps S201 to S205:

[0077] Step S201: based on the access permission, operation information is collected in a preset time range according to a preset collection frequency, and an operation table is obtained; the operation table includes an operation type item and an operation frequency item; the operation type at least includes accelerator pedal stepping frequency and brake pedal stepping frequency;

[0078] Step S202: inputting the operation table into a trained reversible analysis conversion model to obtain behavior information indexed by a time range; wherein the behavior information is text data;

[0079] Step S203: according to the behavior information, traversing the preset historical behavior information, and calculating the similarity in real time;

[0080] Step S204: extracting historical behavior information with a similarity greater than a preset similarity threshold, and converting the extracted historical behavior information into a target operation table according to the reversible analysis conversion model;

[0081] Step S205: calculating the abnormal value of the user according to the target operation table.

[0082] The operation table is a data table, which contains an operation type item and an operation frequency item, and it is difficult to compare tables with tables. Therefore, the operation table is converted into text data, that is, behavior information; the conversion process needs to be reversible, and the simplest way is to directly copy and connect to obtain behavior information, which is saved, and the saved behavior information can be obtained by statistical analysis.

[0083] The behavior information is compared with historical behavior information, and similarity is calculated in real time, similar historical behavior information is read, new behavior information collected is expanded, and the expanded behavior information is inversely converted into an operation table to obtain a target operation table; data in the target operation table is more sufficient, and the calculated abnormal value is more accurate.

[0084] Figure 3 For the second sub-flow chart of the driving fatigue detection method integrating various feature signals, the step of comparing the abnormal value with a preset abnormal threshold includes steps S301 to S303 when the abnormal value reaches the preset abnormal threshold:

[0085] Step S301: reading the abnormal value of the user, and comparing the abnormal value with a preset abnormal threshold;

[0086] Step S302: when the abnormal value reaches the preset abnormal threshold, positioning a social class App in the intelligent mobile terminal based on the access permission;

[0087] Step S303: obtaining social data containing time information in the social class App, and filtering the social data in real time according to the time information.

[0088] Steps S301 to S303 specifically describe the acquisition process of the social data, and the acquisition process is very easy and divided into two steps, namely positioning and filtering. The positioning process needs to have permission, and the user can stop at any time. The purpose of the filtering process is to extract social data that has just occurred, such as social data within 10 minutes or half an hour. Too long social data has little effect on the abnormal value.

[0089] Figure 4 For the third sub-flow chart of the driving fatigue detection method integrating various feature signals, the step of correcting the abnormal value according to the social data includes steps S401 to S405 when the corrected abnormal value is still not lower than the abnormal threshold:

[0090] Step S401: reading the filtered social data, counting the participants of the social data, and classifying the social data according to the participants;

[0091] Step S402: calculating the social characteristics of the same type of social data, the social characteristics including the order of the data sender and the data volume of each piece of social data; wherein, when the social data is voice information, the voice information is converted into text information;

[0092] Step S403: determining the social peace degree of the same type of social data according to the social characteristics;

[0093] Specifically, step S403 includes the following steps:

[0094] S4031, determine the sending-receiving end type of each piece of social data, and the data volume of each piece of social data, the sending-receiving end type including first end to second end and second end to first end;

[0095] S4032, according to the data volume of each piece of social data, the social data sentiment value corresponding to each piece of social data is determined;

[0096] Wherein, the calculation formula of the social data sentiment value corresponding to each piece of social data is:

[0097]

[0098] Wherein, e a→b represents the social data sentiment value of each piece of social data sent from the first end to the second end, γ represents the sentiment correction factor, represents the unit sentiment value corresponding to the i-th word in a single piece of social data sent from the first end to the second end, n1 represents the total number of words in a single piece of social data sent from the first end to the second end, n0 represents the reference word quantity in a single piece of social data, i∈(1, n1].

[0099] Similarly, there is the following formula:

[0100]

[0101] Wherein, e b→a represents the social data sentiment value of each piece of social data sent from the second end to the first end, represents the unit sentiment value corresponding to the j-th word in a single piece of social data sent from the second end to the first end, n2 represents the total number of words in a single piece of social data sent from the second end to the first end.

[0102] S4033, based on the social data sentiment value corresponding to each piece of social data, the social peace degree of this type of social data is calculated.

[0103] The calculation formula of the social peace degree of a single type of social data is:

[0104]

[0105] Wherein, p represents the social peace degree of a single type of social data, p0 represents the reference social peace degree of a single type of social data.

[0106] Step S404: statistics of the social peace degree of all types of social data, calculation of user peace degree;

[0107] Step S405: According to the user peace degree, the abnormal value is corrected, and when the corrected abnormal value is still lower than the abnormal threshold, the monitoring data of the user is acquired based on the acquisition device.

[0108] The above describes the analysis process of social data in detail. First, the number of people with whom the user socializes is counted. When driving, the user may socialize with multiple people at the same time, and the social data of different social partners needs to be classified. Of course, this classification process can occur in the data reading stage. Then, the social peace degree is determined according to the social characteristics, wherein the social characteristics are the interaction characteristics of the two parties sending messages. For example, A and B chat, A sends 10 data, and all of them are "long speeches". Then, B replies with a simple phrase such as "yes", "good", or a single expression, and the possibility of A's emotion will be high. Finally, the average social peace degree is calculated by counting all the social peace degrees between the user and different social partners, which is used as the user peace degree.

[0109] Figure 5 The fourth sub-flow chart of the driving fatigue detection method integrating multiple feature signals is shown. The step of determining the fatigue degree of the user according to the monitoring data includes steps S501 to S504:

[0110] Step S501: Acquire multiple frames of face images, and extract the local texture features and depth features of each frame of face image in sequence;

[0111] Step S502: Cascade the local texture features and the depth features to obtain fusion features, and input the fusion features into a trained classifier to classify to obtain the micro-expression corresponding to the fusion features;

[0112] Step S503: Determine the fatigue degree of the frame of face image according to the micro-expression;

[0113] Step S504: Count the fatigue degrees of multiple frames of face images, and calculate the fatigue degree of the user.

[0114] The purpose of the monitoring data is to acquire the face image of the user, and then perform micro-expression recognition on the face image. This technology is also mentioned in the prior art. According to the recognized micro-expression, the fatigue degree of the user can be determined.

[0115] It is worth mentioning that if the user is in an overly excited state, we also consider that the user belongs to fatigue driving.

[0116] As a preferred embodiment of the technical scheme of the present application, the method further comprises:

[0117] read the screened social data, extract the social data sent by the user in the social data;

[0118] The type of the social data sent by the user is identified, and when the social data contains an image, the fatigue degree is amplified.

[0119] It should be noted that if there is image data sent by the driver in the social data, the fatigue degree needs to be amplified. The fatigue degree obtained by the technical scheme of the present application is ultimately obtained, and some reminding measures will be formulated according to the fatigue degree. If the driver sends a picture during driving, he must be in an inattentive driving state. At this time, the fatigue degree is amplified, which can trigger the reminding measures more quickly.

[0120] Embodiment 2

[0121] Figure 6 The composition structure block diagram of the driving fatigue detection system integrating multiple feature signals is provided. In the embodiment of the present application, a driving fatigue detection system integrating multiple feature signals, the system 10 comprises:

[0122] The access permission acquisition module 11 is configured to receive a fatigue monitoring request sent by a user and acquire access permission of a cab collection device; the collection device comprises an intelligent mobile terminal;

[0123] The abnormal value calculation module 12 is configured to acquire operation information of the user within a preset time range at a time, compare the operation information with preset historical behavior information, and determine an abnormal value of the user according to a comparison result; wherein the historical behavior information is a function of the operation information;

[0124] The social data acquisition module 13 is configured to compare the abnormal value with a preset abnormal threshold value, and acquire social data of the user when the abnormal value reaches the preset abnormal threshold value; wherein the social data comprises text information and voice information;

[0125] The monitoring data acquisition module 14 is configured to correct the abnormal value according to the social data, and acquire monitoring data of the user based on the collection device when the corrected abnormal value is still lower than the abnormal threshold value;

[0126] The fatigue degree determination module 15 is configured to determine a fatigue degree of the user according to the monitoring data.

[0127] Figure 7 The composition structure block diagram of the abnormal value calculation module in the driving fatigue detection system integrating multiple feature signals is provided. The abnormal value calculation module 12 comprises:

[0128] The operation table determination unit 121 is configured to collect operation information according to a preset collection frequency in a preset time range based on the access permission, to obtain an operation table; the operation table includes an operation type item and an operation number item; the operation type at least includes accelerator pedal stepping number and brake pedal stepping number;

[0129] The behavior information generation unit 122 is configured to input the operation table into a trained reversible analysis conversion model, to obtain behavior information indexed by a time range; the behavior information is text data;

[0130] The similarity calculation unit 123 is configured to traverse preset historical behavior information according to the behavior information, and to calculate a similarity in real time;

[0131] The reverse conversion unit 124 is configured to extract historical behavior information with a similarity greater than a preset similarity threshold, and to convert the extracted historical behavior information into a target operation table according to the reversible analysis conversion model;

[0132] The processing execution unit 125 is configured to calculate an abnormal value of a user according to the target operation table.

[0133] Figure 8 The social data acquisition module 13 of the driving fatigue detection system integrating multiple feature signals includes:

[0134] The abnormality comparison unit 131 is configured to read an abnormal value of a user, and to compare the abnormal value with a preset abnormal threshold;

[0135] The App positioning unit 132 is configured to position a social App in an intelligent mobile terminal based on the access permission when the abnormal value reaches the preset abnormal threshold;

[0136] The social data screening unit 133 is configured to acquire social data containing time information in the social App, and to screen the social data in real time according to time information.

[0137] The functions of the driving fatigue detection method integrating multiple feature signals are completed by a computer device, the computer device includes one or more processors and one or more memories, at least one program code is stored in the one or more memories, the program code is loaded and executed by the one or more processors to realize the functions of the driving fatigue detection method integrating multiple feature signals.

[0138] The processor fetches instructions from the memory one by one, analyzes the instructions, and then completes corresponding operations according to the instructions to generate a series of control commands, so that the parts of the computer automatically, continuously and coordinately act as an organic whole, realize the input of the program, the input of the data and the operation and output the results. The arithmetic operation or logical operation generated in the process is completed by the operation unit; the memory includes a read-only memory (ROM), and the ROM is used to store a computer program, and a protection device is arranged outside the memory.

[0139] For example, the computer program can be divided into one or more modules, one or more modules are stored in the memory and executed by the processor to complete the present application. One or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal device.

[0140] Those skilled in the art can understand that the above description of the service device is only an example and does not constitute a limitation on the terminal device, and can include more or less components than the above description, or combine certain components or different components, for example, can include input and output devices, network access devices, buses and the like.

[0141] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The above processor is the control center of the terminal device, and connects each part of the terminal device through various interfaces and lines.

[0142] The memory can be used to store computer programs and / or modules, and the processor realizes various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as an information collection template display function, a product information publishing function, etc.), etc.; and the data storage area can store data created according to the use of the berth state display system (such as product information collection templates corresponding to different product categories, product information to be published by different product providers, etc.). In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0143] If the modules / units integrated in the terminal device are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the modules / units in the above-mentioned embodiment system can also be instructed by a computer program to related hardware to complete, and the computer program can be stored in a computer-readable storage medium. The computer program can realize the functions of the above-mentioned various system embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0144] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0145] The above merely describes the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application, and any equivalent structure or equivalent process conversion, or direct or indirect application in other related technical fields, which are made by using the content of the present application specification and drawings, are also included in the patent protection scope of the present application.

Claims

1. A driving fatigue detection method integrating a plurality of characteristic signals, characterized by, The method comprises: receiving a fatigue monitoring request sent by a user, and obtaining access permission of a cab collection device; the collection device comprises a smart mobile terminal; timely obtaining operation information of the user within a preset time range, comparing the operation information with preset historical behavior information, and determining an abnormal value of the user according to a comparison result; wherein the historical behavior information is a function of the operation information; comparing the abnormal value with a preset abnormal threshold value, and obtaining social data of the user when the abnormal value reaches the preset abnormal threshold value; wherein the social data comprises text information and voice information; correcting the abnormal value according to the social data, and obtaining monitoring data of the user based on the collection device when the corrected abnormal value is still not lower than the abnormal threshold value; determining a fatigue degree of the user according to the monitoring data; wherein the step of correcting the abnormal value according to the social data, and obtaining monitoring data of the user based on the collection device when the corrected abnormal value is still not lower than the abnormal threshold value comprises: reading screened social data, counting participants of the social data, and classifying the social data according to the participants; calculating social characteristics of the same type of social data, wherein the social characteristics comprise an order of data senders and data volume of each piece of social data; wherein when the social data is voice information, the voice information is converted into text information; determining social peace degrees of the same type of social data according to the social characteristics; counting the social peace degrees of all types of social data, and calculating a user peace degree; correcting the abnormal value according to the user peace degree, and obtaining monitoring data of the user based on the collection device when the corrected abnormal value is still not lower than the abnormal threshold value; wherein the step of determining the social peace degrees of the same type of social data according to the social characteristics comprises: determining a sender-receiver type of each piece of social data and data volume of each piece of social data, wherein the sender-receiver type comprises a first end to a second end and a second end to a first end; calculating a social data sentiment value corresponding to each piece of social data according to the data volume of each piece of social data; wherein a calculation formula of the social data sentiment value corresponding to each piece of social data is represented as: ； wherein, represents a social data sentiment value of each piece of social data sent from the first end to the second end, represents an emotion correction factor, represents a unit emotion value corresponding to the first i word in the single piece of social data sent from the first end to the second end, represents a total amount of words in the single piece of social data sent from the first end to the second end, represents a reference word amount in the single piece of social data. Similarly, there is the following formula: ； wherein, represents a social data sentiment value of each piece of social data sent by the second end to the first end, represents a unit sentiment value corresponding to the first j word in a single piece of social data sent by the second end to the first end, represents a total amount of words in a single piece of social data sent by the second end to the first end; based on the social data sentiment value corresponding to each piece of social data, calculating a social peace degree of the same type of social data; a calculation formula of the social peace degree of single type of social data is represented as: ； wherein, represents a social flatness of the single-class social data, represents a reference social flatness of the single-class social data; wherein the step of counting the social peace degrees of all types of social data, and calculating a user peace degree comprises: counting the social peace degrees of all types of social data, and calculating an average social peace degree as the user peace degree.

2. The driving fatigue detection method integrating various feature signals according to claim 1, characterized by, the step of timely obtaining operation information of the user within a preset time range, comparing the operation information with preset historical behavior information, and determining an abnormal value of the user according to a comparison result comprises: According to the access permission, operation information is collected according to a preset collection frequency within a preset time range to obtain an operation table; the operation table includes an operation type item and an operation frequency item; the operation type at least includes accelerator pedal stepping frequency and brake pedal stepping frequency; The operation table is input into a trained reversible analysis conversion model to obtain behavior information indexed by time range; the behavior information is text data; According to the behavior information, preset historical behavior information is traversed to calculate a similarity in real time; The historical behavior information with a similarity greater than a preset similarity threshold is extracted, and the extracted historical behavior information is converted into a target operation table according to the reversible analysis conversion model; According to the target operation table, an abnormal value of the user is calculated.

3. The driving fatigue detection method integrating various feature signals according to claim 1, characterized by, When the abnormal value reaches the preset abnormal threshold, the step of obtaining social data of the user includes: The abnormal value of the user is read, and the abnormal value is compared with the preset abnormal threshold; When the abnormal value reaches the preset abnormal threshold, a social class App in the intelligent mobile terminal is located based on the access permission; Social data containing time information in the social class App is obtained, and the social data is filtered in real time according to the time information.

4. The driving fatigue detection method integrating various feature signals according to claim 1, characterized by, The step of determining the fatigue degree of the user according to the monitoring data includes: A plurality of face images are obtained, and local texture features and depth features of each face image are extracted in sequence; The local texture features and the depth features are cascaded and fused to obtain fusion features, and the fusion features are input into a trained classifier for classification to obtain micro expressions corresponding to the fusion features; According to the micro expressions, the fatigue degree of the frame face image is determined; The fatigue degrees of the plurality of face images are counted, and the fatigue degree of the user is calculated.

5. The driving fatigue detection method integrating various feature signals according to claim 1, characterized by, The method further includes: The filtered social data is read, and social data sent by the user in the social data is extracted; When the social data contains images, the fatigue degree is amplified and corrected.

6. A driving fatigue detection system integrating a plurality of characteristic signals, characterized by, The system includes: An access permission acquisition module is configured to receive a fatigue monitoring request sent by a user and acquire an access permission of a cab collection device; the collection device includes an intelligent mobile terminal; An abnormal value calculation module is configured to acquire operation information of the user within a preset time range at a preset time, compare the operation information with preset historical behavior information, and determine an abnormal value of the user according to a comparison result; the historical behavior information is a function of the operation information; A social data acquisition module is configured to compare the abnormal value with a preset abnormal threshold, and obtain social data of the user when the abnormal value reaches the preset abnormal threshold; the social data includes text information and voice information; A monitoring data acquisition module is configured to correct the abnormal value according to the social data, and acquire monitoring data of the user based on the collection device when the corrected abnormal value is still lower than the abnormal threshold; A fatigue degree determination module is configured to determine a fatigue degree of the user according to the monitoring data. The abnormal value is corrected according to the social data, and when the corrected abnormal value is still not lower than the abnormal threshold, monitoring data of the user is acquired based on the acquisition device. Read the screened social data, count the participants of the social data, and classify the social data according to the participants; Calculate the social characteristics of the same type of social data, including the order of the data sender and the data volume of each piece of social data; wherein, when the social data is voice information, the voice information is converted into text information; Determine the social peace degree of the same type of social data according to the social characteristics; Count the social peace degrees of all types of social data and calculate the user peace degree; Correct the abnormal value according to the user peace degree, and when the corrected abnormal value is still not lower than the abnormal threshold, acquire monitoring data of the user based on the acquisition device; The determination of the social peace degree of the same type of social data according to the social characteristics specifically includes: Determine the sender-receiver type of each piece of social data and the data volume of each piece of social data, wherein the sender-receiver type includes the first end to the second end and the second end to the first end; Calculate the social data sentiment value corresponding to each piece of social data according to the data volume of each piece of social data; The calculation formula of the social data sentiment value corresponding to each piece of social data is as follows: ； wherein, represents a social data sentiment value of each piece of social data sent from the first end to the second end, represents an emotion correction factor, represents a unit emotion value corresponding to the i-th word in the single piece of social data sent from the first end to the second end, i represents a total amount of words in the single piece of social data sent from the first end to the second end, represents a reference word amount in the single piece of social data, and represents a reference word amount in the single piece of social data. Similarly, the following formula exists: ； wherein, represents a social data sentiment value of each piece of social data sent by the second end to the first end, represents a unit sentiment value corresponding to the first j word in the single piece of social data sent by the second end to the first end, represents a total amount of words in the single piece of social data sent by the second end to the first end; Calculate the social peace degree of the same type of social data based on the social data sentiment value corresponding to each piece of social data; The calculation formula of the social peace degree of single type of social data is as follows: ； wherein, represents a social flatness of the single-class social data, represents a reference social flatness of the single-class social data; The calculation of the social peace degree of all types of social data and the calculation of the user peace degree specifically includes: Count the social peace degrees of all types of social data and calculate the average social peace degree, and take the average social peace degree as the user peace degree.

7. The driving fatigue detection system integrating various characteristic signals according to claim 6, characterized in that, The abnormal value calculation module includes: An operation table determination unit is configured to acquire operation information based on the access permission according to a preset acquisition frequency within a preset time range to obtain an operation table; the operation table includes an operation type item and an operation frequency item; the operation type at least includes the number of times of stepping on the accelerator and the number of times of stepping on the brake; A behavior information generation unit is configured to input the operation table into a trained reversible analysis conversion model to obtain behavior information indexed by a time range; wherein, the behavior information is text data; A similarity calculation unit is configured to traverse preset historical behavior information according to the behavior information and calculate the similarity in real time; A reverse conversion unit is configured to extract historical behavior information with a similarity greater than a preset similarity threshold, and convert the extracted historical behavior information into a target operation table according to the reversible analysis conversion model; A processing execution unit is configured to calculate the abnormal value of the user according to the target operation table.

8. The driving fatigue detection system integrating various characteristic signals according to claim 7, characterized in that, The social data acquisition module includes: An abnormal comparison unit is configured to read the abnormal value of the user and compare the abnormal value with a preset abnormal threshold; An App positioning unit is configured to locate a social class App in the intelligent mobile terminal based on the access permission when the abnormal value reaches the preset abnormal threshold; The social data screening unit is configured to obtain social data containing time information in the social application, and to screen the social data in real time according to the time information.

9. A driving fatigue detection device integrating a plurality of characteristic signals, characterized by The driving fatigue detection device integrating multiple feature signals comprises one or more processors and one or more memories, the one or more memories store at least one program code, the program code is loaded and executed by the one or more processors to realize the driving fatigue detection method integrating multiple feature signals according to any one of claims 1 to 5.

Images