Vehicle accelerator pedal monitoring method, vehicle-mounted terminal, auxiliary driving system and medium

Abstract

The present invention provides a vehicle accelerator pedal monitoring method, an on-board terminal, an assisted driving system, and a medium. The method first acquires multiple physiological information of the driver in real time; the multiple physiological information includes thermal imaging data, facial video data, pressure data, and accelerator pedal force data. Next, the method extracts characteristic values ​​of the multiple physiological information at each moment to obtain a driving characteristic vector at each moment. An environmental impact vector is then determined based on the driving characteristic vector during a preset period before the current moment. The driver's braking demand is then determined based on the driving characteristic vector and the environmental impact vector. Finally, an accelerator pedal mis-depression monitoring result is determined based on the driver's braking demand and force data. By calculating the environmental impact on the driver over a period of time and then combining the driver's driving characteristics with the environmental impact on the driving characteristics, the method accurately determines the driver's braking demand, thus avoiding false alarms or incorrect braking.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and in particular relates to a vehicle accelerator pedal monitoring method, a vehicle-mounted terminal, an auxiliary driving system and a medium. Background Art

[0002] When a driver encounters an emergency, they may panic to a certain extent. If they had stepped on the brake pedal in time, they could have avoided or reduced the damage caused by the accident. However, in a panic, the driver may have mistakenly stepped on the accelerator pedal, causing a traffic accident or even serious consequences.

[0003] Prior art vehicles often incorporate driver assistance systems that assess the driver's intent to warn the driver before danger strikes or directly control the vehicle, thereby preventing accidents. However, driver intent is often influenced by multiple factors. Simply capturing facial images or recognizing voice often fails to effectively identify the driver's intent, leading to frequent false warnings and braking by the driver assistance system. Summary of the Invention

[0004] In view of this, the present invention provides a vehicle accelerator pedal monitoring method, a vehicle-mounted terminal, an assisted driving system and a medium, aiming to solve the problem that the existing technology cannot effectively identify the driver's intention.

[0005] A first aspect of an embodiment of the present invention provides a vehicle accelerator pedal monitoring method, comprising:

[0006] Acquire multiple physiological information of the driver in real time, including thermal imaging data, driver's facial video data, cabin pressure data, and accelerator pedal force data;

[0007] Extract the characteristic values ​​of various physiological information at each moment and obtain the driving characteristic vector at each moment;

[0008] Determining an environmental impact vector based on a driving feature vector within a preset period before the current moment;

[0009] Determining the driver's braking demand based on the driving characteristic vector and the environmental impact vector;

[0010] The accelerator pedal mis-pressing monitoring result is determined based on the driver's braking demand and the accelerator pedal force data.

[0011] A second aspect of an embodiment of the present invention provides a vehicle accelerator pedal monitoring device, comprising:

[0012] A data acquisition module is used to obtain various physiological information of the driver in real time; wherein the various physiological information includes: thermal imaging data, driver's facial video data, cabin pressure data, and accelerator pedal force data;

[0013] The feature extraction module is used to extract the characteristic values ​​of various physiological information at each moment and obtain the driving feature vector at each moment;

[0014] An environmental assessment module is used to determine an environmental impact vector based on a driving feature vector within a preset period before the current moment;

[0015] a demand determination module, for determining the driver's braking demand based on the driving characteristic vector and the environmental impact vector;

[0016] The pedal monitoring module is used to determine the accelerator pedal mis-stepping monitoring result based on the driver's braking demand and the accelerator pedal force data.

[0017] A third aspect of an embodiment of the present invention provides a vehicle-mounted terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the vehicle accelerator pedal monitoring method of the first aspect above are implemented.

[0018] A fourth aspect of an embodiment of the present invention provides an assisted driving system, comprising: an infrared thermal imager, at least one video camera, at least one pressure sensor, an accelerator pedal control device, and the vehicle-mounted terminal of the first aspect above.

[0019] A fifth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the vehicle accelerator pedal monitoring method of the first aspect above are implemented.

[0020] The embodiments of the present invention provide a vehicle accelerator pedal monitoring method, on-board terminal, assisted driving system, and medium. The method first acquires multiple physiological information of the driver in real time. The multiple physiological information includes thermal imaging data, driver's facial video data, cabin pressure data, and accelerator pedal force data. The method then extracts characteristic values ​​of the multiple physiological information at each moment to obtain a driving characteristic vector at each moment. The method then determines an environmental impact vector based on the driving characteristic vector within a preset time period before the current moment. The method then determines the driver's braking demand based on the driving characteristic vector and the environmental impact vector. Finally, the method determines the accelerator pedal mis-depression monitoring result based on the driver's braking demand and accelerator pedal force data. The method processes the driving characteristic vectors over a period of time to obtain an environmental impact vector, determines the impact of the environment on the driver during this period, and then combines the driver's driving characteristics with the impact of the environment on the driving characteristics to accurately determine the driver's braking demand, thereby avoiding false alarms or incorrect braking. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a diagram of an application scenario of the vehicle accelerator pedal monitoring method provided by an embodiment of the present invention;

[0023] Figure 2 This is a flow chart of an implementation method of a vehicle accelerator pedal monitoring method provided by an embodiment of the present invention;

[0024] Figure 3 1 is a schematic structural diagram of a vehicle accelerator pedal monitoring device provided by an embodiment of the present invention;

[0025] Figure 4 It is a structural diagram of the vehicle-mounted terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0027] Figure 1This is an application scenario diagram of the vehicle accelerator pedal monitoring method provided by an embodiment of the present invention. Figure 1 As shown, in some embodiments, the vehicle accelerator pedal monitoring method provided by the embodiments of the present invention can be applied to, but is not limited to, this application scenario. In this embodiment of the invention, the system includes: an infrared thermal imager 11, at least one video camera 12, at least one pressure sensor 13, an accelerator pedal control device 14, and an onboard terminal 15.

[0028] Among them, the infrared thermal imager 11 is used to capture thermal imaging data of the driver, at least one video camera 12 is set up around the cab to capture video data of the driver's face, at least one pressure sensor 13 is set up on the driver's seat back, steering wheel, etc. to monitor pressure data on the seat back, steering wheel, etc. in the cab, and the accelerator pedal control device 14 is used to collect force data on the accelerator pedal. The video cameras 12 are set up at different locations in the cab to capture frontal facial video data of the driver in different postures (such as looking at the rearview mirror, the instrument panel, facing forward, etc.). The in-vehicle terminal 15 can be a single-chip microcomputer, ECU, MCU, etc., without limitation here.

[0029] After completing the above data collection, the infrared thermal imager 11, at least one video camera 12, at least one pressure sensor 13, and the accelerator pedal control device 14 report the collected data to the vehicle terminal 15. The vehicle terminal 15 then determines the result of the accelerator pedal misoperation monitoring based on the received data. If the misoperation monitoring result is misoperation, the vehicle terminal 15 instructs the accelerator pedal control device 14 to change the accelerator pedal's function from acceleration to braking.

[0030] Figure 2 FIG. 1 is a flow chart of the implementation of the vehicle accelerator pedal monitoring method provided by an embodiment of the present invention. Figure 2 As shown, in some embodiments, the vehicle accelerator pedal monitoring method is applied to Figure 1 The vehicle terminal 15 shown in FIG. 1 includes:

[0031] S210, acquiring various physiological information of the driver in real time; wherein the various physiological information includes: thermal imaging data, driver's facial video data, cab pressure data, and accelerator pedal force data.

[0032] In the embodiment of the present invention, the various physiological information may also include voice information, heart rate, blood pressure and other vital signs information.

[0033] The vehicle-mounted terminal 15 of the present invention may also be provided with a habit analysis module for acquiring the driver's driving information and analyzing the driver's driving habits. Drivers with different driving habits often exhibit different behaviors when they accidentally step on the accelerator pedal, such as gripping the steering wheel tightly or yelling. Therefore, based on the driver's driving habits, physiological information that is more consistent with the driver's habits can be selected from a variety of physiological information for analysis. This allows the determination of the accelerator pedal accidental stepping monitoring result to be determined with less useful data, eliminating the need to analyze information collected by all sensors / cameras.

[0034] S220 , extracting characteristic values ​​of various physiological information at each moment to obtain a driving characteristic vector at each moment.

[0035] In the embodiments of the present invention, the feature value extraction of multiple physiological information is to convert the physiological information into dimensionless values ​​of the same order of magnitude. For example, if the pressure sensor on the steering wheel detects a pressure of 215N and the pupil size is 3.1mm, the corresponding feature values ​​can be extracted as 2.15 and 3.1, and the corresponding driving feature vector is (2.15, 3.1). The above examples are only for illustration of the present invention and are not intended to be limiting.

[0036] S230 , determining an environmental impact vector based on the driving feature vector within a preset period before the current moment.

[0037] In this embodiment of the present invention, the pre-set period can be the previous hour, the previous two hours, etc., and is not limited here. During a relatively short event period, the driver's environment generally does not change significantly. Accordingly, the portion of the driving feature vector affected by environmental factors is relatively fixed. Therefore, operations such as averaging and taking the mode do not affect the portion of the driving feature vector affected by environmental factors and can mitigate accidental variations in the driving feature vector caused by other factors. In other words, the resulting environmental impact vector represents the portion of the driving feature vector primarily affected by environmental factors.

[0038] In some embodiments, S230 may include: calculating an average value of the driving feature vectors within a preset period to obtain an environmental impact vector.

[0039] In the embodiment of the present invention, since the driving environment is basically unchanged in a short period of time, by calculating the average value of the driving characteristic vector within a preset time period, the changes in the driving characteristic vector caused by other factors can be eliminated to obtain the driving characteristic vector affected by the environment, that is, the environmental impact vector.

[0040] In some embodiments, S230 may include: calculating the average value of the driving feature vector within a preset time period to obtain an average feature vector; obtaining the current driving condition; obtaining a preset standard environment vector corresponding to the current driving condition; and determining the environmental impact vector based on the average feature vector and the preset standard environment vector.

[0041] In the embodiment of the present invention, driving conditions may include low-speed driving, high-speed driving, night driving, low-visibility driving, etc., which are not limited here. The preset standard environment vector is the influence vector of the environment on the driving characteristic vector obtained through a large number of actual vehicle test experiments under each driving condition. If the driving characteristic vector at time t within the preset period T is Then the average eigenvector is The preset standard environment vector under the current driving condition is S i , where i is the category number of the driving condition, then the impact of environmental factors on the driving characteristic vector S can be obtained a :

[0042] S240 : Determine the driver's braking demand based on the driving characteristic vector and the environmental impact vector.

[0043] In an embodiment of the present invention, by subtracting the environmental impact vector from the driving feature vector, the portion of the driving feature vector affected by environmental factors can be removed, and the driving feature vector change caused by the emergency can be obtained, thereby determining the driver's braking demand.

[0044] In the embodiment of the present invention, S240 can be specifically implemented through mathematical statistics algorithms, neural network models, reinforcement learning models, etc., which are not limited here.

[0045] For example, if the driving feature vector is The environmental impact vector is S a , we can get the driving feature vector change ΔS caused by the emergency: Then we can calculate exist When the preset threshold is exceeded, it indicates that the driving feature vector is mainly affected by the emergency. At this time, the braking demand is necessary, otherwise it is not necessary. In order to avoid the influence of accidental errors, the data of multiple moments can be calculated simultaneously when judging the driver's braking demand to obtain multiple Multiple A weighted sum is performed and then compared with a preset threshold to determine the driver's braking demand.

[0046] S250: Determine an accelerator pedal mis-depression monitoring result based on the driver's braking demand and the accelerator pedal force data.

[0047] In an embodiment of the present invention, the driver's braking demand may include the need for braking and the need for not braking. Accordingly, when the force data of the accelerator pedal is 0, the accelerator pedal control device 14 outputs a low level, and when the force data of the accelerator pedal is not 0, the accelerator pedal control device 14 outputs a high level.

[0048] In the embodiment of the present invention, when the vehicle-mounted terminal 15 determines that the braking demand is necessary and receives a high level, it determines that the accelerator pedal's mis-stepping monitoring result is mis-stepping, otherwise it is not mis-stepping.

[0049] In an embodiment of the present invention, the driving characteristic vectors over a period of time are processed to obtain an environmental impact vector, and the impact of the environment on the driver during this period is determined. Then, the driver's driving characteristics and the impact of the environment on the driving characteristics are combined to accurately determine the driver's braking needs, thereby avoiding false warnings or incorrect braking.

[0050] In some embodiments, S240 may include: determining the driver's braking demand based on the driving feature vector, the environmental impact vector, and a pre-established neural network model.

[0051] In an embodiment of the present invention, physiological data of the driver can be collected under various driving environments, during normal driving (corresponding to braking demand, no braking required) and encountering emergencies (corresponding to braking demand, braking required), and the driving feature vector and the environmental impact vector can be calculated based on the above-mentioned S220 and S230 to form a training set for training the neural network.

[0052] In an embodiment of the present invention, after determining the driver's braking demand, the neural network will also output the confidence level of the braking demand, and then the threshold of the accelerator pedal's force data can be set in combination with the confidence level. For example, when the braking demand is that braking is required, the confidence level is 0.9, and the maximum pedaling force of the accelerator pedal (i.e., the force when it is stepped on to the bottom) is 30N, then the accelerator pedal's force threshold is set to 30*(1-0.9)=3N. When the accelerator pedal's force data is less than 3N, i.e., when it is lightly stepped on, no braking is performed, and when it exceeds 3N, braking is performed immediately. Therefore, by setting the accelerator pedal's force threshold based on the confidence level, erroneous braking caused by misjudgment can be reduced, and the vehicle can run normally when the accelerator pedal is lightly stepped on.

[0053] In some embodiments, before S240, the method also includes: obtaining the current driving condition; determining the current correlation between various physiological information under the current driving condition based on a pre-established knowledge graph; wherein the pre-established knowledge graph is used to represent the correlation relationship between various physiological information under each condition; and determining the coupling vector of various physiological information based on the current correlation between various physiological information.

[0054] Accordingly, S240 may include: determining the driver's braking demand according to the coupling vector, the driving characteristic vector, and the environmental impact vector.

[0055] In embodiments of the present invention, different physiological information often directly influence each other. Quantifying this interaction through a coupling vector facilitates analysis of the driver's braking demand. For example, when exiting a tunnel, a sudden change in light intensity can cause a sudden change in pupil size, leading to a subconscious tightening of the steering wheel. However, since there is no braking demand, this does not cause changes in physiological information such as seat back pressure and thermal imaging data. Therefore, a coupling vector between pupil size and steering wheel pressure can be set for tunnel exit conditions. This coupling vector can be removed from the driving feature vector when analyzing the driver's braking demand, thereby minimizing the impact of sudden changes in driving conditions on the driving feature vector and preventing misjudgment of braking demand.

[0056] In some embodiments, S220 may include: extracting characteristic values ​​of thermal imaging data to obtain the driver's body thermal imaging characteristic values ​​and brain thermal imaging characteristic values; extracting the driver's facial video data to obtain the driver's pupil size characteristic values, pupil change rate characteristic values, and blinking frequency characteristic values; extracting cab pressure data to obtain seat back pressure characteristic values ​​and steering wheel pressure characteristic values; determining the driving characteristic vector at each moment based on the body thermal imaging characteristic values, brain thermal imaging characteristic values, pupil size characteristic values, pupil change rate characteristic values, blinking frequency characteristic values, seat back pressure characteristic values, and steering wheel pressure characteristic values ​​at each moment.

[0057] In some embodiments, the monitoring result of the accelerator pedal's mis-stepping includes mis-stepping and normal stepping. Accordingly, after S250 , the method further includes: when the monitoring result of the accelerator pedal's mis-stepping is mis-stepping, switching the function of the accelerator pedal to a braking function.

[0058] In the embodiment of the present invention, the function switching of the accelerator pedal is achieved by the accelerator pedal control device 14 , and can be switched in the form of mechanical or electrical signals, which is not limited here.

[0059] In summary, the beneficial effects of the present invention are specifically as follows:

[0060] By processing the driving characteristic vectors over a period of time to obtain the environmental impact vector, the impact of the environment on the driver during this period is determined. Then, the driver's driving characteristics and the impact of the environment on the driving characteristics are combined to accurately determine the driver's braking needs and avoid false warnings or incorrect braking.

[0061] It should be understood that the order of execution of the steps in the above embodiments does not necessarily mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0062] Figure 3 FIG is a schematic diagram of the structure of a vehicle accelerator pedal monitoring device provided by an embodiment of the present invention. Figure 3 As shown, in some embodiments, the vehicle accelerator pedal monitoring device 3 includes:

[0063] The data acquisition module 310 is used to acquire various physiological information of the driver in real time; wherein the various physiological information includes: thermal imaging data, driver's facial video data, cab pressure data and accelerator pedal force data.

[0064] The feature extraction module 320 is used to extract the feature values ​​of various physiological information at each moment to obtain the driving feature vector at each moment.

[0065] The environment assessment module 330 is configured to determine an environment impact vector based on a driving feature vector within a preset period before the current moment.

[0066] The demand determination module 340 is configured to determine the driver's braking demand based on the driving characteristic vector and the environmental impact vector.

[0067] The pedal monitoring module 350 is used to determine the accelerator pedal mis-stepping monitoring result based on the driver's braking demand and the accelerator pedal force data.

[0068] Optionally, the environment assessment module 330 is specifically configured to calculate an average value of the driving feature vector within a preset period of time to obtain an environment impact vector.

[0069] Optionally, the environmental assessment module 330 is specifically used to calculate the average value of the driving feature vector within a preset time period to obtain an average feature vector; obtain the current driving condition; obtain the preset standard environmental vector corresponding to the current driving condition; and determine the environmental impact vector based on the average feature vector and the preset standard environmental vector.

[0070] Optionally, the demand determination module 340 is specifically configured to determine the driver's braking demand based on the driving feature vector, the environmental impact vector, and a pre-established neural network model.

[0071] Optionally, the vehicle accelerator pedal monitoring device 3 also includes: a coupling calculation module, used to obtain the current driving condition; based on a pre-established knowledge graph, determining the current correlation between various physiological information under the current driving condition; wherein the pre-established knowledge graph is used to represent the correlation relationship between various physiological information under each condition; based on the current correlation between various physiological information, determining the coupling vector of various physiological information; correspondingly, the demand determination module 340 is specifically used to determine the driver's braking demand based on the coupling vector, the driving characteristic vector, and the environmental impact vector.

[0072] Optionally, the feature extraction module 320 is specifically used to extract the characteristic values ​​of thermal imaging data to obtain the driver's body thermal imaging characteristic values ​​and brain thermal imaging characteristic values; extract the driver's facial video data to obtain the driver's pupil size characteristic values, pupil change rate characteristic values ​​and blinking frequency characteristic values; extract the cab pressure data to obtain the seat back pressure characteristic values ​​and steering wheel pressure characteristic values; determine the driving characteristic vector at each moment based on the body thermal imaging characteristic values, brain thermal imaging characteristic values, pupil size characteristic values, pupil change rate characteristic values, blinking frequency characteristic values, seat back pressure characteristic values ​​and steering wheel pressure characteristic values ​​at each moment.

[0073] Optionally, the monitoring result of the accelerator pedal's mis-stepping includes mis-stepping and normal stepping; accordingly, the vehicle accelerator pedal monitoring device 3 also includes: a function switching module, which is used to switch the function of the accelerator pedal to a braking function when the monitoring result of the accelerator pedal's mis-stepping is mis-stepping.

[0074] The vehicle accelerator pedal monitoring device provided in this embodiment can be used to execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0075] Figure 4 Schematic diagram of the structure of the vehicle terminal provided by the embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides a vehicle-mounted terminal 4, which includes: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, the steps in the above-mentioned vehicle accelerator pedal monitoring method embodiments are implemented, such as Figure 1 Alternatively, when the processor 40 executes the computer program 42, the functions of each module / unit in the above-mentioned system embodiments are realized, for example Figure 3 Functions of modules 310 to 350 are shown.

[0076] For example, the computer program 42 may be divided into one or more modules / units, one or more of which are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 42 in the vehicle-mounted terminal 4.

[0077] The vehicle-mounted terminal 4 may be a mobile phone, MCU, ECU, etc., which is not limited here. The vehicle-mounted terminal 4 may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that Figure 4It is only an example of the vehicle-mounted terminal 4 and does not constitute a limitation on the vehicle-mounted terminal 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the vehicle-mounted terminal may also include input and output devices, network access devices, buses, etc.

[0078] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0079] The memory 41 can be an internal storage unit of the vehicle-mounted terminal 4, such as the hard disk or memory of the vehicle-mounted terminal 4. The memory 41 can also be an external storage device of the vehicle-mounted terminal 4, such as a plug-in hard disk equipped on the vehicle-mounted terminal 4, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 41 can also include both the internal storage unit of the vehicle-mounted terminal 4 and an external storage device. The memory 41 is used to store computer programs and other programs and data required by the vehicle-mounted terminal. The memory 41 can also be used to temporarily store data that has been output or is about to be output.

[0080] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned vehicle accelerator pedal monitoring method embodiment are implemented.

[0081] The computer-readable storage medium stores a computer program 42, which includes program instructions. When the program instructions are executed by the processor 40, all or part of the process of the method in the above embodiment is implemented. The computer program 42 can also be used to instruct related hardware to complete the process. The computer program 42 can be stored in a computer-readable storage medium. When the computer program 42 is executed by the processor 40, it can implement the steps of each of the above method embodiments. Among them, the computer program 42 includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0082] The computer-readable storage medium may be an internal storage unit of the vehicle-mounted terminal of any of the aforementioned embodiments, such as a hard disk or memory of the vehicle-mounted terminal. The computer-readable storage medium may also be an external storage device of the vehicle-mounted terminal, such as a plug-in hard disk equipped on the vehicle-mounted terminal, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the vehicle-mounted terminal and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the vehicle-mounted terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0083] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0085] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0086] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0087] In the embodiments provided by the present invention, it should be understood that the disclosed devices / vehicle terminals and methods can be implemented in other ways. For example, the device / vehicle terminal embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0089] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0090] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. Computer-readable media may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0091] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A vehicle accelerator pedal monitoring method, characterized in that: include: Acquire multiple physiological information of the driver in real time; wherein the multiple physiological information includes: thermal imaging data, driver's facial video data, cabin pressure data, and accelerator pedal force data; Extracting characteristic values ​​of the multiple physiological information at each moment to obtain a driving characteristic vector at each moment; Determining an environmental impact vector based on a driving feature vector within a preset period before the current moment; determining a driver's braking demand based on the driving characteristic vector and the environmental impact vector; determining an accelerator pedal mis-depression monitoring result according to the driver's braking demand and the accelerator pedal force data; Before determining the driver's braking demand based on the driving characteristic vector and the environmental impact vector, the method further includes: Get the current driving conditions; Determining the current correlation between various physiological information under the current driving condition based on a pre-established knowledge graph; wherein the pre-established knowledge graph is used to represent the correlation between various physiological information under each driving condition; Determining coupling vectors of various physiological information according to current correlations between the various physiological information; The determining the driver's braking demand according to the driving characteristic vector and the environmental impact vector includes: determining a driver's braking demand based on the coupling vector, the driving characteristic vector, and the environmental impact vector; The monitoring result of the accelerator pedal's mis-stepping includes mis-stepping and normal stepping; After determining the accelerator pedal mis-depression monitoring result, the method further includes: When the result of the mis-stepping monitoring of the accelerator pedal is mis-stepping, the function of the accelerator pedal is switched to a braking function.

2. The vehicle accelerator pedal monitoring method according to claim 1, characterized in that: Determining an environmental impact vector based on a driving feature vector within a preset period before the current moment includes: An average value of the driving characteristic vectors within the preset time period is calculated to obtain the environmental impact vector.

3. The vehicle accelerator pedal monitoring method according to claim 1, characterized in that: Determining an environmental impact vector based on a driving feature vector within a preset period before the current moment includes: Calculating an average value of the driving characteristic vectors within the preset time period to obtain an average characteristic vector; Get the current driving conditions; Obtaining a preset standard environment vector corresponding to the current driving condition; The environmental impact vector is determined according to the average feature vector and the preset standard environment vector.

4. The vehicle accelerator pedal monitoring method according to claim 1, characterized in that: The determining the driver's braking demand according to the driving characteristic vector and the environmental impact vector includes: The driver's braking demand is determined according to the driving characteristic vector, the environmental impact vector and a pre-established neural network model.

5. The vehicle accelerator pedal monitoring method according to claim 1, characterized in that: Extracting the characteristic values ​​of the multiple physiological information at each moment to obtain the driving characteristic vector at each moment includes: Extracting characteristic values ​​of the thermal imaging data to obtain characteristic values ​​of the driver's body and brain; Extracting the driver's facial video data to obtain the driver's pupil size characteristic value, pupil change rate characteristic value, and blink frequency characteristic value; Extracting the cab pressure data to obtain a seat back pressure characteristic value and a steering wheel pressure characteristic value; The driving feature vector at each moment is determined based on the body thermal imaging feature value, brain thermal imaging feature value, pupil size feature value, pupil change rate feature value, blink frequency feature value, seat back pressure feature value and steering wheel pressure feature value at each moment.

6. A vehicle-mounted terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the vehicle accelerator pedal monitoring method according to any one of claims 1 to 5 are implemented.

7. A driving assistance system, characterized in that: include: An infrared thermal imager, at least one video camera, at least one pressure sensor, an accelerator pedal control device, and the vehicle-mounted terminal as claimed in claim 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle accelerator pedal monitoring method according to any one of claims 1 to 5 are implemented.

Images