A driver-machine interaction method using a cockpit intelligent foot mat

Abstract

The present application relates to a kind of driver-machine interaction method using cockpit intelligent foot pad.The cockpit intelligent foot pad includes sensing unit, foot pad body, cover layer and data acquisition device;Multiple mounting holes are respectively arranged in the left foot sensing area and the right foot sensing area of foot pad body, and sensing unit is installed in mounting hole;Cover layer covers the surface of foot pad body.The driver-machine interaction method includes: defining the interface action corresponding to voltage waveform;Driver steps on intelligent foot pad to generate current;Processor identifies;Corresponding interface action is executed.Compared with prior art, the present application has intelligent foot pad taking single electrode working mode, the first friction layer and the second friction layer contact and separate friction to generate electricity, high sensitivity, the voltage signal generated is stable and intuitive, and the complexity of algorithm processing is reduced;Driver-machine interaction method uses intelligent foot pad to identify the action of driver, can execute the function set by foot action, and realizes the identification of the driving intention of driver and other advantages.

Description

TECHNICAL FIELD

[0001] The present application relates to vehicle accessories, in particular to an intelligent foot pad for a cockpit and a driver human-computer interaction method. BACKGROUND

[0002] With the continuous evolution of automotive technology, driver body part monitoring technology has become a key research direction in the field of automotive safety and user experience, contributing significantly to improving driving safety, intelligent driving experience, and human-vehicle interaction. Traditional monitoring methods mainly focus on the upper body of the driver, such as monitoring the face and overall posture, which helps to identify potential fatigue, distraction, and unsafe driving states. However, as the demand for a deeper understanding of driving behavior increases, monitoring of foot movements becomes increasingly important.

[0003] Although cameras are a common monitoring tool that can capture leg movements, they have problems such as high cost, strong light interference, weak privacy, and low accuracy, which restrict their widespread application. To compensate for the shortcomings of camera monitoring, foot pads are introduced as an innovative monitoring means. By integrating pressure sensors into the foot pad, the driver's foot movements can be monitored. Pressure sensing mechanisms include resistive, capacitive, and piezoelectric technologies. However, these sensing mechanisms also have some limitations in foot pressure monitoring, such as the need to use materials with piezoelectric effect and the need for external power supply, making the structure more complex.

[0004] Therefore, how to design a simple and compact intelligent foot pad for a cockpit is a technical problem that needs to be solved. SUMMARY

[0005] The purpose of the present application is to overcome the above-mentioned defects of the prior art and provide an intelligent foot pad for a cockpit and a driver human-computer interaction method.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] According to one aspect of the present application, an intelligent foot pad for a cockpit is provided, comprising a sensing unit, a foot pad body, a cover layer, and a data acquisition device. The sensing unit comprises a static shielding layer, an electrode layer, a first friction layer, and a second friction layer arranged in sequence. The electrode layer, the first friction layer, and the second friction layer are surrounded by a wrapping layer. The first friction layer and the second friction layer have different friction electrode sequences. The first friction layer and the second friction layer are connected to the data acquisition device. The foot pad body is divided into a left foot sensing area and a right foot sensing area. The left foot sensing area and the right foot sensing area are respectively provided with a plurality of mounting holes. The sensing unit is mounted in the mounting hole. The cover layer covers the surface of the foot pad body on which the sensing unit is mounted.

[0008] Preferably, the left foot sensing area is provided with 4*4 array arranged mounting holes; and the right foot sensing area is provided with 3*3 array arranged mounting holes.

[0009] Preferably, the electrode layer is made of conductive material; and the wrapping layer and the covering layer are made of insulating material.

[0010] Preferably, the first friction layer is made of conductive sponge with a thickness of 15 mm; the sensing unit arranged in the left foot sensing area has an area of 4*4 cm 2 ; and the sensing unit arranged in the right foot sensing area has an area of 2*2 cm 2 .

[0011] Preferably, the sensing unit further comprises a substrate layer made of insulating material and located between the electrostatic shielding layer and the foot pad body.

[0012] According to another aspect of the present application, a driver-machine interaction method using the intelligent foot pad for cockpit is provided, which specifically comprises the following steps:

[0013] Step S1, defining interface actions corresponding to voltage waveforms;

[0014] Step S2, the driver performs a set action on the left foot sensing area with the left foot, and the sensing unit of the left foot sensing area generates current; and the driver performs a driving operation on the right foot sensing area with the right foot, and the sensing unit of the right foot sensing area generates current;

[0015] Step S3, the voltage waveforms between the first friction layer and the second friction layer of all the sensing units are transmitted to the processor, and the processor identifies the voltage waveforms;

[0016] Step S4, performing the interface actions corresponding to the voltage waveforms.

[0017] Preferably, in the step S2, the set action performed by the driver with the left foot includes tapping, long pressing and trajectory drawing.

[0018] Preferably, the tapping and long pressing actions are directed to one sensing unit, and the processor identifies the voltage change of the one sensing unit; and the trajectory drawing is directed to multiple sensing units, and the processor jointly identifies the voltage change of the multiple sensing units over time.

[0019] Preferably, in the step S2, the driving operation performed by the driver with the right foot includes normal acceleration, fast acceleration, normal braking, fast braking, intended acceleration operation and intended braking operation.

[0020] Preferably, in the step S3, the voltage waveforms are transmitted to the processor through the first friction layer and the second friction layer.

[0021] Step S301, the data acquisition device collects the time sequence signal generated by the sensing unit;

[0022] Step S302, a light convolutional neural network architecture is introduced to decode the time sequence signal;

[0023] Step S303, the features of the time sequence signal in the time dimension are extracted;

[0024] Step S304, the processor identifies the features and judges the corresponding interface action.

[0025] Compared with the prior art, the present application has the following beneficial effects:

[0026] 1) The intelligent foot pad of the present application adopts a single electrode working mode, the first friction layer and the second friction layer are in contact and separation for friction power generation, which has high sensitivity and stable and intuitive voltage signals, and can reduce the complexity of algorithm processing; the size can be adjusted according to the in-vehicle environment, and the production is convenient, easy to promote and apply; the sensing unit is arranged in different arrays according to different conditions of the left and right feet, which adapts to the actual operation demand;

[0027] 2) The driver-vehicle interaction method of the present application identifies the driver's action by using the intelligent foot pad, which not only can perform the set function through the foot action, but also can realize the identification of the driver's driving intention, so as to timely find potential safety hazards and improve the safety of driving and the intelligent and autonomous degree of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a schematic diagram of the intelligent foot pad structure for the cockpit of the present application;

[0029] Figure 2 is a schematic diagram of the influence of the thickness of the conductive sponge on the output performance of the present application;

[0030] Figure 3 is a schematic diagram of the influence of the area of the conductive sponge on the output performance of the present application;

[0031] Figure 4 is a schematic diagram of the second friction layer material selection test of the present application;

[0032] Figure 5 is a schematic diagram of the response of the sensing unit to different contact forces of the present application;

[0033] Figure 6 is a schematic diagram of the consistency detection of the sensing unit of the present application;

[0034] Figure 7 is a schematic diagram of the cycle stability detection of the sensing unit of the present application;

[0035] Figure 8 This is a schematic diagram of the sensing unit structure of the present invention;

[0036] Figure 9 The waveforms of normal acceleration, rapid acceleration, and acceleration intention of the T8 sensing unit of the present invention are shown.

[0037] Figure 10 The waveforms of normal acceleration, rapid acceleration, and acceleration intention of the T9 sensing unit of the present invention are shown.

[0038] Figure 11 This is a schematic diagram illustrating the response time of different driving operations according to the present invention;

[0039] The numbers in the diagram are as follows:

[0040] 1. Substrate layer, 2. Electrostatic shielding layer, 3. Electrode layer, 4. First friction layer, 5. Second friction layer, 6. Encapsulation layer, 7. Covering layer. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, the present invention provides a smart foot mat for a cockpit, including a sensing unit, a foot mat body, a covering layer 7, and a data acquisition device.

[0044] like Figure 8 As shown, the sensing unit includes a substrate layer 1, an electrostatic shielding layer 2, an electrode layer 3, a first friction layer 4, and a second friction layer 5 arranged sequentially. A surrounding layer 6 is provided around the electrode layer 3, the first friction layer 4, and the second friction layer 5. The materials of the first friction layer 4 and the second friction layer 5 differ in their friction electrode sequence. The first friction layer 4 and the second friction layer 5 are connected to a data acquisition device. An Arduino can be used as the data acquisition device.

[0045] like Figure 2 As shown, the first friction layer 4 is a conductive sponge with a thickness of 15mm. This invention studies the effect of the height of the conductive sponge on the output performance, and tests are conducted on conductive sponges of different heights (area 2×2cm). 2 (5N, friction material is leather). The higher the conductive sponge, the more potential points there are on the friction band, which promotes charge generation but simultaneously reduces electrostatic induction. Results show that a conductive sponge with a height of 15mm generates the highest voltage. Figure 4As shown, the second friction layer 5 is polyethylene terephthalate (PET), and the present application tests several commonly used materials and conductive sponge to form a triboelectric pair to evaluate the performance of the materials (thickness of 15 mm, 5N, area of 2x2 cm2). The results show that PET has the best performance as a friction layer, so we choose PET as the material of the second friction layer 5.

[0046] The electrode layer 3 is a conductive material.

[0047] The wrapping layer 6 and the cover layer 7 are insulating materials.

[0048] The substrate layer 1 is an insulating material with certain support.

[0049] The foot pad body is divided into a left foot sensing area and a right foot sensing area, the left foot sensing area is provided with 16 mounting holes, the right foot sensing area is provided with 9 mounting holes, and the sensing unit is mounted in the mounting hole; the cover layer 7 covers the surface of the foot pad body on which the sensing unit is mounted. Figure 3 As shown, the area of the sensing unit mounted in the left foot sensing area is 4x4 cm 2 , and the area of the sensing unit mounted in the right foot sensing area is 2x2 cm 2 . The present application studies the influence of the area of the first friction layer 4 (conductive sponge) on its output performance, and the area of the conductive sponge affects the voltage amplitude and resolution of the foot pad. In order to balance the two, the influence of the area of the conductive sponge on its output performance is studied (thickness of 15 mm, 5N, friction material is leather). The smaller the area of the conductive sponge, the smaller the voltage amplitude, and when the area is 1x1 cm 2 , the voltage is only 1.5V. In order to balance the complexity of the foot pad and the resolution, 4x4 cm 2 is selected for the interaction of the left foot because the left foot has a large active area; relatively speaking, the right foot has a small active area, and the driving intention is accompanied by a small change in force, so the sensor needs to have high resolution and accuracy. Therefore, 2x2 cm 2 of conductive sponge is selected to make a sensing array with driving intention detection function.

[0050] The triboelectric process is as follows:

[0051] 1) The first friction layer 4 obtains triboelectric charge: when the first friction layer 4 contacts the second friction layer 5, due to friction, the first friction layer 4 will obtain a certain amount of charge;

[0052] 2) Relative separation leads to charge induction: when the second friction layer 5 and the first friction layer 4 are relatively separated, the charge outside the first friction layer 4 will induce opposite charge. For example, if the first friction layer 4 is negatively charged, then the positive charge on the second friction layer 5 will be induced, resulting in charge flow;

[0053] 3) Charge reaches equilibrium and stops flowing: When the second rubbing layer 5 is completely separated from the first rubbing layer 4, the charge in the first rubbing layer 4 will reach an equilibrium state, and the charge flow will stop;

[0054] 4) Reverse flow and AC output: When the second rubbing layer 5 approaches the first rubbing layer 4 again, the charge will flow in the opposite direction until it is in contact again. Through this process of repeated touching and separating, the first rubbing layer 4 generates an alternating current output.

[0055] As shown in Figure 5 , the present application studies the response of the sensing unit to different contact forces on an area of 2x2 cm², from 0 to 35N, Voc increases with the increase of applied force, and then the growth rate of Voc decreases with the increase of applied force. The results show that the R² value of the conductive sponge in the range of 0~35N is 0.98243, the sensitivity is 0.716V / N, and the contact force can be detected.

[0056] As shown in Figure 6 , the linearity consistency of the sensing unit affects the calibration and calculation difficulty, the present application tests the response of multiple sensing units, the correlation is at least 0.9486, which shows that multiple sensing units have high consistency; as shown in Figure 7 , the multiple sensing units are impacted for 10 hours (frequency of 2Hz, force of 5N, area of 2x2cm²), the voltage data of the sensing units in three stages (before, during and after) is measured, and the voltage amplitude remains relatively constant, which confirms that the sensing unit has good cycle stability.

[0057] Example 2

[0058] The present application provides a kind of driver-machine interaction method using cockpit intelligent foot pad, specifically:

[0059] Define the interface action corresponding to the voltage waveform;The left foot of the driver executes the set action and treads the left foot sensing area, and the sensing unit of the left foot sensing area generates current;Right foot executes driving operation and treads right foot sensing area, and the sensing unit of right foot sensing area generates current;Voltage data is collected by data acquisition equipment Arduino;Voltage data is filtered and transmitted to trained convolutional neural network;After the processor identifies voltage waveform, the interface action corresponding to voltage waveform is executed.The result can be displayed in real time through display screen, to achieve the purpose of man-machine interaction and driving intention monitoring.

[0060] The set actions performed by the left foot include single click, double click, long press, sliding, up, down, left, right control sensor unit or drawing a track, etc. In this embodiment, the voltage of single click represents a complete voltage waveform (with up and down peaks), the voltage of double click represents two complete waveforms (with two up and down peaks), and the voltage of long press represents a half waveform (only the up waveform). The end of long press appears a downward waveform, and the interval time between the up waveform and the down waveform is the long press time. The threshold value of the interval time can be set to trigger the long press, for example, 1s is used as the threshold time. Sliding is realized by the cooperation of multiple sensors. Taking right sliding as an example, the sensor units are numbered from S1 to S16. When the foot slides from S4 to S8, each sensor unit generates a complete waveform with a time difference. The direction of sliding can be judged by the time sequence.

[0061] For example: first, step on the footrest in the upper left area of the seat cushion twice to enter the footpad interaction mode, then step on S5 and S12 to control the right frame to move to the air conditioner option, double click S6, S7, S10 and S11 to enter the air conditioner control interface, control the air conditioner temperature to increase by long pressing S2 and S3, control the air conditioner temperature to decrease by pressing S2-S7-S10-S15 in sequence, and finally step on the footrest in the upper left area of the seat cushion twice to exit the footpad interaction mode.

[0062] The voltage waveform diagram of the human-vehicle interaction module can be realized by drawing a track with the foot. In this embodiment, the tracks of “L”, “C”, “Z”, “S” and “O” are reproduced by the sensor array and used to open specific interfaces.

[0063] The tracks of “L”, “C”, “Z”, “S” and “O” correspond to different voltage waveforms. By drawing a track, a specific interface can be opened. From “L”-“C”-“Z”, it corresponds to opening the air conditioner, opening the music and opening the address book.

[0064] When encountering danger, a specific track can be drawn to trigger an alarm. In this embodiment, the SOS track is designed to trigger an alarm signal.

[0065] The driving operations performed by the right foot include normal acceleration, fast acceleration, normal braking, fast braking, intended acceleration operation and intended braking operation. The data acquisition device collects the time sequence signals generated by the sensor units; a lightweight convolutional neural network architecture is introduced to decode the time sequence signals; the features of the time sequence signals in the time dimension are extracted; and the processor identifies the features to determine the corresponding interface actions.

[0066] The sensor units are numbered from T1 to T9. As the degree of stepping increases, the voltage amplitude increases, and at the same time, T4, T5, T2 and T1 will respond. In this embodiment, the single waveforms of the six actions are compared and analyzed.

[0067] like Figure 9 As shown, the T8 sensor responds with a single waveform for normal acceleration, rapid acceleration, and acceleration intent.

[0068] like Figure 10 The image shows a single waveform of normal braking, emergency braking, and braking intent from the T9 sensor.

[0069] like Figure 11 As shown, by comparing voltage amplitude and response time, it can be seen that the voltage amplitude generated by sudden stomping (including sudden acceleration and sudden braking) is the largest, while the voltage amplitude generated by intentional stomping (including intentional acceleration and intentional braking) is the smallest. This is because when the intention to stomp occurs, the driver's foot exerts force but does not produce a displacement change, and the electrical signal is mostly generated by the full contact of the porous structure on the surface of the conductive sponge and the internal self-friction, so the signal is relatively small. However, when the stomping action occurs, the downward pressure of the foot creates a larger contact area between the two friction layers, thus generating a larger electrical signal. At the same time, the faster the stomping speed, the larger the voltage amplitude generated. In terms of response time, the response time is shortest when intentionally stomping (including intentional acceleration and intentional braking), followed by sudden stomping (including sudden acceleration and sudden braking), and the response time is longest when normal stomping (including normal acceleration and normal braking).

[0070] This invention introduces a convolutional neural network (CNN) to classify six different driving actions and intentions. Considering the limited computing power in actual car cockpits, this embodiment designs a lighter, shallower CNN structure to reduce the basic computing power required for model inference. A simple 2D CNN with only two convolutional layers is constructed to decode the electrical time-series signal. This approach follows the processing methods in conventional computer vision; therefore, a signal input with a size of 240×1 is transformed to 15×16 to be treated as a single image. Excessively long time spans of the signal may cause the model to fail in contextual feature attention. 32 and 64 3×3 filter kernels are used respectively to support fine-grained feature extraction by the model and double the receptive field in the temporal dimension.

[0071] In this embodiment, signal samples were collected, a portion of which was used as the training set, which was further divided into a validation set (10% of the training set) and a test set for validation. Furthermore, early stopping techniques were used to reduce the risk of overfitting. The model's performance was validated on the test set for six different categories of driving control intention signals.

[0072] To more intuitively understand the model's representation process of high-dimensional signals, the low-dimensional signal feature embeddings are obtained from the output of the last convolutional layer of the CNN, and then obtained by clustering after PCA dimensionality reduction.

[0073] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A pilot-machine interaction method using an intelligent foot mat for a cockpit, characterized by, The cockpit intelligent foot pad comprises a sensing unit, a foot pad body, a covering layer (7) and a data acquisition device, the sensing unit comprises an electrostatic shielding layer (2), an electrode layer (3), a first friction layer (4) and a second friction layer (5) arranged in sequence, and a wrapping layer (6) is arranged around the electrode layer (3), the first friction layer (4) and the second friction layer (5); the material of the first friction layer (4) and the second friction layer (5) has a friction electrode sequence difference, the first friction layer (4) and the second friction layer (5) are connected with the data acquisition device; the foot pad body is divided into a left foot sensing area and a right foot sensing area, a plurality of mounting holes are arranged in the left foot sensing area and the right foot sensing area respectively, and the sensing unit is mounted in the mounting holes; and the covering layer (7) covers the surface of the foot pad body on which the sensing unit is mounted. The driver-vehicle interaction method specifically comprises the following steps: Step S1, defining the interface action corresponding to the voltage waveform; Step S2, the driver's left foot performs a set action to step on the left foot sensing area, and the sensing unit of the left foot sensing area generates current; the right foot performs a driving operation to step on the right foot sensing area, and the sensing unit of the right foot sensing area generates current; Step S3, the voltage waveform between the first friction layer (4) and the second friction layer (5) of all sensing units is transmitted to the processor, and the processor identifies the voltage waveform; Step S4, performing the interface action corresponding to the voltage waveform; Step S3 specifically comprises: Step S301, the data acquisition device acquires the time sequence signal generated by the sensing unit; Step S302, introducing a light convolutional neural network architecture to decode the time sequence signal; Step S303, extracting the feature of the time sequence signal in the time dimension; Step S304, the processor identifies the feature to determine the corresponding interface action.

2. The pilot-machine interaction method employing the intelligent foot mat for the cockpit according to claim 1, characterized in that, The left foot sensing area is provided with array-arranged mounting holes; and the right foot sensing area is provided with array-arranged mounting holes.

3. The driver-machine interaction method employing the intelligent foot mat for the cockpit according to claim 1, characterized in that, The electrode layer (3) is made of conductive material; and the wrapping layer (6) and the covering layer (7) are made of insulating material.

4. The driver-machine interaction method employing the intelligent foot mat for the cockpit according to claim 1, characterized in that, The first friction layer (4) is conductive sponge with a thickness of 15 mm; the area of the sensing unit installed in the left foot sensing area is 4*4 cm 2 , and the area of the sensing unit installed in the right foot sensing area is 2*2 cm 2 .

5. The driver-machine interaction method employing the intelligent foot mat for the cockpit according to claim 1, characterized in that, The sensing unit further comprises a substrate layer (1) made of insulating material and located between the electrostatic shielding layer (2) and the foot pad body.

6. The driver-machine interaction method employing the intelligent foot mat for the cockpit according to claim 1, characterized in that, In step S2, the set action performed by the driver's left foot includes tapping, long pressing and trajectory drawing.

7. The pilot-machine interaction method with the intelligent foot mat for the cockpit according to claim 6, characterized in that, The tapping and long pressing actions are directed to one sensing unit, and the processor identifies the voltage change of one sensing unit; the trajectory drawing is directed to multiple sensing units, and the processor jointly identifies the voltage change of multiple sensing units over time.

8. The pilot-machine interaction method with intelligent foot mat for cockpit according to claim 1, characterized in that, In step S2, the driving operation performed by the driver's right foot includes normal acceleration, rapid acceleration, normal braking, rapid braking, intended acceleration operation and intended braking operation.

Images