Pedal mis-touch detection method and device, computer device and storage medium

Abstract

The application relates to a pedal false touch detection method and device, computer equipment, a storage medium and a computer program product. The method monitors the triggering condition of the vehicle accelerator pedal, obtains the multimodal data inside the vehicle and the environmental data outside the vehicle when the vehicle accelerator pedal is triggered, and performs feature extraction to obtain driving posture features, driving state features and driving environment features, determines the probability of false triggering of the accelerator pedal according to the extracted features, and then judges whether the accelerator pedal is false triggered. Since the embodiment determines whether the accelerator pedal is false triggered based on the feature extraction of the multimodal data inside the vehicle and the environmental data outside the vehicle, compared with the traditional technology, the accuracy of judging whether the accelerator pedal is false triggered can be improved, and the driving safety can be improved, and the occurrence of accidents can be avoided.

Description

Technical Field

[0001] This application relates to the field of automotive technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for detecting accidental pedal contact. Background Technology

[0002] With the improvement of people's living standards, cars have become widespread. Accidentally pressing the accelerator pedal while driving is a relatively common occurrence. In traditional gasoline-powered vehicles, due to slow acceleration, low horsepower, and the use of manual transmissions, accidentally pressing the accelerator pedal usually doesn't lead to a major disaster. However, for new electric vehicles, equipped with high-power dual motors, they can typically accelerate to 100 km / h in just a few seconds, and the accelerator pedal is exceptionally sensitive. Therefore, accidentally pressing the accelerator pedal can potentially lead to catastrophic accidents.

[0003] Traditional technologies currently focus on identifying external vehicle environments and detecting accidental touches by sensors mounted on the accelerator pedal. This involves assessing the complexity of the surrounding environment, whether the accelerator pedal is rotating at a large angular velocity, and using threshold values ​​to determine if a touch is accidental.

[0004] However, current methods for determining whether the accelerator pedal has been accidentally pressed rely heavily on the sensitivity of external vehicle sensors and cameras, and the means of determining whether the accelerator pedal has been accidentally pressed are limited. If the vehicle's front radar or camera is interfered with, or if the driver habitually accelerates by pressing the accelerator pedal hard, the "accidental press system" will be triggered, causing the vehicle to fail to start normally. Therefore, the accuracy of current methods for determining whether the accelerator pedal has been accidentally pressed is relatively low. Summary of the Invention

[0005] Therefore, it is necessary to address the technical problem of low accuracy in determining whether the accelerator pedal has been accidentally touched in the aforementioned traditional technologies by providing a detection method, device, computer equipment, computer-readable storage medium, and computer program product that can accurately determine whether the pedal has been accidentally touched.

[0006] Firstly, this application provides a method for detecting accidental pedal activation. The method includes:

[0007] When the accelerator pedal of the vehicle is detected to be triggered, multimodal data inside the vehicle and environmental data outside the vehicle are acquired.

[0008] Feature extraction is performed on the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving posture features, driving state features and driving environment features;

[0009] The probability of the accelerator pedal being accidentally touched is determined based on the driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0010] In one embodiment, the multimodal data inside the vehicle includes audio information from inside the vehicle, facial images, posture images, and physiological index information of the target user; the step of extracting features from the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features, and driving environment features includes: using a preset posture feature extraction network to extract posture features from the facial images and posture images of the target user inside the vehicle to obtain corresponding driving posture features; using a preset state feature extraction network to extract state features from the facial images and physiological index information of the target user inside the vehicle to obtain corresponding driving state features; and using a preset environmental feature extraction network to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving environment features.

[0011] In one embodiment, determining the probability of the accelerator pedal being accidentally touched based on the driving posture features, driving state features, and driving environment features includes: obtaining user attribute information of the target user; determining a matching user weight based on the user attribute information; and calculating the product of the user weight and the driving posture features, driving state features, and driving environment features to obtain the probability of the accelerator pedal being accidentally touched.

[0012] In one embodiment, determining the probability of the accelerator pedal being accidentally touched based on the driving posture features, driving state features, and driving environment features includes: acquiring user attribute information of the target user; determining a matching user weight based on the user attribute information; determining a first feature weight corresponding to the driving posture features, a second feature weight corresponding to the driving state features, and a third feature weight corresponding to the driving environment features based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture features, the driving state features, and the driving environment features.

[0013] In one embodiment, determining the probability of the accelerator pedal being accidentally touched based on the driving posture features, driving state features, and driving environment features includes: determining a first feature weight corresponding to the driving posture features, a second feature weight corresponding to the driving state features, and a third feature weight corresponding to the driving environment features based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture features, the driving state features, and the driving environment features.

[0014] In one embodiment, determining the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics, and driving environment characteristics includes: obtaining the angular velocity when the accelerator pedal is triggered; and determining the probability of the accelerator pedal being accidentally touched based on the angular velocity, the driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0015] Secondly, this application also provides a device for detecting accidental pedal activation. The device includes:

[0016] The data acquisition module is used to acquire multimodal data inside the vehicle and environmental data outside the vehicle when the accelerator pedal of the vehicle is detected to be triggered.

[0017] The feature extraction module is used to extract features from the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features and driving environment features.

[0018] The accidental touch detection module is used to determine the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0019] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first aspect above.

[0020] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0021] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0022] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for detecting accidental accelerator pedal activation monitor the triggering of the vehicle's accelerator pedal. When the accelerator pedal is detected to be activated, multimodal data from inside the vehicle and environmental data from outside the vehicle are acquired and feature extraction is performed to obtain driving posture features, driving state features, and driving environment features. Based on the extracted features, the probability of accidental accelerator pedal activation is determined, thereby determining whether the accelerator pedal has been accidentally activated. Since this embodiment determines whether the accelerator pedal has been accidentally activated based on feature extraction from multimodal data from inside the vehicle and environmental data from outside the vehicle, it improves the accuracy of determining whether the accelerator pedal has been accidentally activated compared to traditional technologies, thereby improving driving safety and preventing accidents. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a method for detecting accidental pedal touch in one embodiment;

[0024] Figure 2 This is a flowchart illustrating the feature extraction steps in one embodiment;

[0025] Figure 3 This is a flowchart illustrating the steps for determining the probability of the accelerator pedal being accidentally touched in one embodiment.

[0026] Figure 4 This is a flowchart illustrating the steps for determining the probability of the accelerator pedal being accidentally touched in another embodiment;

[0027] Figure 5 This is a flowchart illustrating the steps for determining the probability of the accelerator pedal being accidentally touched in yet another embodiment;

[0028] Figure 6 This is a flowchart illustrating the steps for determining the probability of the accelerator pedal being accidentally touched in another embodiment;

[0029] Figure 7 This is a structural block diagram of a pedal accident detection device in one embodiment;

[0030] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] In one embodiment, such as Figure 1As shown, a method for detecting accidental pedal touch is provided. This embodiment illustrates the application of this method to an in-vehicle device. It is understood that this method can also be applied to a server, or to a system including both an in-vehicle device and a server, and implemented through the interaction between the in-vehicle device and the server. In this embodiment, the method may include the following steps:

[0033] Step 102: When the accelerator pedal of the vehicle is detected to be triggered, acquire multimodal data inside the vehicle and environmental data outside the vehicle.

[0034] Multimodal data includes, but is not limited to, image data, audio data, and related user data, which can be collected through image acquisition modules, audio acquisition modules, and user data acquisition modules installed inside the vehicle. The user refers to the current driver of the vehicle. Environmental data can be relevant data about the vehicle's current environment, such as data collected from external sensors and cameras. Accelerator pedal triggering refers to the situation where the accelerator pedal is touched (e.g., pressed), initiating an acceleration command.

[0035] In this embodiment, the in-vehicle device can monitor the triggering of the vehicle's accelerator pedal in real time. When the accelerator pedal is detected to be triggered, for example, when the accelerator pedal is detected to be pressed, the device can acquire the multimodal data inside the vehicle and the environmental data outside the vehicle at the current moment.

[0036] Step 104: Extract features from the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving posture features, driving state features and driving environment features.

[0037] Feature extraction, in machine learning, pattern recognition, and image processing, is the process of extracting useful information (i.e., features) from initial measurement data. Specifically, driving posture features can be relevant information reflecting whether the user's driving posture is appropriate. Driving state features can be relevant information reflecting whether the user's driving state is normal. Driving environment features can be relevant information reflecting whether the current driving environment is normal.

[0038] In this embodiment, the in-vehicle device performs feature extraction based on the multimodal data inside the vehicle and the environmental data outside the vehicle obtained in the above steps, thereby obtaining the extracted driving posture features, driving state features and driving environment features.

[0039] Step 106: Determine the probability of the accelerator pedal being accidentally touched based on driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0040] The probability of the accelerator pedal being accidentally pressed reflects the likelihood of it being accidentally pressed. For example, a higher probability of accidental pressing means a greater likelihood that the accelerator pedal was pressed accidentally, rather than reflecting the user's true intention to accelerate. Conversely, a lower probability of accidental pressing means a lower likelihood that the accelerator pedal was pressed accidentally, reflecting the user's true intention to accelerate.

[0041] In this embodiment, the vehicle-mounted device can determine the probability of the accelerator pedal being accidentally touched based on driving posture characteristics, driving state characteristics, and driving environment characteristics, and then determine whether the accelerator pedal has been accidentally touched.

[0042] In the aforementioned method for detecting accidental accelerator pedal activation, the onboard device monitors the triggering of the vehicle's accelerator pedal. When the accelerator pedal is detected to be activated, it acquires multimodal data from inside the vehicle and environmental data from outside the vehicle, and performs feature extraction to obtain driving posture features, driving state features, and driving environment features. Based on the extracted features, it determines the probability of accidental accelerator pedal activation, and thus judges whether the accelerator pedal has been accidentally activated. Since this embodiment determines whether the accelerator pedal has been accidentally activated based on feature extraction from multimodal data from inside the vehicle and environmental data from outside the vehicle, it improves the accuracy of determining whether the accelerator pedal has been accidentally activated compared to traditional technologies, thereby improving driving safety and preventing accidents.

[0043] In one embodiment, the multimodal data inside the vehicle includes in-vehicle audio information, facial images of the target user, posture images, and physiological indicators. The target user can be the user in the driver's seat, i.e., the driver. The posture image is an image reflecting the overall posture of the target user, such as a full-body image. The physiological indicators include, but are not limited to, the target user's heart rate, body surface temperature, and humidity.

[0044] Then as Figure 2 As shown, feature extraction is performed on multimodal data inside the vehicle and environmental data outside the vehicle to obtain corresponding driving posture features, driving state features, and driving environment features, which may specifically include:

[0045] Step 202: Use a preset posture feature extraction network to extract posture features from the facial and posture images of the target user inside the vehicle to obtain the corresponding driving posture features.

[0046] The posture feature extraction network can be a pre-trained machine learning network or neural network used to extract the driving posture features of the target user. Driving posture features reflect whether the target user's driving posture is appropriate. Therefore, the posture feature extraction network is obtained by training the network using facial and posture images of users in both normal and abnormal driving postures until the network converges. For example, facial and posture images captured when the driver is speaking to the back seat while the vehicle is in motion are considered training data for abnormal driving postures. In other words, all facial or posture images that violate normal driving requirements are considered training data for abnormal driving postures, while facial or posture images that comply with normal driving requirements are considered training data for normal driving postures.

[0047] In this embodiment, the in-vehicle device extracts posture features from the facial and posture images of the target user inside the vehicle using a preset posture feature extraction network, thereby obtaining the corresponding driving posture features. For example, the in-vehicle device can input the facial and posture images of the target user inside the vehicle into the preset posture feature extraction network to obtain the driving posture features output by the network. These features can specifically be the probability or score of abnormal driving postures.

[0048] Step 204: Use a preset state feature extraction network to extract state features from the facial images and physiological indicators of the target user inside the vehicle to obtain the corresponding driving state features.

[0049] Similarly, the state feature extraction network can be a pre-trained machine learning network or neural network used to extract the driving state features of the target user. Driving state features can reflect whether the target user's driving state is normal. Therefore, the state feature extraction network can be obtained by training the network with training data such as facial images and physiological indicators of users under normal and abnormal driving states until the network converges. Since human body temperature, humidity, and heart rate will significantly increase and facial appearance will change when encountering unexpected situations, facial images and physiological indicators of users under normal and abnormal driving states can be collected as training data based on this.

[0050] In this embodiment, the in-vehicle device extracts state features from the facial image and physiological indicators of the target user inside the vehicle using a preset state feature extraction network, thereby obtaining the corresponding driving state features. For example, the in-vehicle device can input the facial image and physiological indicators of the target user inside the vehicle into the preset state feature extraction network to obtain the driving state features output by the network. These features can specifically be the probability or score of an abnormal driving state.

[0051] Step 206: Use a preset environmental feature extraction network to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving environment features.

[0052] The environmental feature extraction network can be a pre-trained machine learning network or neural network used to extract driving environment features of the vehicle's surroundings. Driving environment features can reflect whether the current driving environment is normal. Therefore, the environmental feature extraction network can be obtained by training the network using audio information from inside the vehicle under normal and abnormal driving conditions and environmental data from outside the vehicle until the network converges.

[0053] In this embodiment, the in-vehicle device extracts environmental features from the audio information inside the vehicle and the environmental data outside the vehicle using a preset environmental feature extraction network, thereby obtaining the corresponding driving environment features. For example, the in-vehicle device can input the audio information inside the vehicle and the environmental data outside the vehicle into the preset environmental feature extraction network to obtain the driving environment features output by the network. These features can specifically be the probability or score of abnormal driving environments.

[0054] In the above embodiments, by employing a pre-trained feature extraction network to extract features from multimodal data inside the vehicle and environmental data outside the vehicle, corresponding driving posture features, driving state features, and driving environment features are obtained, thereby improving the accuracy of feature extraction.

[0055] In one embodiment, the probability of the accelerator pedal being accidentally touched can be the product of the extracted driving posture features, driving state features, and driving environment features.

[0056] In one embodiment, such as Figure 3 As shown, in step 106, the probability of the accelerator pedal being accidentally touched is determined based on driving posture characteristics, driving state characteristics, and driving environment characteristics. Specifically, this may include:

[0057] Step 302: Obtain the user attribute information of the target user and determine the matching user weight based on the user attribute information.

[0058] The user attribute information can be user characteristics, such as, but not limited to, gender, age, and height. User weight, determined based on the user attribute information, reflects the degree of importance of the user in the evaluation process of accidental accelerator pedal activation. Specifically, different user weights can be pre-set based on different user attribute information.

[0059] Therefore, in this embodiment, the in-vehicle device can obtain the user attribute information of the target user, and then determine the matching user weight based on the user attribute information.

[0060] Step 304: Calculate the probability of the accelerator pedal being accidentally touched based on the user weight.

[0061] Specifically, the in-vehicle device can calculate the product of user weight with driving posture features, driving state features and driving environment features, and use this product as the probability that the accelerator pedal is accidentally touched.

[0062] In the above embodiments, the in-vehicle device obtains the user attribute information of the target user, determines the matching user weight based on the user attribute information, and calculates the probability of the accelerator pedal being accidentally touched based on the user weight. Since this embodiment refers to the user weight corresponding to the user attribute information when calculating the probability of the accelerator pedal being accidentally touched, it can further improve the accuracy of identifying whether the accelerator pedal has been accidentally touched.

[0063] In one embodiment, such as Figure 4 As shown, in step 106, the probability of the accelerator pedal being accidentally touched is determined based on driving posture characteristics, driving state characteristics, and driving environment characteristics. Specifically, this may include:

[0064] Step 402: Determine the feature weights.

[0065] The feature weights can include a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature. The feature weights are used to represent the importance of the corresponding feature in the evaluation process of the accelerator pedal being accidentally touched. Specifically, the feature weights corresponding to each feature can be pre-set based on the training data.

[0066] Therefore, in this embodiment, based on the pre-set feature weights, the first feature weight corresponding to the driving posture feature, the second feature weight corresponding to the driving state feature, and the third feature weight corresponding to the driving environment feature can be determined.

[0067] Step 404: Calculate the probability of the accelerator pedal being accidentally touched based on the feature weights.

[0068] Specifically, the on-board device can determine the probability of the accelerator pedal being accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature. For example, it can calculate the first product of the first feature weight and the driving posture feature, the second product of the second feature weight and the driving state feature, and the third product of the third feature weight and the driving environment feature, and use the sum of the first, second, and third products as the probability of the accelerator pedal being accidentally touched.

[0069] In the above embodiments, the in-vehicle device can calculate the probability of the accelerator pedal being accidentally touched based on pre-set feature weights. Since this embodiment refers to the weights of different features, i.e., the importance of different features, when calculating the probability of the accelerator pedal being accidentally touched, it can further improve the accuracy of identifying whether the accelerator pedal has been accidentally touched.

[0070] In one embodiment, such as Figure 5 As shown, in step 106, the probability of the accelerator pedal being accidentally touched is determined based on driving posture characteristics, driving state characteristics, and driving environment characteristics. Specifically, this may include:

[0071] Step 502: Obtain the user attribute information of the target user and determine the matching user weight based on the user attribute information.

[0072] In this embodiment, the user weight can be determined using parameters. Figure 3 Step 302 will not be described again in this embodiment.

[0073] Step 504: Determine the feature weights.

[0074] In this embodiment, the feature weights can be determined using parameters. Figure 4 Step 402 will not be described again in this embodiment.

[0075] Step 506: Calculate the probability of the accelerator pedal being accidentally touched based on user weight and feature weight.

[0076] Specifically, the in-vehicle device can determine the probability of the accelerator pedal being accidentally touched based on user weight, first feature weight, second feature weight, third feature weight, driving posture features, driving state features, and driving environment features.

[0077] For example, the first product of the first feature weight and the driving posture feature can be calculated, the second product of the second feature weight and the driving state feature can be calculated, and the third product of the third feature weight and the driving environment feature can be calculated. The sum of the first product, the second product and the third product can be calculated, and the product of this sum and the user weight can be used as the probability that the accelerator pedal is accidentally touched.

[0078] In the above embodiments, the in-vehicle device acquires the user attribute information of the target user, determines the matching user weight based on the user attribute information, and calculates the probability of the accelerator pedal being accidentally touched based on the pre-set feature weights. Because this embodiment considers not only the user weight corresponding to the user attribute information but also the weights of different features (i.e., the importance of different features) when calculating the probability of the accelerator pedal being accidentally touched, it can more accurately identify whether the accelerator pedal has been accidentally touched, further improving driving safety.

[0079] In one embodiment, such as Figure 6As shown, in step 106, the probability of the accelerator pedal being accidentally touched is determined based on driving posture characteristics, driving state characteristics, and driving environment characteristics. Specifically, this may include:

[0080] Step 602: Obtain the angular velocity when the accelerator pedal is triggered.

[0081] Here, angular velocity can be the normalized angular velocity when the accelerator pedal is depressed. Specifically, the onboard equipment can monitor the angular velocity of the accelerator pedal in real time, thus obtaining the corresponding angular velocity when the accelerator pedal is triggered.

[0082] Step 604: Determine the probability of the accelerator pedal being accidentally touched based on angular velocity, driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0083] Specifically, the vehicle-mounted device can calculate the product of driving posture features, driving state features, and driving environment features, and multiply the product by the angular velocity obtained above. The result can be used as the probability of the accelerator pedal being accidentally touched, thereby improving the accuracy of the vehicle-mounted device in recognizing that the accelerator pedal has been accidentally touched.

[0084] In one scenario, the in-vehicle device can also calculate the product of the user weight with the driving posture features, driving state features and driving environment features, and multiply the product with the angular velocity obtained above. The result can be used as the probability of the accelerator pedal being accidentally touched, thereby further improving the accuracy of the in-vehicle device in recognizing that the accelerator pedal has been accidentally touched.

[0085] In one scenario, the in-vehicle device can also calculate the first product of the first feature weight and the driving posture feature, the second product of the second feature weight and the driving state feature, and the third product of the third feature weight and the driving environment feature. The sum of the first, second, and third products is then calculated and multiplied by the obtained angular velocity. The result can be used as the probability of the accelerator pedal being accidentally touched, thereby further improving the accuracy of the in-vehicle device in recognizing that the accelerator pedal has been accidentally touched.

[0086] In one scenario, the in-vehicle device can also calculate the first product of the first feature weight and the driving posture feature, the second product of the second feature weight and the driving state feature, and the third product of the third feature weight and the driving environment feature. The device can also calculate the sum of the first, second, and third products and multiply the sum by the user weight and the angular velocity obtained above. The result can be used as the probability of the accelerator pedal being accidentally touched, thereby further improving the accuracy of the in-vehicle device in recognizing that the accelerator pedal has been accidentally touched.

[0087] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0088] Based on the same inventive concept, this application also provides a pedal mis-touch detection device for implementing the pedal mis-touch detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more pedal mis-touch detection device embodiments provided below can be found in the limitations of the pedal mis-touch detection method described above, and will not be repeated here.

[0089] In one embodiment, such as Figure 7 As shown, a device for detecting accidental pedal touch is provided, comprising: a data acquisition module 702, a feature extraction module 704, and an accidental touch recognition module 706, wherein:

[0090] The data acquisition module 702 is used to acquire multimodal data inside the vehicle and environmental data outside the vehicle when the accelerator pedal of the vehicle is detected to be triggered.

[0091] The feature extraction module 704 is used to extract features from the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features and driving environment features.

[0092] The accidental touch recognition module 706 is used to determine the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics and driving environment characteristics.

[0093] In one embodiment, the multimodal data inside the vehicle includes audio information from inside the vehicle, facial images, posture images, and physiological index information of the target user; the feature extraction module is specifically used to: extract posture features from the facial images and posture images of the target user inside the vehicle using a preset posture feature extraction network to obtain corresponding driving posture features; extract state features from the facial images and physiological index information of the target user inside the vehicle using a preset state feature extraction network to obtain corresponding driving state features; and extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle using a preset environmental feature extraction network to obtain corresponding driving environment features.

[0094] In one embodiment, the accidental touch recognition module is specifically used to: obtain user attribute information of the target user, determine the matching user weight based on the user attribute information, and calculate the product of the user weight with the driving posture feature, driving state feature and driving environment feature to obtain the probability that the accelerator pedal is accidentally touched.

[0095] In one embodiment, the accidental touch recognition module is further configured to: acquire user attribute information of the target user, determine matching user weights based on the user attribute information; determine a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature based on preset feature weights; and determine the probability of the accelerator pedal being accidentally touched based on the user weights, the first feature weights, the second feature weights, the third feature weights, the driving posture feature, the driving state feature, and the driving environment feature.

[0096] In one embodiment, the accidental touch recognition module is further configured to: determine a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature based on preset feature weights; and determine the probability that the accelerator pedal is accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0097] In one embodiment, the accidental touch recognition module is further configured to: obtain the angular velocity when the accelerator pedal is triggered; and determine the probability that the accelerator pedal is accidentally touched based on the angular velocity, the driving posture characteristics, the driving state characteristics, and the driving environment characteristics.

[0098] The modules in the aforementioned pedal misoperation detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0099] In one embodiment, a computer device is provided, which may be an in-vehicle device, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for detecting accidental pedal touch. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0100] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0101] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0102] When the accelerator pedal of the vehicle is detected to be triggered, multimodal data inside the vehicle and environmental data outside the vehicle are acquired.

[0103] Feature extraction is performed on the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving posture features, driving state features and driving environment features;

[0104] The probability of the accelerator pedal being accidentally touched is determined based on the driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0105] In one embodiment, the multimodal data inside the vehicle includes audio information inside the vehicle, facial images, posture images, and physiological index information of the target user; when the processor executes the computer program, it further implements the following steps: using a preset posture feature extraction network to extract posture features from the facial images and posture images of the target user inside the vehicle to obtain corresponding driving posture features; using a preset state feature extraction network to extract state features from the facial images and physiological index information of the target user inside the vehicle to obtain corresponding driving state features; and using a preset environmental feature extraction network to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving environment features.

[0106] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining user attribute information of the target user, determining the matching user weight based on the user attribute information, and calculating the product of the user weight with the driving posture feature, driving state feature and driving environment feature to obtain the probability that the accelerator pedal is accidentally touched.

[0107] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining user attribute information of the target user; determining a matching user weight based on the user attribute information; determining a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0108] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0109] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining the angular velocity when the accelerator pedal is triggered; and determining the probability that the accelerator pedal is accidentally touched based on the angular velocity, the driving posture characteristics, the driving state characteristics, and the driving environment characteristics.

[0110] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0111] When the accelerator pedal of the vehicle is detected to be triggered, multimodal data inside the vehicle and environmental data outside the vehicle are acquired.

[0112] Feature extraction is performed on the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving posture features, driving state features and driving environment features;

[0113] The probability of the accelerator pedal being accidentally touched is determined based on the driving posture characteristics, driving state characteristics, and driving environment characteristics.

[0114] In one embodiment, the multimodal data inside the vehicle includes audio information from inside the vehicle, facial images, posture images, and physiological index information of the target user; when the computer program is executed by the processor, it further implements the following steps: using a preset posture feature extraction network to extract posture features from the facial images and posture images of the target user inside the vehicle to obtain corresponding driving posture features; using a preset state feature extraction network to extract state features from the facial images and physiological index information of the target user inside the vehicle to obtain corresponding driving state features; and using a preset environmental feature extraction network to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving environment features.

[0115] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining user attribute information of the target user, determining the matching user weight based on the user attribute information, and calculating the product of the user weight with the driving posture feature, driving state feature and driving environment feature to obtain the probability that the accelerator pedal is accidentally touched.

[0116] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: obtaining user attribute information of the target user; determining a matching user weight based on the user attribute information; determining a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0117] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining a first feature weight corresponding to the driving posture feature, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0118] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the angular velocity when the accelerator pedal is triggered; and determining the probability that the accelerator pedal is accidentally touched based on the angular velocity, the driving posture characteristics, the driving state characteristics, and the driving environment characteristics.

[0119] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0120] When the accelerator pedal of the vehicle is detected to be triggered, multimodal data inside the vehicle and environmental data outside the vehicle are acquired.

[0121] Feature extraction is performed on the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving posture features, driving state features and driving environment features;

[0122] Based on the driving posture characteristics, driving state characteristics, and driving environment characteristics, the probability of the accelerator pedal being accidentally touched is determined.

[0123] In one embodiment, the multimodal data inside the vehicle includes audio information from inside the vehicle, facial images, posture images, and physiological index information of the target user; when the computer program is executed by the processor, it further implements the following steps: using a preset posture feature extraction network to extract posture features from the facial images and posture images of the target user inside the vehicle to obtain corresponding driving posture features; using a preset state feature extraction network to extract state features from the facial images and physiological index information of the target user inside the vehicle to obtain corresponding driving state features; and using a preset environmental feature extraction network to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving environment features.

[0124] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining user attribute information of the target user, determining the matching user weight based on the user attribute information, and calculating the product of the user weight with the driving posture feature, driving state feature and driving environment feature to obtain the probability that the accelerator pedal is accidentally touched.

[0125] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: obtaining user attribute information of the target user; determining a matching user weight based on the user attribute information; determining a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0126] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining a first feature weight corresponding to the driving posture feature, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature based on preset feature weights; and determining the probability of the accelerator pedal being accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

[0127] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the angular velocity when the accelerator pedal is triggered; and determining the probability that the accelerator pedal is accidentally touched based on the angular velocity, the driving posture characteristics, the driving state characteristics, and the driving environment characteristics.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0130] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting accidental pedal touch, characterized in that, The method includes: When the accelerator pedal of the vehicle is detected to be triggered, multimodal data inside the vehicle and environmental data outside the vehicle are acquired; the multimodal data inside the vehicle includes audio information inside the vehicle, facial images, posture images and physiological index information of the target user; the environmental data outside the vehicle includes relevant data collected by external sensors and cameras. A preset posture feature extraction network is used to extract posture features from the facial and posture images of the target user inside the vehicle to obtain the corresponding driving posture features; A preset state feature extraction network is used to extract state features from the facial images and physiological indicators of the target user inside the vehicle to obtain the corresponding driving state features. A preset environmental feature extraction network is used to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle to obtain the corresponding driving environment features. Based on the driving posture characteristics, driving state characteristics, and driving environment characteristics, the probability of the accelerator pedal being accidentally touched is determined; The step of determining the probability of the accelerator pedal being accidentally touched based on the driving posture features, driving state features, and driving environment features includes: obtaining the user attribute information of the target user; determining the matching user weight based on the user attribute information; and calculating the product of the user weight and the driving posture features, driving state features, and driving environment features to obtain the probability of the accelerator pedal being accidentally touched.

2. The method according to claim 1, characterized in that, Determining the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics, and driving environment characteristics includes: Obtain the user attribute information of the target user, and determine the matching user weight based on the user attribute information; The first feature weight corresponding to the driving posture feature is determined according to the preset feature weights, the second feature weight corresponding to the driving state feature is determined, and the third feature weight corresponding to the driving environment feature is determined. The probability of the accelerator pedal being accidentally touched is determined based on the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

3. The method according to claim 2, characterized in that, Determining the probability of the accelerator pedal being accidentally touched based on the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature includes: Calculate the first product of the first feature weight and the driving posture feature, calculate the second product of the second feature weight and the driving state feature, and calculate the third product of the third feature weight and the driving environment feature. Calculate the sum of the first product, the second product, and the third product, and multiply the sum by the user weight as the probability that the accelerator pedal is accidentally touched.

4. The method according to claim 1, characterized in that, Determining the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics, and driving environment characteristics includes: The first feature weight corresponding to the driving posture feature is determined according to the preset feature weights, the second feature weight corresponding to the driving state feature is determined, and the third feature weight corresponding to the driving environment feature is determined. The probability of the accelerator pedal being accidentally touched is determined based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

5. The method according to claim 4, characterized in that, The step of determining the probability of the accelerator pedal being accidentally touched based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature includes: Calculate the first product of the first feature weight and the driving posture feature, calculate the second product of the second feature weight and the driving state feature, and calculate the third product of the third feature weight and the driving environment feature. Calculate the sum of the first product, the second product, and the third product, and use this sum as the probability that the accelerator pedal is accidentally pressed.

6. The method according to claim 1, characterized in that, Determining the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics, and driving environment characteristics includes: Obtain the angular velocity when the accelerator pedal is triggered; The probability of the accelerator pedal being accidentally touched is determined based on the angular velocity, the driving posture characteristics, the driving state characteristics, and the driving environment characteristics.

7. The method according to claim 6, characterized in that, Determining the probability of the accelerator pedal being accidentally touched based on the angular velocity, driving posture characteristics, driving state characteristics, and driving environment characteristics includes: Calculate the product of the driving posture features, driving state features, and driving environment features; Multiplying the product by the angular velocity yields the probability that the accelerator pedal is accidentally pressed.

8. A device for detecting accidental pedal contact, characterized in that, The device includes: The data acquisition module is used to acquire multimodal data inside the vehicle and environmental data outside the vehicle when the accelerator pedal of the vehicle is detected to be triggered. The multimodal data inside the vehicle includes audio information inside the vehicle, facial images, posture images and physiological index information of the target user. The environmental data outside the vehicle includes relevant data collected by external sensors and cameras. The feature extraction module is used to extract posture features from the facial images and posture images of the target user inside the vehicle using a preset posture feature extraction network to obtain corresponding driving posture features; to extract state features from the facial images and physiological index information of the target user inside the vehicle using a preset state feature extraction network to obtain corresponding driving state features; and to extract environmental features from the audio information inside the vehicle and the environmental data outside the vehicle using a preset environmental feature extraction network to obtain corresponding driving environment features. The accidental touch detection module is used to determine the probability of the accelerator pedal being accidentally touched based on the driving posture characteristics, driving state characteristics, and driving environment characteristics. The accidental touch recognition module is specifically used to: obtain the user attribute information of the target user, determine the matching user weight based on the user attribute information, and calculate the product of the user weight with the driving posture feature, driving state feature and driving environment feature to obtain the probability that the accelerator pedal is accidentally touched.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images