Knock recognition method and vehicle

Abstract

The invention discloses a knock identification method and a vehicle. The knock identification method comprises the following steps: acquiring current knock data of a user; and based on the current knock data of the user and the historical knock data set of the same user, carrying out knock identification to obtain a knock identification result. According to the invention, the accuracy of knock recognition can be improved, and the user experience is improved.

Description

Technical Field

[0001] The present application belongs to the field of vehicle technology, and specifically relates to a tap recognition method and a vehicle, as well as a tap recognition device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of intelligent vehicle skin, elastic wave sensors are being used to detect and identify tapping events. Furthermore, in-depth research into the tapping habits and anatomy of different users has revealed significant differences in tapping behavior. However, existing tap recognition technology relies on training a tap recognition model based on a large amount of tapping data collected from different users. This model is then used to perform tap recognition. However, due to the significant differences in user tapping behavior, this results in low accuracy. Summary of the Invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a tap recognition method and vehicle, as well as a tap recognition device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of tap recognition and enhance the user experience.

[0004] In a first aspect, an embodiment of the present application provides a tap recognition method, comprising:

[0005] Get the user's current tapping data;

[0006] Knock recognition is performed based on the user's current knock data and the same user's historical knock data set to obtain a knock recognition result.

[0007] In some embodiments, before obtaining the user's current tapping data, the method further includes:

[0008] Determines whether the user triggered a tap event.

[0009] In some embodiments, the tap recognition based on the user's current tap data and the user's historical tap data set includes:

[0010] Determine a current feature vector of the user's current tapping data;

[0011] Determining a historical feature vector based on a historical knocking dataset;

[0012] Knock recognition is performed based on the current feature vector and historical feature vectors of the current knock data.

[0013] In some embodiments, determining the historical feature vector based on the historical knocking dataset includes:

[0014] A historical feature vector is determined based on M historical knock data in the historical knock data set, where M is an integer greater than or equal to 1.

[0015] In some embodiments, the M historical knocking data are the first M historical knocking data with the highest confidence ranking in the historical knocking data set.

[0016] In some embodiments, the above-mentioned determination of the historical feature vector based on the M historical knock data in the historical knock data set includes:

[0017] Determine M historical feature vectors based on M historical knocking data;

[0018] Perform weighted calculation on the M historical feature vectors, and use the obtained weighted feature vector as the final historical feature vector.

[0019] In some embodiments, the above-mentioned determination of the historical feature vector based on the M historical knock data in the historical knock data set includes:

[0020] M historical feature vectors are determined based on the M historical knocking data, and the historical knocking data and the historical feature vectors correspond one to one.

[0021] In some embodiments, the above-mentioned tap recognition based on the current feature vector and the historical feature vector of the current tap data includes:

[0022] Determine the similarity between the current feature vector and the historical feature vector;

[0023] Knock recognition is performed based on similarity and a preset similarity threshold.

[0024] In some embodiments, determining the similarity between the current feature vector and the historical feature vector includes:

[0025] Determine the similarity between the current feature vector and the weighted feature vector.

[0026] In some embodiments, determining the similarity between the current feature vector and the historical feature vector includes:

[0027] Determine the first similarity between the current feature vector and the M historical feature vectors respectively;

[0028] A weighted calculation is performed on the M first similarities to obtain a second similarity.

[0029] In some embodiments, determining the similarity between the current feature vector and the historical feature vector includes:

[0030] The similarity between the current feature vector and the historical feature vector is determined based on a similarity calculation method.

[0031] In some embodiments, the similarity calculation method includes at least one of cosine similarity, Mahalanobis distance, and Euclidean distance.

[0032] In some embodiments, performing tap recognition based on similarity and a preset similarity threshold to obtain a tap recognition result includes:

[0033] When the similarity is greater than or equal to a preset similarity threshold, it is determined that the user's tapping event is valid.

[0034] In some embodiments, it further includes:

[0035] When the similarity is less than the preset similarity threshold, tap recognition is performed based on the current feature vector and the single-click classification model.

[0036] In some embodiments, before performing the tap recognition based on the current feature vector and the historical feature vector of the current tap data, the method further includes:

[0037] It is determined that the amount of historical knock data in the historical knock data set is greater than or equal to a preset amount threshold.

[0038] In some embodiments, if the amount of historical knock data in the historical knock data set is less than a preset amount threshold, the method further includes:

[0039] Perform tap recognition based on the current feature vector and the single-tap classification model.

[0040] In some embodiments, it further includes:

[0041] The historical tapping data set is updated based on the current tapping data.

[0042] In some embodiments, the tap recognition result includes that the tap event is valid, and the method further includes:

[0043] If the tap event is determined to be valid based on the current tap data of at least two user-triggered tap events, the tap type and / or tap operation content corresponding to the tap event is determined.

[0044] In some embodiments, determining the tap type and / or tap operation content corresponding to the tap event includes:

[0045] Based on current tap data of at least two user-triggered tap events, a tap type and / or a tap operation content of the tap event is determined.

[0046] In some embodiments, the above-mentioned knocking type includes at least one of continuous knocking and single-clicking, and the knocking operation content includes at least one of raising the window, lowering the window, and opening the door.

[0047] In a second aspect, an embodiment of the present application provides a tapping recognition device, comprising:

[0048] Data acquisition module, used to obtain the user's current tapping data;

[0049] The tap recognition module is used to perform tap recognition based on the user's current tap data and the historical tap data set of the same user to obtain a tap recognition result.

[0050] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the tap recognition method described in the first aspect are implemented.

[0051] In a fourth aspect, an embodiment of the present application provides a vehicle comprising the electronic device described in the third aspect.

[0052] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the tap recognition method described in the first aspect are implemented.

[0053] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed by a processor of a vehicle, implements the steps of the tapping recognition method as described in the first aspect.

[0054] The technical solution provided by this application obtains the user's current tapping data, performs tapping recognition based on the user's current tapping data and the same user's historical tapping data set to obtain a tapping recognition result. Since the above technical solution can utilize each user's own historical tapping data to construct each user's historical tapping data set, and then performs tapping recognition based on the current tapping data collected daily by each user and combined with the corresponding historical tapping data set, it can improve the accuracy of tapping recognition and thus enhance the user experience.

[0055] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0057] Figure 1 This is a flowchart of a tapping recognition method according to an embodiment of the present application;

[0058] Figure 2 for Figure 1The specific execution flow chart of step 102 is shown;

[0059] Figure 3 Schematic diagram of another tapping recognition method according to an embodiment of the present application;

[0060] Figure 4 Schematic diagram of a flow chart of another tapping recognition method according to an embodiment of the present application;

[0061] Figure 5 This is a flow chart of another tapping recognition method according to an embodiment of the present application;

[0062] Figure 6 This is a structural diagram of a tapping recognition device in an embodiment of the present application;

[0063] Figure 7 This is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0064] Figure 8 This is a structural diagram of a vehicle in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0066] It should be understood that the various steps described in the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0067] As described in the background technology, existing tapping recognition technology usually collects a large amount of tapping data from different users, then uses the collected data to train a tapping recognition model, and then uses the trained tapping recognition model to perform tapping recognition on the tapping behaviors of different users. However, due to the large differences in behavioral habits of different users, the data set composed of the tapping data collected from different users lacks diversity, resulting in low tapping recognition accuracy.

[0068] In addition, the above-mentioned tap recognition model is usually updated through over-the-air download. However, frequent over-the-air download updates not only interrupt the user's normal usage process and cause inconvenience, but may also cause update failures due to factors such as network instability and excessively large update packages, further affecting the user experience. At the same time, continuous over-the-air download updates also mean higher bandwidth consumption, server load, and labor cost investment, which invisibly increases the overall maintenance cost of the system.

[0069] Therefore, the embodiment of the present application provides a technical solution for a tapping recognition method. Figure 1 A flowchart of the tapping recognition method provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the following steps are included:

[0070] Step 101: Obtain the user's current tapping data;

[0071] The current knocking data of the technical solution provided in the embodiment of the present application refers to the data set collected when the user knocks on the device during daily use of the device, including information such as the force, position, time and duration of the knocking, which can be collected in real time by an elastic wave sensor installed on the device. In this application, the above-mentioned device can be a vehicle. The above-mentioned elastic wave sensor is a sensor that detects and senses physical quantities by using the propagation characteristics of elastic waves in a medium. It captures the elastic wave signals generated by knocking, vibration, etc. inside or on the surface of an object, and uses signal processing and analysis technology to extract information related to the physical quantity. The number and installation location of the elastic wave sensors can be set according to the actual needs of the project. For example, an elastic wave sensor can be installed at each door of the vehicle.

[0072] Step 102: Perform tapping recognition based on the user's current tapping data and the user's historical tapping data set to obtain a tapping recognition result.

[0073] On the basis of step 101, the user's current tapping data collected is further utilized, and tapping recognition is performed in combination with the stored historical tapping data set of the user, and a tapping recognition result is obtained, that is, multiple tapping data of the same user are used to judge whether the user's tapping is valid, and the judgment result obtained is the tapping recognition result. The above-mentioned tapping recognition result includes a valid tapping event and an invalid tapping event; the above-mentioned historical tapping data set includes historical tapping data, which is collected by elastic wave sensors when the user taps the device during the historical period when the user uses the device, and is stored in the device, and it can belong to the same user. The specific number of historical tapping data can be determined according to factors such as the storage space, performance, and computing power resources of the device. For example, for a vehicle, 100 historical tapping data can be set to be saved. In this way, it can ensure a certain scale of data volume to support the accuracy and reliability of subsequent operations, and can effectively reduce the amount of calculation and improve the operating efficiency of the device.

[0074] As mentioned above, the above method can utilize each user's own historical tapping data to construct a historical tapping data set for each user, and then perform tapping recognition based on the current tapping data collected daily by each user and combined with the corresponding historical tapping data set, which can improve the accuracy of tapping recognition and thus enhance the user experience.

[0075] In an embodiment of the present application, before obtaining the user's current knocking data, it is also possible to determine whether the user has triggered a knocking event. Specifically, when the device meets the status conditions, it starts to monitor whether the user has triggered a knocking event. If a knocking event is detected during the monitoring period, the user's current knocking data is obtained, and subsequent tasks are completed based on the above knocking data. If no knocking event triggered by the user is detected during the monitoring time, the system will continue to maintain the monitoring state until the conditions are met or other interrupt instructions are received. The above status conditions include usage permission state, stop state, etc. A knocking event refers to a vibration signal event captured by the sensor when the user physically knocks on the surface of a device equipped with a high-precision elastic wave sensor. For example, for a vehicle, when the vehicle is in a user-active unlocked state and the vehicle is stationary, the system will immediately activate the knocking monitoring function and continuously monitor whether the user triggers subsequent operations by knocking on specific parts of the vehicle, such as doors and windows, such as lowering windows, opening doors, etc.

[0076] In some embodiments, Figure 2 for Figure 1 The specific execution flow chart of step 102 is shown in FIG. Figure 2 As shown, the following steps are included:

[0077] Step 201: Determine the current feature vector of the user's current tapping data;

[0078] This step further processes the acquired user's current tapping data for subsequent tapping recognition. Specifically, the user's current tapping data can be filtered to remove low-frequency noise and high-frequency interference. Feature extraction is then performed on the filtered current tapping data to obtain a corresponding feature vector. The feature vector represents the specific characteristics of the tapping event, such as amplitude, duration, and tapping force.

[0079] Step 202: determining a historical feature vector based on the historical knocking data set;

[0080] Similar to step 201, the historical tapping data in the historical tapping data set stored in the device is first filtered. Then, the features of the filtered historical tapping data are extracted to obtain a corresponding historical feature vector. The final historical feature vector is determined based on the historical feature vector. The historical feature vector is used to represent the specific features of the tapping event triggered by the same user. It can include a single feature vector or multiple feature vectors, and its specific value can be determined according to the actual needs of the project. The final historical feature vector is then used for subsequent tapping recognition.

[0081] Step 203: Perform tap recognition based on the current feature vector and the historical feature vector of the current tap data.

[0082] The knock recognition is completed by performing operations on the obtained current feature vector and historical feature vector, and combining with the corresponding knock recognition method. The above-mentioned knock recognition method may include algorithms based on feature similarity, pattern matching and other methods. Among them, the algorithm based on feature similarity can complete the knock recognition by calculating the similarity between the current feature vector and the historical feature vector. For example, the similarity between the above-mentioned feature vectors is calculated using similarity calculation methods such as cosine similarity and Mahalanobis distance.

[0083] In an embodiment of the present application, when determining a historical feature vector based on a historical tapping data set of the same user, specifically, the historical feature vector can be determined using a portion of the historical tapping data in the historical tapping data set, that is, the historical feature vector can be determined based on M historical tapping data in the historical tapping data set, where M is pre-set and can be an integer greater than or equal to 1, such as 10. In the present application, a specific method for determining the M historical tapping data used to calculate the historical feature vector is to first sort the historical tapping data from high to low according to their confidence level, and then select the first M historical tapping data from the sorted result to determine the historical feature vector.

[0084] Specifically, when determining the historical feature vector based on the selected M historical knocking data, there are the following two cases:

[0085] The first scenario involves determining a historical feature vector based on M historical knock data. Specifically, feature extraction is performed on the M historical knock data to obtain M historical feature vectors. These M historical feature vectors are then weighted to obtain a weighted feature vector, which serves as the final historical feature vector. Weights can be assigned based on the confidence level of the M historical feature vectors, and then weighted calculation is performed on the M historical feature vectors based on these weights to obtain the weighted feature vector.

[0086] The second case: determining M historical feature vectors based on M historical knocking data, that is, performing feature extraction on the M historical knocking data to obtain M corresponding historical feature vectors, that is, M final historical feature vectors.

[0087] In the embodiment of the present application, the tapping recognition can be completed based on the current feature vector and the historical feature vector of the current tapping data, in combination with an algorithm based on feature similarity. Specifically, Figure 3 As shown, the following steps are included:

[0088] Step 301: Determine the similarity between the current feature vector and the historical feature vector;

[0089] This step corresponds to the case where the historical feature vector is determined based on M historical knocking data, and also includes the following two cases:

[0090] The first case: calculate the similarity between the current feature vector and the weighted feature vector.

[0091] The second scenario is: first, the similarity between the current feature vector and each of the M historical feature vectors is calculated to obtain a first similarity. Then, a weighted sum is performed on the M first similarities. The result is the similarity between the current feature vector and the historical feature vector, i.e., the second similarity. When performing the weighted calculation on the M first similarities, the weight of the first similarity corresponding to each historical tap data item can be determined based on its confidence level. Then, the M first similarities are weighted and summed based on the weights to obtain the second similarity.

[0092] In an embodiment of the present application, the similarity between the current feature vector and the historical feature vector can be calculated by a similarity calculation method, and the similarity calculation method may include cosine similarity, Mahalanobis distance, and Euclidean distance. Among them, cosine similarity quantifies the similarity between two feature vectors by calculating the cosine value of the angle between the two feature vectors. For example, when the cosine similarity method is used to calculate the similarity between the current feature vector and the historical feature vector, the above calculation can be completed by the following formula:

[0093]

[0094] Where Cosine Similarity represents cosine similarity, F* represents the current feature vector, F represents the historical feature vector, F*.F represents the dot product of the current feature vector and the historical feature vector, ||F*|| represents the modulus of the current feature vector, and ||F|| represents the modulus of the historical feature vector.

[0095] The modulus of the current eigenvector and the historical eigenvector can be calculated using the modulus formula. Here, the modulus of the historical eigenvector is used as an example for explanation:

[0096]

[0097] Where ||F|| represents the modulus of the historical feature vector, n represents that the historical feature vector is an n-dimensional feature vector, and F i Represents the i-th component of the n-dimensional historical feature vector.

[0098] For the second case, the above formula can be used to calculate M times to obtain M first similarities, and then the second similarity can be calculated according to the weight of each first similarity.

[0099] Typically, the value range of cosine similarity is [-1, 1], that is, when the direction of the current feature vector is exactly the same as that of the historical feature vector, the similarity value is 1. Conversely, when the directions of the two feature vectors are completely opposite, the similarity value is -1. When the two feature vectors are perpendicular to each other, the similarity value is 0, indicating that the two feature vectors have no correlation in direction. In an embodiment of the present application, when the cosine similarity method is used to calculate the similarity between the current feature vector and the historical feature vector, only the positive similarity of the two feature vectors can be considered, without considering the reverse or negative similarity. Therefore, the value range of cosine similarity can be constrained, that is, only the part with a value range of [0, 1] is retained. In this way, not only can the algorithm logic be simplified and the calculation efficiency be improved, but also the similarity results can be made more in line with actual needs and the result interpretability can be enhanced.

[0100] Step 302: Perform tap recognition based on the similarity and a preset similarity threshold.

[0101] After obtaining the similarity between the feature vectors in step 301, the value of the similarity can be further compared with the preset similarity threshold to obtain a similarity comparison result, and then the tap recognition task is completed according to the similarity comparison result. Specifically, if the similarity comparison result is that the similarity value is greater than or equal to the preset similarity threshold, then the user's tap event is determined to be valid; if the similarity comparison result is that the similarity value is less than the preset similarity threshold, then the user's tap event is determined to be invalid. In this way, the device can effectively judge new tap events, thereby ensuring that only tap events that are similar and meaningful to the historical tap data of the same user are further processed, thereby improving the accuracy and efficiency of tap recognition and reducing false alarms or omissions. The preset similarity threshold is a benchmark for judging whether a tap event is valid. By setting this threshold, the device can effectively filter out invalid tap events caused by noise or misoperation while ensuring recognition accuracy.

[0102] In some embodiments, after determining that the tap event triggered by the user is valid, the tap type, tap operation content, etc. of the tap event can be further determined. Specifically, after completing the tap recognition based on the current tap data of at least two tap events triggered by the user and the historical tap data set of the same user, the tap type, tap operation content, etc. of the above tap event can be determined based on the above at least two tap recognition results. For example, for a scenario in which the user triggers two tap events, when determining the tap type of the tap event: after determining that the first tap event is valid, continue to judge the validity of the second tap event. If it is determined that the second tap event is also valid and the time interval between the two tap events meets the preset time interval, it is determined that the user has performed continuous tapping. At this time, the above two tap events are continuous tapping events, and the device can subsequently perform corresponding operations according to the tap operation content matching the above continuous tap events; if the second tap event is invalid, it is determined that the user has performed a single click. At this time, the above first tap event is a single click event, and the subsequent device performs corresponding operations according to the tap operation content matching the single click event. The preset time interval may be a default time interval of the device or a user-defined time interval.

[0103] When determining the type of tap operation of a tap event, after determining that the first tap event is valid, the validity of the second tap event is further determined. If the second tap event is also valid, and the time interval between the two tap events satisfies a preset time interval, the corresponding tap operation content is selected so that the device performs the corresponding operation according to the above tap operation content. If the second tap event is invalid, the corresponding tap operation content is also selected. The above tap type can include at least one of continuous tapping and single tapping, wherein continuous tapping can be two taps, three taps, etc. The tap operation content can be pre-set, such as some operation contents set when the device leaves the factory, or some operation contents subsequently customized by the user, such as raising the window, lowering the window, and opening the door.

[0104] In some embodiments, if the current tap event is determined to be invalid, the current feature vector can continue to be used in combination with a single-click classification model for tap recognition. Specifically, knuckle tap recognition can be determined based on the current feature vector. If the single-click classification model is used to determine that the current tap event was triggered by the user tapping their knuckles, subsequent operations are performed accordingly, such as determining the tap type of the subsequent tap event. Otherwise, the tap recognition is terminated. The single-click classification model can perform knuckle tap recognition based on the current feature vector and assign a confidence level to each feature vector.

[0105] In some embodiments, before performing tap recognition based on the current feature vector and the historical feature vector, the number of historical tap data in the historical tap data set of the same user stored in the device can be further judged. Specifically, the number of the above historical data is compared with a preset number threshold. When it is greater than or equal to the preset number threshold, tap recognition can be performed based on the current feature vector and the historical feature vector. When the number of historical tap data is less than the preset number threshold, the current feature vector is used and combined with the single-click classification model to perform tap recognition, thereby ensuring that the system can still maintain operation and perform tap recognition when the amount of historical tap data is insufficient. The above-mentioned preset number threshold is used to limit the number of historical tap data stored in the device to ensure that the cache burden of the device is reduced and the amount of calculation is reduced while the amount of data is sufficient. Its specific value can be determined based on factors such as the storage space size, performance, and computing power resources of the device. In this way, the device can flexibly respond to different situations, thereby ensuring the accuracy and reliability of tap recognition while also improving the overall performance and response speed of the device.

[0106] In the embodiment of the present application, in the process of performing tap recognition using the user's current tap data and the user's historical tap data set, the historical tap data set may also be updated using the current tap data. Specifically, the following situations may be involved:

[0107] In the first case, if the amount of historical tapping data in the historical tapping data set stored in the device is less than a preset threshold value, the user's current tapping data and information such as the confidence level of the tapping data will continue to be stored in the form of n-dimensional data until the amount of historical tapping data in the historical tapping data set is greater than or equal to the preset threshold value, so as to ensure the amount of data, thereby laying a data foundation for subsequent tapping recognition or other downstream tasks. The above n represents the number of dimensions of the data, and the specific value depends on the type and magnitude of the information actually stored. In this way, by adopting the form of n-dimensional data for storage, it is possible not only to effectively reduce data storage space and improve the efficiency of data retrieval and processing, but also to ensure good compatibility and comparability between tapping data from different sources and different types. In this case, the stored n-dimensional data does not include similarity information.

[0108] In the second case, if the amount of historical tapping data in the historical tapping data set of the same user stored in the device is greater than or equal to the preset quantity threshold, and the similarity between the calculated current feature vector and the historical feature vector is greater than or equal to the preset similarity threshold, that is, after determining that the tapping event is valid, the user's current tapping data, similarity value, confidence level and other information are also stored in the device in the form of n-dimensional data, and the earliest stored historical tapping data of the user is deleted to replace the earliest stored historical data, thereby ensuring that the historical tapping data in the historical tapping data set is always the user's recent tapping data, thereby adapting to changes in the user's needs and improving the accuracy of tapping recognition.

[0109] In some embodiments, the continuously updated historical tapping data set can also be evaluated regularly. Specifically, the historical tapping data set can be evaluated based on the historical tapping data in the historical tapping data set and a preset knuckle tapping threshold. During the evaluation, all tapping recognition records in the historical period will be reviewed. Whenever a tapping event is identified by the single-click classification model, attention will be paid to those tapping events that are determined to be triggered by the user's knuckle tapping. Subsequently, the total number of such events occurring in the historical period is counted, and the ratio between the number and the preset number threshold is further calculated. If the value is greater than or equal to the preset knuckle tapping threshold, it is considered that the historical tapping data set at this time is of high quality, that is, it contains a sufficient number of representative events. The knuckle tapping data can provide strong support for the subsequent tapping recognition task; if the above ratio value is less than the preset knuckle tapping threshold, it is considered that the quality of the historical tapping data set at this time is poor, which means that the number of knuckle tapping data in the current historical tapping data set is insufficient. If the historical tapping data set is used for tapping recognition, it may lead to poor tapping recognition accuracy. In this case, it can be improved by updating the historical feature vector, that is, selecting X historical tapping data to determine a new historical feature vector, where X is an integer greater than or equal to 1, and its value can be determined according to actual needs. It can be the same as M or different, and the X historical tapping data can include one or more of the M historical tapping data, or may not include any of them. In this way, the historical feature vector can be continuously optimized, thereby improving the accuracy and stability of tapping recognition. The above knuckle tapping threshold is used to measure whether the number of knuckle tapping data in the historical tapping data set meets the requirements, and its value can be determined according to the actual needs of the project. For example, in some application scenarios with extremely high requirements for tapping recognition accuracy, it may be necessary to set a relatively high knuckle tapping threshold.

[0110] In some embodiments, when the device changes users, the historical knocking data set stored in the device can be initialized. Specifically, the user can trigger the initialization instruction, for example, find the corresponding option in the menu interface of the device and click to confirm, or the user manually clears the data in the historical knocking data set, which can be done by deleting one by one or using the one-click clear function to achieve the purpose of initialization. Afterwards, the knocking data of the user is re-collected during the daily use of the new user and stored in the form of n-dimensional data to reconstruct the historical knocking data set of the new user. In this way, not only can the recognition accuracy of the knocking behavior of the new user be improved, but also the applicability of the device to different users can be enhanced, so that it can flexibly adapt to the usage habits and needs of various users, thereby improving the usage experience of new users and then improving the satisfaction of new users with the device. For example, for a vehicle, based on the knocking data collected during the historical period, the process of constructing a historical knocking data set and using it for knocking recognition is as follows: Figure 4 As shown, the following steps are included:

[0111] Step 401: monitor and collect tapping data during actual use;

[0112] This step is to start the vehicle's knock monitoring function when the vehicle is in the owner's active unlocking state and the vehicle is stationary, that is, the owner approaches the vehicle and the vehicle is stationary, and when a knock event triggered by the user is detected during the monitoring period, collect the knock data, such as historical knock data and current knock data.

[0113] Step 402: classify the tapping data using the terminal model;

[0114] The collected tapping data is filtered to remove noise, and then fed into the terminal model, such as the click classification model, to determine whether the tapping data belongs to knuckle tapping data, that is, the tapping data generated when the user triggers a tapping event through the knuckles, thereby ensuring the quality of the historical tapping dataset constructed subsequently.

[0115] Step 403: storing the classified tapping data;

[0116] This step is to store the knocking data in the form of n-dimensional data after determining that the knocking data belongs to the knuckle knocking data class in step 402, and not store the other types of data, thereby constructing a historical knocking data set for the user.

[0117] Step 404: Determine the current feature vector and the historical feature vector;

[0118] When the amount of historical knocking data in the constructed historical knocking data set is greater than or equal to the preset threshold, feature extraction is further performed on the current knocking data and part of the historical knocking data to obtain the current feature vector and the historical feature vector. The specific process can be referred to Figure 2 The process of determining the feature vector in the illustrated embodiment.

[0119] Step 405: Perform tap recognition based on the similarity between feature vectors.

[0120] Based on step 404, the similarity between the current feature vector and the historical feature vector is calculated using the similarity calculation method. The specific process can be referred to Figure 3 In the embodiment shown, tap recognition is then performed based on the above-mentioned similarity value and the preset similarity threshold, that is, tap events with a similarity value greater than or equal to the similarity threshold are determined to be valid tap events, thereby filtering out tap events with large deviations, that is, invalid tap events.

[0121] In the embodiment of the present application, for a vehicle, the user's tapping recognition is adaptively realized based on the historical tapping data set and the current tapping data currently collected by the elastic wave sensor, and the specific process of updating the historical tapping data set using the current tapping data is as follows: Figure 5 As shown, the following steps are included:

[0122] Step 501: Enable tap event detection.

[0123] When the vehicle is in the owner's active unlocking state and the vehicle is stationary, the device's knock event monitoring function is started and knock events are detected during the monitoring period.

[0124] Step 502: determine whether a tap event is detected;

[0125] This step is to determine whether a knock event is detected during the monitoring period after the device turns on the knock event monitoring function. If a knock event is detected, execute step 503, otherwise execute step 501 and continue detecting until a stop instruction or an interrupt signal is received.

[0126] Step 503: Determine the current feature vector;

[0127] When a tap event is detected, current tap data is acquired, and features are extracted from the tap data to obtain a current feature vector.

[0128] Step 504: Determine the historical feature vector;

[0129] This step can also be performed before step 503 or simultaneously with step 503, that is, using part of the historical knocking data in the historical knocking data set stored in the vehicle to determine the historical feature vector, for example, using 10 of the historical knocking data to determine one or more historical feature vectors. The specific process can be referred to Figure 2 The embodiments shown will not be described in detail here.

[0130] Step 505: determine whether the amount of historical knocking data in the historical knocking data set is greater than a quantity threshold;

[0131] This step determines the amount of data in the historical tapping dataset stored in the vehicle, i.e., compares it with a preset threshold. If the threshold is greater than or equal to the threshold, step 506 is executed; otherwise, step 511 is executed. The threshold can be determined based on factors such as the device's storage space, performance, and computing resources. For example, for a vehicle, it can be set to 100.

[0132] Step 506: Calculate the similarity between the current feature vector and the historical feature vector;

[0133] After determining in step 505 that the amount of historical knock data in the historical knock data set is greater than or equal to the preset threshold value, a similarity calculation method is used to calculate the similarity between the current feature vector and the historical feature vector, for example, a cosine similarity method is used. The specific process can be referred to Figure 3 The embodiments shown will not be described in detail here.

[0134] Step 507: Determine whether the similarity is greater than or equal to a similarity threshold;

[0135] After obtaining the similarity between the current feature vector and the historical feature vector, the similarity is then compared with a preset similarity threshold. If the similarity is greater than or equal to the threshold, the tap event triggered by the user is determined to be valid, and steps 508 and 509 are executed. Otherwise, step 511 is executed. The similarity threshold can be determined based on project requirements. For example, for devices with higher precision requirements, the similarity threshold can be set to 0.95.

[0136] Step 508: Determine the knock type of the knock event;

[0137] After determining that the tap event triggered by the user is valid, the type of the tap event can be further determined. Specifically, the tap type of the tap event can be determined by combining the historical tap data collected recently in the historical tap data set, that is, traversing the tap recognition records of the above historical tap data, and completing the determination of the current tap event type based on whether the corresponding historical tap event is valid. For example, if a tap event corresponding to a historical tap data collected before the current tap data is collected is valid, and the current tap event is valid, it is determined that the user has performed continuous tapping, and it is determined that the current tap event belongs to the continuous tap type. Here, the determination of the type of tap event is used as an example for explanation. In addition, after determining that the tap event triggered by the user is valid, the tap operation content corresponding to the above tap event can also be directly determined.

[0138] Step 509: Save the data in the form of n-dimensional data;

[0139] While executing step 508, the current tap data of the valid tap event, the similarity of the current tap data, the confidence level and other information can also be stored in the form of n-dimensional data. The data stored each time can be used to update the historical tap data set. For details, please refer to the process of updating the historical tap data set in the above embodiment.

[0140] Step 510: determine whether the tapping times ratio is greater than or equal to the knuckle tapping threshold;

[0141] On the basis of step 509, the historical knocking data set can also be regularly evaluated. Specifically, the number of historical knocking data in the historical knocking data set that belongs to the knuckle knocking data is first counted, and then the ratio of the knuckle knocking data to the total amount of historical knocking data in the historical knocking data set is calculated. Then, the above ratio is compared with the preset knuckle knocking threshold. If the ratio is greater than or equal to the knuckle knocking threshold, step 504 can be executed. Otherwise, step 501 is executed to update the historical feature vector. The value of the above knuckle knocking threshold can be determined according to the actual needs of the project. For example, in some application scenarios with extremely high requirements for knocking recognition accuracy, it may be set to 0.95.

[0142] Step 511: Classify by clicking on the classification model;

[0143] In step 505, if it is determined that the amount of historical tapping data in the historical tapping data set is less than the preset quantity threshold, or if it is determined in step 507 that the calculated similarity is less than the preset similarity threshold, the determined current feature vector is sent to the single-click classification model for classification, that is, a preliminary judgment is made as to whether the current tapping data belongs to knuckle tapping data to ensure data quality.

[0144] Step 512: determine whether the current tapping data is knuckle tapping data;

[0145] Based on step 511, whether the current tapping data is knuckle tapping data is further determined to improve the accuracy of subsequent operations, for example, by using a convolutional neural network to complete the above determination. If the current data is determined to be knuckle tapping data, step 508 is executed; otherwise, step 513 is executed.

[0146] Step 513: End the tapping recognition.

[0147] After determining in step 512 that the current tapping data is not knuckle tapping data, the tapping recognition is terminated.

[0148] In an embodiment of the present application, when the system uses current tap data and a historical tap data set of the same user to perform a tap recognition task, the historical tap data set can be dynamically updated simultaneously. This self-learning-based tap recognition method can be adjusted and optimized in real time, thereby reducing the frequency of optimizing the recognition method through over-the-air downloads, reducing system maintenance costs, and improving the user experience.

[0149] With the above Figure 1-Figure 5 Corresponding to any provided tap recognition method, the embodiments of the present application provide a corresponding device. Figure 6 This is a schematic diagram of the structure of a knock recognition device in an example of this application, such as Figure 6As shown, the tapping recognition device includes a data acquisition module 11 and a tapping recognition module 12, wherein the data acquisition module 11 is used for acquiring the user's current tapping data; the tapping recognition module 12 is used for performing tapping recognition based on the user's current tapping data and the same user's historical tapping data set to obtain a tapping recognition result.

[0150] In some embodiments, before obtaining the user's current tapping data, the method further includes:

[0151] Determines whether the user triggered a tap event.

[0152] In some embodiments, the tap recognition based on the user's current tap data and the user's historical tap data set includes:

[0153] Determine a current feature vector of the user's current tapping data;

[0154] Determining a historical feature vector based on a historical knocking dataset;

[0155] Knock recognition is performed based on the current feature vector and historical feature vectors of the current knock data.

[0156] In some embodiments, determining the historical feature vector based on the historical knocking dataset includes:

[0157] A historical feature vector is determined based on M historical knock data in the historical knock data set, where M is an integer greater than or equal to 1.

[0158] In some embodiments, the M historical knocking data are the first M historical knocking data with the highest confidence ranking in the historical knocking data set.

[0159] In some embodiments, the above-mentioned determination of the historical feature vector based on the M historical knock data in the historical knock data set includes:

[0160] Determine M historical feature vectors based on M historical knocking data;

[0161] Perform weighted calculation on the M historical feature vectors, and use the obtained weighted feature vector as the final historical feature vector.

[0162] In some embodiments, the above-mentioned determination of the historical feature vector based on the M historical knock data in the historical knock data set includes:

[0163] M historical feature vectors are determined based on the M historical knocking data, and the historical knocking data and the historical feature vectors correspond one to one.

[0164] In some embodiments, the above-mentioned tap recognition based on the current feature vector and the historical feature vector of the current tap data includes:

[0165] Determine the similarity between the current feature vector and the historical feature vector;

[0166] Knock recognition is performed based on similarity and a preset similarity threshold.

[0167] In some embodiments, determining the similarity between the current feature vector and the historical feature vector includes:

[0168] Determine the similarity between the current feature vector and the weighted feature vector.

[0169] In some embodiments, determining the similarity between the current feature vector and the historical feature vector includes:

[0170] Determine the first similarity between the current feature vector and the M historical feature vectors respectively;

[0171] A weighted calculation is performed on the M first similarities to obtain a second similarity.

[0172] In some embodiments, determining the similarity between the current feature vector and the historical feature vector includes:

[0173] The similarity between the current feature vector and the historical feature vector is determined based on a similarity calculation method.

[0174] In some embodiments, the similarity calculation method includes at least one of cosine similarity, Mahalanobis distance, and Euclidean distance.

[0175] In some embodiments, performing tap recognition based on similarity and a preset similarity threshold to obtain a tap recognition result includes:

[0176] When the similarity is greater than or equal to a preset similarity threshold, it is determined that the user's tapping event is valid.

[0177] In some embodiments, it further includes:

[0178] When the similarity is less than the preset similarity threshold, tap recognition is performed based on the current feature vector and the single-click classification model.

[0179] In some embodiments, before performing the tap recognition based on the current feature vector and the historical feature vector of the current tap data, the method further includes:

[0180] It is determined that the amount of historical knock data in the historical knock data set is greater than or equal to a preset amount threshold.

[0181] In some embodiments, if the amount of historical knock data in the historical knock data set is less than a preset amount threshold, the method further includes:

[0182] Perform tap recognition based on the current feature vector and the single-tap classification model.

[0183] In some embodiments, it further includes:

[0184] The historical tapping data set is updated based on the current tapping data.

[0185] In some embodiments, the tap recognition result includes that the tap event is valid, and the method further includes:

[0186] If the tap event is determined to be valid based on the current tap data of at least two user-triggered tap events, the tap type and / or tap operation content corresponding to the tap event is determined.

[0187] In some embodiments, determining the tap type and / or tap operation content corresponding to the tap event includes:

[0188] Based on current tap data of at least two user-triggered tap events, a tap type and / or a tap operation content of the tap event is determined.

[0189] In some embodiments, the above-mentioned knocking type includes at least one of continuous knocking and single-clicking, and the knocking operation content includes at least one of raising the window, lowering the window, and opening the door.

[0190] As described above, the above-mentioned tapping recognition device is used for tapping recognition, and only the tapping data collected daily by each user at different times can be used for tapping recognition, which can effectively avoid the situation where a large amount of tapping data of the user is specially collected during non-use periods. It can not only improve the accuracy of tapping recognition, but also enhance the user experience of multiple users, thereby better meeting the needs of multiple users.

[0191] The present application also provides an electronic device, such as Figure 7 As shown, the electronic device includes: a processor 21 and a memory 22, the memory 22 stores programs or instructions that can be run on the processor, and the above programs or instructions are executed by the processor to achieve the following Figure 1-Figure 5 The steps of the knock recognition method.

[0192] The present application also provides a vehicle, such as Figure 8 As shown, the vehicle includes Figure 7 The electronic device 31 shown in the figure, the above-mentioned vehicle can collect the user's tapping data at different times on a daily basis through the elastic wave sensor installed thereon, and then use these tapping data to perform tapping recognition, which can improve the accuracy of the vehicle's tapping recognition, thereby not only enhancing the overall performance of the vehicle, but also enabling it to better adapt to the operating habits of different users, and thus better meet the usage needs of multiple users.

[0193] The present application also provides a computer-readable storage medium, wherein a program or instruction is stored on the computer-readable storage medium, and when the program or instruction is executed by a processor, the following is realized: Figure 1-Figure 5 The steps of the knock recognition method.

[0194] The present application also provides a computer program product, which, when executed by a processor of a vehicle, implements the following Figure 1-Figure 5 The steps of the knock recognition method.

[0195] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0196] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general-purpose hardware platform, or of course, by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes a number of instructions for enabling a terminal or network-side device to execute the methods described in each embodiment of the present application.

[0197] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of this application and the scope of protection of the claims. These implementation methods are all within the protection of this application.

Claims

1. A knock recognition method, characterized in that: include: Get the user's current tapping data; Knock recognition is performed based on the current knock data of the user and a historical knock data set of the same user to obtain a knock recognition result.

2. The method according to claim 1, characterized in that Before obtaining the current tapping data of the user, the method further includes: Determines whether the user triggered a tap event.

3. The method according to claim 1, wherein the performing of tap recognition based on the current tap data of the user and the historical tap data set of the same user comprises: Determining a current feature vector of the current tapping data of the user; Determining a historical feature vector based on the historical knock data set; Knock recognition is performed based on the current feature vector of the current knock data and the historical feature vector.

4. The method according to claim 3, characterized in that The determining of the historical feature vector based on the historical knocking data set includes: The historical feature vector is determined based on M historical knock data in the historical knock data set, where M is an integer greater than or equal to 1.

5. The method according to claim 4, characterized in that The M historical knocking data are the first M historical knocking data with the highest confidence ranking in the historical knocking data set.

6. The method according to claim 4, characterized in that The determining the historical feature vector based on the M historical knock data in the historical knock data set includes: Determine M historical feature vectors based on the M historical knocking data; A weighted calculation is performed on the M historical feature vectors, and the obtained weighted feature vector is used as the final historical feature vector.

7. The method according to claim 4, characterized in that The determining the historical feature vector based on the M historical knock data in the historical knock data set includes: M historical feature vectors are determined based on the M historical knocking data, and the historical knocking data and the historical feature vectors correspond one to one.

8. The method according to any one of claims 3 to 7, characterized in that: The performing knock recognition based on the current feature vector of the current knock data and the historical feature vector includes: Determining the similarity between the current feature vector and the historical feature vector; Knock recognition is performed based on the similarity and a preset similarity threshold.

9. The method according to claim 8, characterized in that Determining the similarity between the current feature vector and the historical feature vector includes: Determine the similarity between the current feature vector and the weighted feature vector.

10. The method according to claim 8, characterized in that Determining the similarity between the current feature vector and the historical feature vector includes: Determining first similarities between the current feature vector and each of the M historical feature vectors; A weighted calculation is performed on the M first similarities to obtain a second similarity.

11. The method according to claim 8, characterized in that Determining the similarity between the current feature vector and the historical feature vector includes: The similarity between the current feature vector and the historical feature vector is determined based on a similarity calculation method.

12. The method according to claim 11, characterized in that The similarity calculation method includes at least one of cosine similarity, Mahalanobis distance and Euclidean distance.

13. The method according to claim 8, characterized in that The performing tap recognition based on the similarity and a preset similarity threshold to obtain a tap recognition result includes: When the similarity is greater than or equal to the preset similarity threshold, it is determined that the user's tapping event is valid.

14. The method according to claim 8, characterized in that Also includes: When the similarity is less than the preset similarity threshold, tap recognition is performed based on the current feature vector and a single-click classification model.

15. The method according to claim 3, characterized in that Before performing the knock recognition based on the current feature vector of the current knock data and the historical feature vector, the method further includes: It is determined that the amount of historical knock data in the historical knock data set is greater than or equal to a preset amount threshold.

16. The method according to claim 15, characterized in that If the amount of historical knock data in the historical knock data set is less than a preset amount threshold, the method further includes: Perform tap recognition based on the current feature vector and a single-tap classification model.

17. The method according to any one of claims 1 to 16, characterized in that: Also includes: The historical knock data set is updated based on the current knock data.

18. The method according to any one of claims 1 to 16, characterized in that: The knock recognition result includes that the knock event is valid, and the method further includes: If the tap event is determined to be valid based on the current tap data of at least two user-triggered tap events, the tap type and / or tap operation content corresponding to the tap event is determined.

19. The method according to claim 18, characterized in that Determining the tap type and / or tap operation content corresponding to the tap event includes: Based on the current tap data of the at least two user-triggered tap events, the tap type and / or tap operation content of the tap event is determined.

20. The method according to any one of claims 18-19, characterized in that: The knocking type includes at least one of continuous knocking and single knocking, and the knocking operation content includes at least one of raising a window, lowering a window, and opening a door.

21. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the tap recognition method according to any one of claims 1 to 20 are implemented.

22. A vehicle, characterized in that: The electronic device comprising claim 21.

23. A computer-readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the tap recognition method according to any one of claims 1 to 20 are implemented.

24. A computer program product, characterized in that When the program product is executed by a processor of an electronic device, the steps of the tap recognition method according to any one of claims 1 to 20 are implemented.

Images