Finger pressing state detection method, and training method, electronic device and medium

Abstract

The present application relates to the technical field of signal detection. Disclosed are a finger pressing state detection method, and a training method, an electronic device and a medium, which are used for improving the generalization capability of finger pressing state detection. The finger pressing state detection method comprises: preprocessing an ultrasonic echo signal to obtain model input data, wherein the ultrasonic echo signal is a corresponding ultrasonic echo signal after an ultrasonic array transmits an ultrasonic signal to a pressing region; and inputting the model input data into a trained neural network model, such that the neural network model outputs a finger pressing state.

Description

Finger pressing state detection, training method, electronic device and medium

[0001] The present application claims priority to the Chinese application No. 202411126520.7, filed on August 15, 2024, entitled "Finger pressing state detection, training method, electronic device and medium", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of signal detection, in particular to a finger pressing state detection, training method, electronic device and medium. BACKGROUND

[0003] When using electronic devices such as mobile phones, computers, etc. containing touch control functions, users need to touch the screen or other touch control positions with their fingers to operate or identify. For example, touch unlocking, single-click confirmation, double-click screenshot, etc. Generally, the electronic device first captures the action and then triggers the corresponding touch control function.

[0004] An ultrasonic system can emit ultrasonic waves, receive reflected sound waves, and obtain digital signals after a series of signal conversions. Due to the influence of acoustic impedance on sound wave propagation, there are differences in acoustic impedance of different reflection surfaces, and there are also differences in ultrasonic echo intensity. Therefore, there are differences in ultrasonic echo intensity between the finger and the touch control contact surface, and between the air and the touch control contact surface. The above differences can be used to detect user finger actions and determine the finger pressing state.

[0005] In the traditional scheme, under the condition of keeping the ultrasonic wave emission frequency, temperature, etc. the same, the finger pressing state (lifted or pressed, etc.) can be detected by setting a reference threshold value. However, the ultrasonic signal is affected by factors such as emission frequency, temperature, ultrasonic wave flight time, and ultrasonic module difference, and the signal intensity under different conditions has a large difference. Therefore, to determine the finger pressing state by threshold value, a large amount of data under different conditions needs to be covered and the corresponding reference threshold value needs to be set, which has certain difficulty in covering all different situations, resulting in poor generalization ability of the traditional detection scheme. SUMMARY

[0006] The present application provides a finger pressing state detection, training method, electronic device and medium to solve the technical problem of poor generalization ability in the traditional finger pressing state detection scheme.

[0007] Based on the above technical problems, the following solutions are provided:

[0008] In a first aspect, a finger pressing state detection method is provided, the method comprising:

[0009] The ultrasonic echo signals are preprocessed to obtain model input data, the ultrasonic echo signals being corresponding to ultrasonic echo signals after an ultrasonic array emits ultrasonic signals to a pressing area;

[0010] The model input data is input into the trained neural network model, so that the neural network model outputs a finger pressing state.

[0011] In an implementation manner, the neural network model comprises a neural network model constructed based on a recurrent neural network, or a neural network model constructed based on a convolutional neural network.

[0012] In an implementation manner,

[0013] The neural network model comprises a first classification decision layer and a multi-layer recurrent neural network, an output of the multi-layer recurrent neural network being connected to the first classification decision layer;

[0014] Or;

[0015] The neural network model comprises a second classification decision layer and a single-layer recurrent neural network, an output of the single-layer recurrent neural network being connected to the second classification decision layer.

[0016] In an implementation manner, the preprocessing of the ultrasonic echo signals to obtain the model input data comprises:

[0017] Single-frame ultrasonic echo signals are spatially multi-point sampled to obtain spatial sampling data corresponding to the single-frame ultrasonic echo signals;

[0018] The spatial sampling data is converted into a first data vector;

[0019] The model input data of the neural network model is obtained according to the first data vector.

[0020] In an implementation manner, the spatial sampling of the single-frame ultrasonic echo signals to obtain the spatial sampling data corresponding to the single-frame ultrasonic echo signals comprises:

[0021] The single-frame ultrasonic echo signals are respectively spatially sampled according to preset row-column intervals to obtain the spatial sampling data corresponding to the single-frame ultrasonic echo signals.

[0022] In an implementation manner, the obtaining of the model input data of the neural network model according to the first data vector comprises:

[0023] The first data vector is normalized, and the normalized first data vector is the model input data of the neural network model.

[0024] In an implementation manner, the pre-processing of the ultrasonic echo signal to obtain the model input data comprises:

[0025] Converting an echo intensity feature value of a single frame of the ultrasonic echo signal collected at a current time step and echo intensity feature values corresponding to other time steps respectively into a second data vector, the other time steps being time steps before the current time step;

[0026] Obtaining model input data of the neural network model according to the second data vector.

[0027] In an implementation manner, the other time steps are N-1 continuous time steps before the current time step N.

[0028] In an implementation manner, the obtaining of the model input data of the neural network model according to the second data vector comprises:

[0029] Performing normalization processing on the second data vector, and the second data vector after the normalization processing being the model input data of the neural network model.

[0030] In an implementation manner, the finger pressing state output by the neural network model comprises a classification determined state or a classification pending state, and the classification determined state is any pressing state in at least two categories of pressing states.

[0031] In an implementation manner, the at least two categories of pressing states comprise a finger touch state and a finger non-touch state, or the at least two categories of pressing states comprise a finger touch state, a finger non-touch state, a critical state from touch to non-touch, and a critical state from non-touch to touch.

[0032] In an implementation manner, after the inputting of the model input data into the trained neural network model to make the neural network model output a finger pressing state, the method further comprises:

[0033] When the output finger pressing state is the classification determined state, taking the classification determined state as a final finger pressing state, and updating the output classification determined state to the state memory sequence in time sequence;

[0034] When the output finger pressing state is the classification pending state, taking a pressing state recorded last in the state memory sequence as a final finger pressing state, or taking a pressing state with the highest state probability in the state memory sequence as a final output finger pressing state, and updating the state memory sequence correspondingly.

[0035] In a second aspect, a neural network model training method is provided, and the method comprises:

[0036] obtaining training sample data, the training sample data comprising finger pressing state sample data and corresponding pressing state classification labels, the finger pressing state sample data being constructed based on an ultrasonic echo sample signal, the ultrasonic echo sample signal being an ultrasonic reflection signal after an ultrasonic wave sample signal is emitted to a pressing area;

[0037] training a neural network model based on the training sample data until a neural network model satisfying a preset model condition is obtained;

[0038] the trained neural network model is used to output a finger pressing state.

[0039] In an implementation manner, the neural network model comprises a neural network model constructed based on a recurrent neural network; or a neural network model constructed based on a convolutional neural network.

[0040] In an implementation manner:

[0041] the neural network model comprises a first classification decision layer and a multi-layer recurrent neural network, an output of the multi-layer recurrent neural network being connected to the first classification decision layer;

[0042] or;

[0043] the neural network model comprises a second classification decision layer and a single-layer recurrent neural network, an output of the single-layer recurrent neural network being connected to the second classification decision layer.

[0044] In an implementation manner, the finger pressing state sample data comprises a first sample data vector;

[0045] the first sample data vector is converted from spatial domain sampling sample data, the spatial domain sampling sample data being obtained based on spatial domain multi-point sampling of a single-frame ultrasonic echo sample signal.

[0046] In an implementation manner, the spatial domain sampling sample data is obtained by spatial domain sampling of the single-frame ultrasonic echo sample signal at a preset interval.

[0047] In an implementation manner, the first sample data vector is a normalized sample data vector.

[0048] In an implementation manner, the finger pressing state sample data comprises a second sample data vector;

[0049] the second sample data vector is converted from echo intensity feature values corresponding to multiple single-frame ultrasonic echo sample signals, the multiple single-frame ultrasonic echo sample signals being single-frame ultrasonic echo sample signals corresponding to multiple different time steps respectively.

[0050] In an implementation manner, the plurality of different time steps are continuous time steps.

[0051] In an implementation manner, the second sample data vector is a normalized sample data vector.

[0052] In an implementation manner, the finger pressing state output by the neural network model includes a classified determined state or a classified pending state, and the classified determined state is any pressing state of at least two categories of pressing states.

[0053] In an implementation manner, the at least two categories of pressing states include a finger touch state and a finger non-touch state, or the at least two categories of pressing states include a finger touch state, a finger non-touch state, a critical state from touch to non-touch, and a critical state from non-touch to touch.

[0054] In a third aspect, a finger pressing state detection apparatus is provided, and the finger pressing state detection apparatus includes:

[0055] A processing module is configured to pre-process an ultrasonic echo signal to obtain model input data, the ultrasonic echo signal being an ultrasonic echo signal corresponding to an ultrasonic wave signal emitted by an ultrasonic array to a pressing area;

[0056] An input module is configured to input the model input data into a trained neural network model, so that the neural network model outputs a finger pressing state.

[0057] In a fourth aspect, a neural network model training apparatus is provided, and the neural network model training apparatus includes:

[0058] An acquisition module is configured to acquire training sample data, the training sample data including finger pressing state sample data and corresponding pressing state classification labels, the finger pressing state sample data being constructed based on an ultrasonic echo sample signal, the ultrasonic echo sample signal being an ultrasonic reflection signal after an ultrasonic wave sample signal is emitted to a pressing area;

[0059] A training module is configured to train a neural network model based on the training sample data until a neural network model satisfying a preset model condition is obtained, the trained neural network model being configured to output a finger pressing state.

[0060] In a fifth aspect, an electronic device includes an ultrasonic piezoelectric sensor, an analog-to-digital converter, a data processing center, and a central controller, wherein the ultrasonic piezoelectric sensor array, the analog-to-digital converter, and the data processing center are connected to the central controller.

[0061] The ultrasonic piezoelectric sensor is configured to transmit ultrasonic signals to the pressing area and receive ultrasonic echo signals in response to the central controller.

[0062] The analog-to-digital converter is configured to convert the ultrasonic echo signals into digital signals in response to the central controller.

[0063] The data processing center is configured to implement the steps of the finger pressing state detection method according to any one of the preceding embodiments in response to the central controller.

[0064] In a sixth aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the neural network model training method according to any one of the preceding embodiments when executing the computer program.

[0065] In a seventh aspect, a readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the method according to any one of the preceding embodiments when executed by a processor.

[0066] In an eighth aspect, a computer program product is provided, which includes a computer program, and the computer program implements the steps of the method according to any one of the preceding embodiments when executed by a processor.

[0067] In one of the above-mentioned implementations, the model input data is obtained by preprocessing the ultrasonic echo signals, and the model input data is input into the trained neural network model to make the neural network model output the corresponding finger pressing state. The intensity features of the ultrasonic echo signals corresponding to different finger pressing states are different, and the preprocessed ultrasonic echo signals, i.e., the model input data, represent the changes in the intensity features. The trained neural network model outputs the finger pressing state based on the model input data, without the need for setting a reference threshold comparison mode, which can eliminate the differences between different ultrasonic modules (such as different ultrasonic piezoelectric sensors used by different electronic devices), the same network model parameters have better generalization ability between different ultrasonic modules, and can effectively eliminate the differences in various different scenarios (temperature, frequency, etc.) and different finger states, can cover a variety of special scenarios such as high temperature, low temperature, and wet fingers, and has stronger generalization ability and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without any creative labor.

[0069] Fig. 1 is a schematic diagram of a system framework of a processing system according to an embodiment of the present application;

[0070] Fig. 2 is a schematic diagram of a principle of signal emission model and signal receiving model of an ultrasonic array according to an embodiment of the present application;

[0071] Fig. 3 is a schematic diagram of a comparison of ultrasonic echo signal intensity in a finger touch state and a finger non-touch state according to an embodiment of the present application;

[0072] Fig. 4 is a schematic diagram of a flow of a finger pressing state detection method according to an embodiment of the present application;

[0073] Fig. 5 is a schematic diagram of a model architecture of a neural network model based on a multi-layer recurrent neural network according to an embodiment of the present application;

[0074] Fig. 6 is a schematic diagram of a model architecture of a neural network model based on a single-layer recurrent neural network according to an embodiment of the present application;

[0075] Fig. 7 is a schematic diagram of a neuron node relationship after unfolding of a hidden layer in Fig. 5 or Fig. 6;

[0076] Fig. 8 is a schematic diagram of a data processing flow when a spatial multi-point detection scheme is used in a finger pressing state detection method according to an embodiment of the present application;

[0077] Fig. 9 is a schematic diagram of a data processing flow when a time sequence detection scheme is used in a finger pressing state detection method according to an embodiment of the present application;

[0078] Fig. 10 is a schematic diagram of a comparison of a falling edge of a down state and a rising edge of an up state according to an embodiment of the present application;

[0079] Fig. 11 is a schematic diagram of a flow of decision-making with assistance of a state memory sequence according to an embodiment of the present application;

[0080] Fig. 12 is a schematic diagram of another flow of decision-making with assistance of a state memory sequence according to an embodiment of the present application;

[0081] Fig. 13 is a schematic diagram of a structure of a finger pressing state detection apparatus according to an embodiment of the present application;

[0082] Fig. 14 is a schematic diagram of a structure of a neural network model training apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0083] Clearly, the described embodiments are only some but not all of embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort should be within the scope of the present application.

[0084] It is to be understood that the application can assume various alternative forms of embodiment, and it is not to be limited to the embodiments set forth and described herein. Rather, the embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. In the drawings, like reference numerals refer to like elements throughout the several views, and the size and relative sizes of layers and regions can be exaggerated in some of the drawings for clarity.

[0085] It will be understood that when an element or layer is referred to as being "on" or "adjacent" or "connected" or "coupled" to another element or layer, it can be directly on, adjacent, connected or coupled to the other element or layer or one or more intervening elements or layers can be present. In contrast, when an element is referred to as being "directly on," "directly adjacent," "directly connected" or "directly coupled" to another element or layer, there are no intervening elements or layers present. It will be understood that, although the terms first, second, third, etc. can be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the present application.

[0086] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0087] For a thorough understanding of the present application, reference should be made to the following detailed description, in conjunction with the accompanying drawings, in which:

[0088] The embodiment of the present application provides a finger pressing state detection scheme, which is used for detecting a finger pressing state, for example, including but not limited to a finger touch state and a finger non-touch state. Based on the detected finger pressing state, the scheme can be applied to a fingerprint identification scene or other scenes containing a touch function, for example, when a user uses an electronic device such as a mobile phone or a computer, the user needs to touch and press a region (such as a screen or other touchable regions, such as an entity contact plate surface) to operate or identify. The touch function includes but is not limited to, for example, touch unlocking, single-click confirmation, double-click screenshot, and the like, and the specific implementation is not limited. Generally, the electronic device needs to capture the finger pressing state and then trigger the corresponding touch function. In addition, the electronic device includes a product device such as a mobile phone, a computer, a tablet and the like, and the specific implementation is not limited.

[0089] In order to facilitate understanding of the embodiment of the present application, a processing system framework used in the embodiment of the present application is introduced first, as shown in FIG. 1, the processing system framework provided by the present application includes a finger pressing state detection device and a touch function device, wherein in the example of FIG. 1, the touch function device includes but is not limited to a fingerprint unlocking function device, and can also be other touch function systems, and the embodiment of the present application is not limited. The following describes the processing system taking the fingerprint unlocking as an example.

[0090] As a framework example, the finger pressing state detection device includes an ultrasonic piezoelectric sensor, an analog-to-digital converter, a storage unit, a data processing center and a central controller, wherein the ultrasonic piezoelectric sensor, the analog-to-digital converter, the storage unit and the data processing center are connected with the central controller. The ultrasonic piezoelectric sensor includes an ultrasonic array and a coupling layer structure, and the ultrasonic array can be a PVDF (polyvinylidene fluoride) array, and the specific implementation is not limited. The analog-to-digital converter can be a high-precision analog-to-digital converter, and the specific implementation is not limited. The central controller is used for controlling the ultrasonic piezoelectric sensor array to complete excitation and recovery of the ultrasonic array, the analog-to-digital converter is used for completing digital signal conversion of the analog ultrasonic echo signal, the ultrasonic echo signal after analog-to-digital conversion is transmitted to the data processing center through the storage unit, and the data processing center detects the finger pressing state based on the ultrasonic echo signal by using the finger pressing state detection method provided by the embodiment of the present application.

[0091] The fingerprint unlocking function device includes a fingerprint processing algorithm module, which performs preliminary processing of fingerprint data after detecting the finger pressing state, and further processes through a fingerprint identification algorithm to complete functions such as related fingerprint registration and fingerprint identification unlocking.

[0092] It should be noted that FIG. 1 is only a schematic, and does not limit the actual processing framework.

[0093] Continuing to refer to FIG. 2, FIG. 2 shows a schematic diagram of the ultrasonic signal emission model and the ultrasonic echo signal receiving model of the ultrasonic piezoelectric sensor, taking the pressing area of the screen as an example. In the diagram, the ultrasonic array of the ultrasonic piezoelectric sensor emits ultrasonic signals to the screen at a fixed frequency, and the ultrasonic signals penetrate the screen to reach the ridge and valley positions of the finger skin layer. Since the screen is in contact with the ridge (fingerprint protrusion) and the valley (fingerprint depression) of the finger is not in contact with the screen, the ultrasonic echo signal in the no-finger-coverage state is taken as Base, and the ultrasonic echo signal in the finger-coverage state is taken as Touch. The difference between Touch and Base is the texture image of the ridge, i.e., the fingerprint texture.

[0094] Since there is a large acoustic impedance difference between the human skin and the air, the reflectivity of the ultrasonic signals is different when they propagate to different contact surfaces. Therefore, the ultrasonic echo signal intensity in the finger-touch state is different from that in the air-sampling state (no coverage on the screen, i.e., the non-touch state of the finger). The difference can be used to determine the finger pressing state. The reflectivity calculation method in the touch and non-touch states can be as follows:

[0095] wherein z air ,z finger ,z s respectively represent the impedance of the air, the impedance of the human skin, and the impedance of the screen.

[0096] In general, the impedance of the screen is much larger than that of the air. The impedance of the air is about 430 Rayl, and the impedance of the currently used screen is about 13 MRayl. In the non-touch state of the finger, when the ultrasonic wave propagates from the screen to the air in the screen contact surface, the impedance of the air can be ignored, and the ultrasonic signal is almost fully reflected. In the touch state of the finger, reflection and transmission occur simultaneously, part of the ultrasonic signal passes through the finger skin, and part of the ultrasonic signal is reflected. Therefore, the ultrasonic echo signal intensity is greater in the non-touch state of the finger, and the ultrasonic echo signal intensity decreases significantly in the touch state of the finger. From the above analysis, it can be seen that the ultrasonic echo signal intensity changes in different finger pressing states. According to the above ultrasonic echo signal intensity variation characteristics, the finger pressing state can be determined.

[0097] As introduced in the background, in the traditional scheme, a large amount of data covering different scene conditions is needed, and corresponding benchmark thresholds are set, and it is difficult to cover all different situations, for example, the signal strength under the same condition itself also has a large difference, and the ultrasonic echo signal changes more when the finger is in different states, such as when the finger is dry, the ultrasonic echo signal strength is lower, and when the finger is wet or the finger is wet, the ultrasonic echo signal strength is higher; in addition, under different temperatures, the signal strength is also different, under high temperature conditions, the ultrasonic echo signal strength is higher, and under low temperature conditions, the ultrasonic echo signal strength is smaller. The generalization ability of the scheme is poor, and the accuracy is also low.

[0098] Therefore, in the embodiments of the present application, based on the large difference in ultrasonic echo signal strength under different conditions, the problem of poor generalization of the traditional scheme is proposed. A finger pressing state detection scheme based on a neural network model is proposed, which will be described in detail below.

[0099] As shown in FIG. 4, taking the finger pressing state detection system shown in FIG. 1 as an example, the embodiments of the present application provide a finger pressing state detection method based on a neural network model, which includes the following steps:

[0100] S10: Preprocess the ultrasonic echo signal to obtain model input data, the ultrasonic echo signal being the ultrasonic echo signal corresponding to the ultrasonic array emitting ultrasonic signals to the pressing area.

[0101] The electronic device includes a finger pressing state detection apparatus and a pressing area, which can be a touch screen or other touch position, such as a touch panel, etc., without limitation. The electronic device uses the ultrasonic array of the ultrasonic piezoelectric sensor to emit ultrasonic signals to the pressing area at a fixed frequency, and uses the ultrasonic array of the ultrasonic piezoelectric sensor to receive the reflected signal formed by the emitted ultrasonic signal, that is, the ultrasonic echo signal, which is converted into a digital form by an analog-to-digital converter and reaches the data processing center through a storage unit. The data processing center preprocesses the ultrasonic echo signal in the storage unit to obtain model input data.

[0102] S20: Input the model input data into the trained neural network model, so that the neural network model outputs the finger pressing state.

[0103] In the embodiment of the present application, the neural network model based on the output of the ultrasonic echo signal for finger pressing state is pre-trained. The neural network model is trained based on a large amount of training sample data. The neural network model includes but is not limited to a neural network model constructed based on a recurrent neural network (RNN) or a convolutional neural network (CNN). The network parameters of the neural network model are obtained by training the finger pressing data covering multiple conditions such as multiple scenes and multiple temperatures constructed in the early stage. The finger pressing data obtained in each scene has pressing classification label information. In the model training, the model can be trained by setting the model training mode until the required neural network model is obtained, that is, the trained neural network model is obtained. For the model training content, refer to the description of the subsequent embodiments.

[0104] In this step, the data processing center pre-processes the ultrasonic echo signal to obtain model input data, and inputs the model input data into the trained neural network model to make the neural network model output the corresponding finger pressing state. For example, the trained neural network model can be built-in in the electronic device or deployed on the server side for model calling. The specific implementation is not limited.

[0105] It can be seen that in this embodiment, a finger pressing state detection method based on a neural network model is provided. The model input data is obtained by pre-processing the ultrasonic echo signal, and the model input data is input into the trained neural network model to make the neural network model output the corresponding finger pressing state. The intensity characteristics of the ultrasonic echo signals corresponding to different finger pressing states are different. The pre-processed ultrasonic echo signal, that is, the model input data, represents the intensity characteristic change. The trained neural network model outputs the finger pressing state based on the model input data. It is not necessary to compare the reference threshold value, which can eliminate the difference between different ultrasonic modules (such as different ultrasonic piezoelectric sensors used in different electronic devices). The same network model parameters have better generalization ability between different ultrasonic modules. Moreover, the difference between different scenes (temperature, frequency, etc.) and different finger states can be effectively eliminated, and multiple special scenes such as high temperature, low temperature, and wet fingers can be covered. The method has stronger generalization ability and accuracy.

[0106] It should be noted that, in the embodiments of the present application, the neural network model can be constructed based on a recurrent neural network (RNN) or a convolutional neural network (CNN), and the specific implementation is not limited. Taking the construction of the neural network model based on the recurrent neural network as an example, in an embodiment, the neural network model includes a first classification decision layer and a multi-layer recurrent neural network, and the output of the multi-layer recurrent neural network is connected to the corresponding first classification decision layer; or the neural network model includes a second classification decision layer and a single-layer recurrent neural network, and the output of the single-layer recurrent neural network is connected to the second classification decision layer. That is, the recurrent neural network used can be a single-layer recurrent neural network or a multi-layer recurrent neural network. It should be understood that the first classification decision layer and the second classification decision layer are only expressions used to distinguish between single-layer and multi-layer networks, and the two can be the same classification decision layer.

[0107] The single-layer recurrent neural network or the multi-layer recurrent neural network used can be determined according to actual needs, and the specific implementation is not limited. Since the finger pressing state detection is a pressing state detection process over a relatively long period of time, and in the recurrent neural network, each current state is related to the previous state or the subsequent state, therefore, by using the neural network model based on the recurrent neural network, the association of the previous or subsequent state will be beneficial to more accurately detecting the finger pressing state.

[0108] To facilitate understanding of the subsequent detection scheme, the neural network model constructed based on the recurrent neural network provided in the embodiments of the present application will be introduced first.

[0109] As an example, as shown in FIG. 5, the neural network model includes a first classification decision layer and a multi-layer recurrent neural network, and the output of the multi-layer recurrent neural network is connected to the first classification decision layer. Since the multi-layer recurrent neural network is related to the time sequence state, it is expanded according to the time sequence state, then the multi-layer recurrent neural network includes hidden layer networks corresponding to different time sequences respectively, and the hidden layer networks respectively represent network architectures corresponding to different time sequences. As can be seen, the hidden layer network includes n layers of hidden layers, n≥2, the n layers of hidden layers in the hidden layer network are connected in turn, and the output of the hidden layer of the hidden layer network of the previous time sequence is transmitted to the hidden layer of the same layer of the next time sequence, and the output of the last layer of the hidden layer network, that is, the output of the multi-layer recurrent neural network, is connected to the first classification decision layer. The classification decision layer in FIG. 5 is the first classification decision layer.

[0110] Specifically, please continue to refer to FIG. 5, in this example, according to the time sequence expansion, x t0 ,x t1 ,...,x tn respectively represent the input data of the hidden layer network corresponding to different time sequences t0-tn in the multi-layer recurrent neural network, y t0,y t1 ,...,y tn represent the finger press state output by the corresponding time sequence classification decision layer. The hidden layer network includes hidden layer 1, hidden layer 2,..., hidden layer n, the output of hidden layer n is connected to the input of the classification decision layer, the value of each hidden layer of the hidden layer network of the previous time sequence is transmitted to the same layer hidden layer of the next time sequence, for example, the output of hidden layer 1 at time sequence t0 is transmitted to the hidden layer 1 corresponding to time sequence t1, the output of the hidden layer 1 corresponding to time sequence t1 is transmitted to the hidden layer 1 corresponding to time sequence t2, the transmission content includes the state vector of the hidden layer output, for example, the state vector 1 is transmitted from hidden layer 1 at time sequence t0 to hidden layer 1 at time sequence t1, the output of the next hidden layer is transmitted to the previous hidden layer, and the hidden layer relationship of the same layer or other layers is similar, which is not described in detail here. In addition, it is worth mentioning that there are weight matrix parameters in the multi-layer recurrent neural network, which will be introduced and explained later when introducing the single-layer recurrent neural network, and the specific number of hidden layers in the multi-layer recurrent neural network can be adjusted, and the size of the model input data can also be adjusted, which are not limited in the embodiments of the present application.

[0111] As an example, as shown in FIG. 6, the neural network model includes a second classification decision layer and a single-layer recurrent neural network, the output of the single-layer recurrent neural network is connected to the second classification decision layer, since the single-layer recurrent neural network is related to the time sequence state, it is expanded according to the time sequence state, then the single-layer recurrent neural network includes hidden layer networks corresponding to different time sequences respectively, the hidden layer network represents the network architecture corresponding to different time sequences respectively, the hidden layer network of the single-layer recurrent neural network only includes one layer of hidden layer, and the output of the hidden layer of the previous time sequence is transmitted to the hidden layer of the next time sequence, and the output of the hidden layer is connected to the second classification decision layer. Unlike the multi-layer recurrent neural network, the single-layer recurrent neural network only includes a single layer of hidden layer.

[0112] Specifically, as shown in FIG. 6, in the example of FIG. 6, three time sequences are taken as an example to expand and explain, x t-1 ,x t1 ,x t+1 represent the input data of the hidden layer network corresponding to the three time sequences respectively, y t-1 ,y t1 ,,y t+1 represent the finger press state output by the corresponding time sequence classification decision layer. The hidden layer network includes a hidden layer, the output of hidden layer 1 corresponding to time sequence t-1 is transmitted to the hidden layer at time sequence t1, and the output of the hidden layer corresponding to time sequence t1 is transmitted to the hidden layer corresponding to time sequence t+1.

[0113] In the above single-layer or multi-layer recurrent neural network, the weight matrix U, W, V is a linear relationship parameter in the recurrent neural network. Here, taking the single-layer recurrent neural network shown in FIG. 6 as an example, the working principle of the neural network model based on the single-layer recurrent neural network is further described. The network framework of the neural network model based on the single-layer recurrent neural network is as follows: y t = φ (O t ) O t = g (V · S t ) S t = f (U · x t + W · S t-1 )

[0114] Wherein, f(*), g(*) represents the activation function of the corresponding layer, φ(*) represents the classification decision algorithm adopted in the classification decision layer. Exemplarily, the classification decision algorithm can be a binary classification or multi-classification algorithm, which is not limited in particular. S t represents the value of the hidden layer corresponding to the time sequence t, that is, the aforementioned state vector, S t-1 represents the value of the hidden layer corresponding to the time sequence t-1, S t+1 represents the value of the hidden layer corresponding to the time sequence t+1, O t represents the output of the hidden layer corresponding to the time sequence t, O t-1 represents the output of the hidden layer corresponding to the time sequence t-1, O t+1 represents the output of the classification decision layer corresponding to the time sequence t+1, y t represents the output of the classification decision layer corresponding to the time sequence t, y t-1 represents the output of the classification decision layer corresponding to the time sequence t-1, y t+1 represents the output of the classification decision layer corresponding to the time sequence t+1, the weight matrix U represents the weight matrix from the model input to the hidden layer, the weight matrix V represents the weight matrix from the hidden layer to the classification decision layer, and the weight matrix W represents the weight matrix from the hidden layer of the previous time sequence to the hidden layer of the corresponding next time sequence. Therefore, the input vector After the classification decision layer outputs and makes a decision, the final output of the finger pressing state of the time sequence is obtained. It can be seen that the output state of each time sequence is not only affected by the current input, but also related to the information of the previous time sequence.

[0115] Further, in an embodiment, the number of neuron nodes contained in the hidden layer in the above single-layer or multi-layer recurrent neural network can also be adjusted, which can be trained according to the setting parameters. The embodiment of the present application is not limited. As an example, the network expansion of the single-layer recurrent neural network with nodes can be shown in FIG. 7.x t represents the input vector of the current time sequence t, h 11 , h 21 ,... hm1 the value of the hidden layer neuron node representing the time sequence t, h 12 , h 21 ,... h m2 the value of the hidden layer neuron node representing the previous time sequence t-1, f1, f2,... h c the activation function of the hidden layer representing the current time sequence t, y t the finger pressing state output by the classification decision layer at the time sequence t.

[0116] It should be noted that the above embodiment is based on a neural network model constructed based on a recurrent neural network, and based on this, at least two detection schemes are provided based on the neural network model constructed based on the recurrent neural network. The two detection schemes differ in the processing of model input data, and each has its corresponding benefits, which will be described below.

[0117] The first scheme is a spatial multi-point detection scheme

[0118] As shown in FIG. 8, in an embodiment, based on the above neural network model constructed based on a recurrent neural network, in step S10, that is, the ultrasonic echo signal is preprocessed to obtain model input data, including:

[0119] S101A: Spatial multi-point sampling is performed on a single frame of the ultrasonic echo signal to obtain spatial sampling data corresponding to the single frame of the ultrasonic echo signal;

[0120] S102A: The spatial sampling data is converted into a first data vector;

[0121] S103A: The model input data of the neural network model is obtained according to the first data vector.

[0122] This embodiment is one of the above spatial multi-point detection schemes. The data processing center obtains a single frame of the ultrasonic echo signal, performs spatial multi-point sampling on the single frame of the ultrasonic echo signal, and thus obtains spatial sampling data corresponding to the single frame of the ultrasonic echo signal.

[0123] As shown in FIG. 2, FIG. 2 takes the pressing area as an example to show the schematic diagram of the ultrasonic signal emission model and the ultrasonic echo signal receiving model of the ultrasonic piezoelectric sensor. In the model, the ultrasonic array of the ultrasonic piezoelectric sensor emits ultrasonic signals to the screen at a fixed frequency, and the ultrasonic signals penetrate the screen to reach the ridge and valley positions of the finger skin layer. Since the screen is in contact with the ridge (fingerprint protrusion), the valley (fingerprint depression) of the finger is not in contact with the screen, the ultrasonic echo signal in the no-finger-covering state is taken as the Base, and the ultrasonic echo signal in the finger-covering state is taken as the Touch. The difference between the Touch and the Base is the texture image of the ridge, i.e., the fingerprint texture.

[0124] Generally, the impedance of the screen is much larger than the impedance of the air, and the impedance of the air is about 430 Rayl. For example, assuming that the current screen impedance is about 13 MRayl, when the ultrasonic wave from the screen reaches the screen contact surface of the air in the no-finger-touching state, the impedance of the air can be ignored, and the ultrasonic signal is almost fully reflected. In the finger-touching state, reflection and transmission occur simultaneously, part of the ultrasonic signal passes through the finger skin, and part of the ultrasonic signal is reflected. Therefore, the ultrasonic echo signal intensity is greater in the no-finger-touching state, and the ultrasonic echo signal intensity decreases significantly in the finger-touching state. As shown in FIG. 3, assuming that 1000 frames of data are collected under the condition of 50 Hz collection frequency and repeated finger covering-lifting-covering-lifting actions for 20 seconds, the relationship between the ultrasonic echo signal and the finger pressing state is obtained as shown in FIG. 3. In the figure, up represents the no-finger-touching state, down represents the finger-touching state, the signal value is the full-screen intensity average, and the horizontal axis represents the frame number. As can be seen from FIG. 3, the ultrasonic echo signal intensity value corresponding to the ridge position is low when the finger is touching, and there is a difference in the ultrasonic echo signal intensity value between the valley position. Therefore, based on the above characteristics, the ultrasonic echo intensity variation feature can be extracted by using the spatial multi-point sampling method.

[0125] For example, assuming that the spatial size of a single frame of ultrasonic echo signal after analog-to-digital conversion is m*n, each point in the spatial domain of the single frame of ultrasonic echo signal represents the ultrasonic echo signal intensity corresponding to the point. The spatial multi-point detection scheme is to perform spatial multi-point sampling on the single frame of ultrasonic echo signal. Based on the obtained spatial sampling data, a first data vector matching the input size of the recurrent neural network is obtained by conversion. Then, the model input data of the neural network model is obtained according to the first data vector, and the model input data is input into the recurrent neural network in the neural network model. After the weight calculation of the recurrent neural network, the output of the recurrent neural network is given to the classification decision layer in the neural network model, so that the classification decision layer outputs the current finger pressing state.

[0126] It can be seen that in this embodiment, a finger pressing state detection method based on a neural network model is provided. The neural network model constructed based on the recurrent neural network is combined to perform spatial multi-point sampling on a single frame of ultrasonic echo signal to form network input of the recurrent neural network, so as to identify the finger pressing state. In this embodiment, in addition to the sampling reference threshold comparison mode which improves the applicability and generalization, the model input data size can also be reduced due to the use of multi-point spatial sampling, and the processing efficiency is improved.

[0127] In combination with the above embodiments, in an embodiment, in step S101A, that is, the spatial sampling on the collected single frame of ultrasonic echo signal is performed to obtain spatial sampling data corresponding to the single frame of ultrasonic echo signal, which includes: performing spatial sampling on the single frame of ultrasonic echo signal according to a preset row interval and a preset column interval to obtain spatial sampling data corresponding to the single frame of ultrasonic echo signal.

[0128] For example, assuming that the spatial size of a single frame of ultrasonic echo signal data is m*n, the m*n size of Base and Touch frames are sampled by a row interval of r1 and a column interval of c1 to obtain an m1*n1 matrix. Finally, the sampling matrix can form a 1*(m1*n1) matrix vector, that is, a first data vector, as the input of the aforementioned recurrent neural network, that is, the input of the neural network model. It should be noted that the row interval r1 and the column interval c1 can be set according to requirements, and are not limited in particular.

[0129] In this embodiment, when the ultrasonic echo signal is sampled by multiple points in space, the preset row and column intervals can be used for sampling. In addition to reducing the data size and improving the generalization ability, the interval sampling method can effectively reduce the loss of the change characteristics of the ultrasonic echo signal to ensure the detection accuracy.

[0130] Of course, in other embodiments, the specific sampling interval of the preset row and column intervals can be determined according to actual conditions, and the embodiments of the present application are not limited; in addition, a non-row and column interval spatial multi-point sampling method can also be used, and the embodiments of the present application are not limited in particular.

[0131] In an embodiment, in step S103A, that is, the model input data of the neural network model is obtained according to the first data vector, which includes: performing normalization processing on the first data vector, and the normalized first data vector is the model input data of the neural network model.

[0132] In this embodiment, the normalization processing of the first data vector obtained by conversion means that the normalization processing is performed on all vector elements in the first data vector, so that the element values in the first data vector are in a unified range. Therefore, the first data vector after the normalization processing is the model input data of the neural network model, which can effectively reduce the calculation amount of the subsequent neural network model, thereby improving the overall finger pressing state detection efficiency. It should be noted that in other embodiments, the first data vector after conversion can also be used as the model input data, which is not limited in particular.

[0133] The foregoing embodiments describe the spatial multipoint detection scheme provided by the embodiments of the present application. The following describes a second scheme provided by the embodiments of the present application.

[0134] The second scheme is a time sequence detection scheme

[0135] As shown in FIG. 9, in an embodiment, based on the neural network model constructed based on the recurrent neural network, in step S10, that is, the ultrasonic echo signal is preprocessed to obtain model input data, including:

[0136] S101B: converting the echo intensity feature value of the single frame of the ultrasonic echo signal collected at the current time step and the echo intensity feature values corresponding to other time steps respectively into a second data vector, the other time steps being time steps before the current time step;

[0137] S102B: obtaining model input data of the neural network model according to the second data vector.

[0138] This embodiment is one of the above-mentioned time sequence detection schemes. After the data processing center obtains the ultrasonic echo signals collected at each time step, it calculates the ultrasonic echo intensity feature values of the single frame of the ultrasonic echo signal at each time step, which correspond to the intensity values corresponding to each time step Taking the data time step sequence N as an example, assuming that the current time step is T N , the corresponding echo intensity feature value of the current time step T N is Then, this embodiment converts the echo intensity feature value of the single frame of the ultrasonic echo signal collected at the current time step T N and the echo intensity feature values corresponding to other time steps respectively into a second data vector matching the input size of the recurrent neural network The other time steps are time steps before the current time step T N , and then obtains the model input data of the neural network model according to the second data vector.

[0139] As shown in FIG. 10, which is a schematic diagram of ultrasonic echo intensity variation comparison between up state and down state, the horizontal coordinate represents time, and the vertical coordinate represents ultrasonic echo intensity. As can be seen from FIG. 10, there is an obvious falling edge in the state change from up to down (as shown in the left drawing in FIG. 10), and there is an obvious rising edge in the state change from down to up (as shown in the right drawing in FIG. 10). Therefore, the current finger pressing state can be further determined by the continuous time sequence variation of different frames.

[0140] It can be seen that in this embodiment, a finger pressing state detection method based on a neural network model is provided. The echo intensity feature values of the single frame of the ultrasonic echo signal collected at the current time step and the echo intensity feature values respectively corresponding to other time steps are converted to form network inputs of a recurrent neural network, so as to identify the finger pressing state. In this embodiment, in addition to the sampling reference threshold comparison mode, the applicability and generalization are improved. Since there is a change in the time domain when the finger is in the up-down action, the spatial sequence of multiple time steps is used as the model input, the finger pressing state can be detected by the spatial state change in time, the continuous action can be captured, and more scenes can be applied, such as completely covering the ultrasonic echo intensity change in the high-temperature or low-temperature scene, so as to realize more accurate or various finger pressing state detection.

[0141] In combination with the above embodiments, in an embodiment, the other time steps are the previous N-1 continuous time steps of the current time step N, that is, in this embodiment, the echo intensity feature values of the single frame of the ultrasonic echo signal collected at the current time step N and the echo intensity feature values respectively corresponding to the previous N-1 continuous time steps of the current time step N are converted into a second data vector, that is:

[0142] For example, denotes the second data vector, respectively denote the echo intensity feature values respectively corresponding to the current time step N and the previous N-1 continuous time steps of the current time step N.

[0143] In this embodiment, the finger pressing state can be detected by the spatial state change in time, the more accurate continuous action can be captured, and more accurate or various finger pressing state detection can be realized. Of course, in other embodiments, the other time steps can also be the echo intensity feature values respectively corresponding to the previous interval continuous time steps, and the specific embodiments of the present application are not limited.

[0144] In an embodiment, in step S102B, the model input data of the neural network model is obtained according to the second data vector, including: performing normalization processing on the second data vector, and the second data vector after the normalization processing is the model input data of the neural network model.

[0145] In this embodiment, the normalization processing on the transformed second data vector means that normalization processing is performed on all vector elements in the second data vector, so that the element values in the second data vector are within a unified range. Therefore, the second data vector after the normalization processing is the model input data of the neural network model, which can effectively reduce the calculation amount of the neural network model, thereby improving the overall finger pressing state detection efficiency. It should be noted that in other embodiments, the transformed second data vector can also be used as the model input data, which is not limited.

[0146] In an embodiment, the echo intensity feature value includes an echo intensity mean value of a single frame of the ultrasonic echo signal, or the echo intensity feature value includes a minimum echo intensity value in a single frame of the ultrasonic echo signal, which is not limited. The use of the echo intensity mean value can more comprehensively take the single frame signal feature, and the use of the minimum echo intensity value simplifies the calculation.

[0147] For example, in the above embodiment, the echo intensity mean value can be an arithmetic mean value or other forms of mean value, which is not limited in the embodiments of the present application and can be set according to the requirements to ensure the flexibility of the scheme.

[0148] It should be understood that in the above detection scheme embodiment, a neural network model based on a recurrent neural network is taken as an example for description, and in other embodiments, other neural network models, such as a convolutional neural network, can also be processed using the same detection scheme for model input data, which is not limited.

[0149] It should be noted that the classification decision layer in the above neural network model can use a binary classification algorithm or a three-classification or other multi-classification, which is not limited.

[0150] In an embodiment, the finger pressing state output by the neural network model includes a classification determined state or a classification pending state. The classification determined state is any pressing state of at least two categories of pressing states, and the classification determined state means that the pressing state is a clear state, which can be used for subsequent touch function applications, such as a finger touch state, a finger non-touch state, or other pressing states. The classification pending state means a pending state that needs to be further confirmed.

[0151] In the embodiment, the classification decision layer in the neural network model can adopt a binary classification algorithm or a three-class or other multi-classification algorithm, thereby improving the adaptability and application scenarios of the scheme and having higher application value.

[0152] In an embodiment, the at least two categories of pressing states include a finger touch state and a finger non-touch state; or the at least two categories of pressing states include a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, and a non-touch-to-touch critical state.

[0153] That is, in one case, the finger pressing state output by the neural network model is a finger touch state, a finger non-touch state, or a pending state; in another case, the finger pressing state output by the neural network model is a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, a non-touch-to-touch critical state, or a pending state.

[0154] Of course, in other embodiments, the finger pressing state output by the neural network model can also only have a classification determination state, for example, when a binary classification is adopted, in one case, the finger pressing state output by the neural network model is a finger touch state or a finger non-touch state; in another case, the finger pressing state output by the neural network model is a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, or a non-touch-to-touch critical state.

[0155] In combination with the above embodiments, in an embodiment, when the finger pressing state output by the neural network model includes a classification determination state or a classification pending state, after step S20, that is, after the model input data is input into the trained neural network model to make the neural network model output a finger pressing state, the method further includes:

[0156] S101: when the output finger pressing state is the classification determination state, the classification determination state is taken as the final finger pressing state, and the output classification determination state is updated to the state memory sequence in time sequence;

[0157] S102: when the output finger pressing state is the classification pending state, the last recorded pressing state in the state memory sequence is taken as the final finger pressing state, and is updated to the state memory sequence in time sequence.

[0158] For example, assuming that the classification determination state includes a first state and a second state, this embodiment assists the decision of the finger pressing state by setting a constantly updated state memory sequence, and the pending state means that the current neural network model cannot output the first state or the second state, but is in a condition that needs to further determine the final finger pressing state, in which case:

[0159] When the output finger pressing state is the first type of state, the first type of state is taken as the final finger pressing state, and the output first type of state is updated to the state memory sequence in time sequence;

[0160] When the output finger pressing state is the second type of state, the second type of state is taken as the final finger pressing state, and the output second type of state is updated to the state memory sequence in time sequence;

[0161] When the output finger pressing state is the classification pending state, the last recorded pressing state in the latest state memory sequence is taken as the final finger pressing state and is updated to the state memory sequence in time sequence.

[0162] Wherein, the first type of state and the second type of state are different finger pressing states, and the specific states are not limited, for example, in an embodiment, the first type of state is a finger touch state, and the second type of state is a finger non-touch state.

[0163] Taking the first type of state as a finger touch state and the second type of state as a finger non-touch state as an example, it should be understood that the finger can be in an up or down state for a long time, and in this case, the data, especially the time step sequence data form, can not exist a rising edge or a falling edge. For the above-mentioned case, a state memory sequence MemSeq is introduced, and a fixed length state memory sequence MemSeq is set. The classification decision layer includes multiple classification states, including 0-up, 1-down, and 2-maintain. That is, the classification decision layer outputs 0 to represent a finger non-touch state, outputs 1 to represent a finger touch state, and outputs 2 to represent a classification pending state. The final finger pressing state is determined by combining the output of the neural network model with the state memory sequence MemSeq.

[0164] Specifically, please refer to FIG. 11, when the finger pressing state output by the neural network model is a finger touch state, that is, the output of the classification decision layer O t = 1, then the final finger pressing state y t = 1, and the output finger touch state y t = 1 is updated to the state memory sequence MemSeq in time sequence; when the finger pressing state output by the neural network model is the finger non-touch state, that is, the output of the classification decision layer O t = 0, the final finger pressing state y t = 0, and the output finger non-touch state y t= 0 corresponds to update to the state memory sequence MemSeq; when the finger pressing state output by the neural network model is the pending state, i.e., the O t = 2, the last recorded state from the latest state memory sequence MemSeq is obtained as the final output finger pressing state, i.e., y t = MemSeq[-1], and the state is updated to the state memory sequence MemSeq. In general, taking the above three classification cases as examples, i.e.,

[0165] When the classification decision layer outputs the up or down state, the final output finger pressing state is the corresponding classification output state (up or down state), and the state memory sequence MemSeq is updated correspondingly;

[0166] When the classification decision layer outputs the maintain state, the previous state recorded by the state memory sequence MemSeq determines that the final output finger pressing state is up or down, and the state memory sequence MemSeq is updated correspondingly.

[0167] In this embodiment, the state memory sequence MemSeq assists in decision-making, which can improve the state richness, make the detection scheme more accurate, and also provide a corresponding processing mechanism when the neural network model cannot accurately output the up or down state, ensure the implementation of the scheme, and have a relatively high accuracy rate under different signal sampling frequencies.

[0168] In combination with the above embodiment, in an embodiment, when the finger pressing state output by the neural network model includes a classification determined state or a classification pending state, after step S20, i.e., after the model input data is input into the trained neural network model to make the neural network model output the finger pressing state, the method further includes:

[0169] S201: When the output finger pressing state is the classification determined state, the classification determined state is taken as the final finger pressing state, and the output classification determined state is updated to the state memory sequence in time sequence;

[0170] S202: When the output finger pressing state is the classification pending state, the pressing state with the highest state probability from the latest state memory sequence is obtained as the final output finger pressing state, and is updated to the state memory sequence correspondingly.

[0171] For example, in the case of three classifications, the finger pressing state of the neural network model includes a first-class state, a second-class state, or a classification pending state, where the pending state refers to a situation where the current neural network model cannot output whether it is a first-class state or a second-class state, but needs to further determine the final finger pressing state, in which case:

[0172] When the output finger pressing state is the first-class state, the first-class state is taken as the final finger pressing state, and the output first-class state is updated to the state memory sequence in chronological order;

[0173] When the output finger pressing state is the second-class state, the second-class state is taken as the final finger pressing state, and the output second-class state is updated to the state memory sequence in chronological order;

[0174] When the output finger pressing state is the classification pending state, the state with the highest probability from the latest state memory sequence is taken as the final finger pressing state, and is updated to the state memory sequence.

[0175] Wherein, the first-class state and the second-class state are different finger pressing states, and the specific states are not limited, for example, in an embodiment, the first-class state is a finger touch state, and the second-class state is a finger non-touch state.

[0176] In this embodiment, a state memory sequence MemSeq of a fixed length is also set, the classification decision layer includes multiple classification states, including 0-up, 1-down, and 2-maintain. The final finger pressing state is determined by combining the output of the neural network model with the state memory sequence MemSeq record. The difference from the above embodiment is that, in this embodiment, when the classification decision layer output is maintain, the highest probability of the state memory sequence MemSeq record is combined to determine the final output finger pressing state.

[0177] Taking the above three classifications as an example, please refer to FIG. 12, when the neural network model output finger pressing state is the finger touch state, i.e. the classification decision layer output O t = 1, the final finger pressing state y t = 1, and the output finger touch state y t = 1 is updated to the state memory sequence MemSeq in chronological order; when the neural network model output finger pressing state is the finger non-touch state, i.e. the classification decision layer output O t = 0, the final finger pressing state y t = 0, and the output finger non-touch state yt = 0 corresponds to updating to the state memory sequence MemSeq; when the finger pressing state output by the neural network model is the classification pending state, that is, the O t = 2, the pressing state with the highest state probability in the state memory sequence MemSeq is taken as the final finger pressing state, and specifically, as an example, it is first judged whether the number of times of 0 appearing in the state memory sequence MemSeq is greater than the number of times of 1 appearing, if yes, y t = 0; if not, y t = 1, and the last state is updated to the state memory sequence MemSeq. In general, that is, that is:

[0178] When the classification decision layer outputs the up or down state, the final finger pressing state is the corresponding classification output (up or down state), and the state memory sequence MemSeq is updated correspondingly;

[0179] When the classification decision layer outputs the maintain state, the state with the maximum probability of 0 or 1 recorded in the state memory sequence MemSeq is taken as the final finger pressing state, and the state memory sequence MemSeq is updated correspondingly.

[0180] In this embodiment, it is another scheme of assisting decision by the state memory sequence MemSeq, which can improve the state richness, make the detection scheme more accurate, and also provide a corresponding processing mechanism when the neural network model cannot accurately output the up or down state, ensure the realizability, and can have a relatively high accuracy rate under different signal sampling frequencies. In addition, since the previous state probability processing is adopted, the fault tolerance of the scheme in actual application can be higher.

[0181] It should be understood that in the above embodiment, the length of the state memory sequence MemSeq can be adjusted, which is not limited, and for example, it can be an infinite sequence with a length of 1 to infinity; it can also be a fixed sequence set according to requirements. Moreover, for other multi-classification cases, the state memory sequence can also be used to assist decision, and the above three classifications are only exemplary and do not limit the embodiments of the present application.

[0182] In an embodiment, the method can further determine the user finger action through the state recorded by the state memory sequence MemSeq, for example, the state memory sequence MemSeq [1, 1, 1, 1, 1, 1] can be regarded as a long press; the state memory sequence MemSeq [0, 0, 1, 1, 0, 0] can be regarded as a single click; the state memory sequence MemSeq [1, 1, 0, 0, 1, 1] can be regarded as a double click, so that more finger action detection can be realized.

[0183] In this embodiment, by further utilizing the state memory sequence MemSeq, the finger action can be quickly detected based on the previous finger pressing state detection result.

[0184] In an embodiment, when implementing the above-mentioned embodiment, the data acquisition frequency can be adjusted according to the requirements, not only 50Hz, but also other acquisition frequencies. The higher the data acquisition frequency, the more the strength gradual change points of the up-down process, which is beneficial to improve the detection accuracy and efficiency.

[0185] It should be noted that the above-mentioned embodiment takes the final output result of up or down as an example. In other embodiments, other output final classification states can also be included. The neural network model can be set for training according to the requirements, and the specific implementation is not limited.

[0186] In summary, the embodiment of the present application provides a finger pressing state detection method based on a neural network model, which has the following technical effects: without setting a reference threshold comparison method, the differences between different ultrasonic modules can be eliminated, the same network model parameters have better generalization ability between different ultrasonic modules, and the differences in various different scenarios (temperature, frequency, etc.) and different finger states can be effectively eliminated, which can cover various special scenarios such as high temperature, low temperature, and wet fingers, and has stronger generalization ability and accuracy.

[0187] The above-mentioned embodiment describes a finger pressing state detection method based on a neural network model provided by the embodiment of the present application. When implementing the above-mentioned method, a corresponding neural network model needs to be trained. The neural network model training method provided by the embodiment of the present application is described below.

[0188] In an embodiment, a neural network model training method is provided, which includes:

[0189] S01: Obtain training sample data, which includes finger pressing state sample data and its corresponding pressing state classification label. The finger pressing state sample data is constructed based on an ultrasonic echo sample signal, and the ultrasonic echo sample signal is an ultrasonic reflection signal after an ultrasonic wave sample signal is emitted to a pressing area;

[0190] S02: Based on the training sample data, train the neural network model until a neural network model that meets the preset model condition is obtained.

[0191] In this embodiment, the model architecture of the neural network model needs to be constructed first, and the sample data for training is trained, the sample data for training includes finger pressing state sample data and its corresponding pressing state classification label, the finger pressing state sample data is constructed based on the ultrasonic echo sample signal, and the ultrasonic echo sample signal is the ultrasonic reflection signal after the ultrasonic wave sample signal is emitted to the pressing area. Then, based on the training sample data, the neural network model is trained until the neural network model satisfying the preset model condition is obtained, and the trained neural network model is used to output the finger pressing state.

[0192] It should be understood that, as an example, the neural network model parameters are obtained by supervised learning on a large amount of training sample data, the model parameters of the neural network model are obtained by training the finger pressing state data set covering multiple scenes and multiple temperatures constructed in advance, and each scene has a classification label of pressing. In the specific training, the parameter value of the loss function can be solved by the stochastic gradient descent method, and the parameter is updated by the chain rule of reverse propagation until the optimal model parameter is obtained. The model parameters include model weight coefficients, and the preset model condition at least includes the optimal condition of the above model parameters.

[0193] It should be noted that, according to the application requirement, the required neural network model can be trained to match the input data in the application. In an embodiment, the above-mentioned neural network model includes but is not limited to a neural network model based on a recurrent neural network or a neural network model based on a convolutional neural network.

[0194] Further, the neural network model includes a first classification decision layer and a multi-layer recurrent neural network, the output of the multi-layer recurrent neural network is connected to the first classification decision layer;

[0195] Or;

[0196] The neural network model includes a second classification decision layer and a single-layer recurrent neural network, the output of the single-layer recurrent neural network is connected to the second classification decision layer.

[0197] The specific content of the model architecture can be referred to the foregoing description, which will not be repeated here.

[0198] It should be noted that, based on the foregoing description of the spatial multipoint detection scheme and the time sequence detection scheme, during the model training, the corresponding training data needs to be constructed for targeted application, which will be described below.

[0199] In an embodiment, the finger pressing state sample data includes a first sample data vector; the first sample data vector is obtained by converting the spatial sampling sample data, and the spatial sampling sample data is obtained by spatial multipoint sampling based on a single-frame ultrasonic echo sample signal.

[0200] In an embodiment, the spatial domain sampling sample data is obtained by spatial domain sampling a single frame of the ultrasonic echo sample signal at preset intervals.

[0201] In an embodiment, the first sample data vector is a normalized sample data vector.

[0202] In this embodiment, for the spatial domain multi-point detection scheme, the neural network model required by the spatial domain multi-point detection scheme can be trained accordingly. For more details, please refer to the description of the foregoing embodiments. The difference is that the model input data at this time is the training sample data.

[0203] In an embodiment, the finger pressing state sample data includes a second sample data vector.

[0204] The second sample data vector is converted from echo intensity feature values corresponding to multiple single-frame ultrasonic echo sample signals respectively, and the multiple single-frame ultrasonic echo sample signals are single-frame ultrasonic echo sample signals corresponding to multiple different time steps respectively.

[0205] In an embodiment, the echo intensity feature value includes an echo intensity mean value of a single frame of the ultrasonic echo signal, or the echo intensity feature value includes a minimum echo intensity value in a single frame of the ultrasonic echo signal.

[0206] In an embodiment, the multiple different time steps are consecutive time steps.

[0207] In an embodiment, the second sample data vector is a normalized sample data vector.

[0208] In this embodiment, for the time sequence detection scheme, the neural network model required by the time sequence detection scheme can be trained accordingly. For more details, please refer to the description of the foregoing embodiments. The difference is that the model input data at this time is the training sample data.

[0209] It should be noted that according to the detection requirement, the algorithm of the classification decision layer in the neural network model can be set, such as binary classification or ternary classification or other multi-classification, and the specific classification type of each classification. When constructing the sample data, set the corresponding classification label according to the algorithm of the classification decision layer, so that the output of the trained neural network model corresponds to the classification state.

[0210] In an embodiment, the finger pressing state output by the neural network model includes a classification determination state or a classification pending state, and the classification determination state is any pressing state of at least two categories of pressing states.

[0211] In an embodiment, the at least two categories of pressing states include a finger touch state and a finger non-touch state; or the at least two categories of pressing states include a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, and a non-touch-to-touch critical state.

[0212] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0213] The above mainly describes part of the method provided by the embodiments of the present application, and the following describes the related device, medium and equipment embodiments provided by the embodiments of the present application.

[0214] In an embodiment, a finger pressing state detection device is provided, which corresponds to the finger pressing state detection method in the above embodiment. As shown in FIG. 13, the finger pressing state detection device includes a processing module 101 and an input module 102. The functions of each functional module are described in detail as follows.

[0215] The processing module 101 is configured to pre-process an ultrasonic echo signal to obtain model input data, the ultrasonic echo signal being an ultrasonic echo signal corresponding to an ultrasonic array emitting an ultrasonic signal to a pressing area;

[0216] The input module 102 is configured to input the model input data into a trained neural network model, so that the neural network model outputs a finger pressing state.

[0217] In combination with the above embodiment, in an embodiment, the neural network model includes a neural network model constructed based on a recurrent neural network; or a neural network model constructed based on a convolutional neural network.

[0218] In combination with the above embodiment, in an embodiment:

[0219] The neural network model includes a first classification decision layer and a multi-layer recurrent neural network, and an output of the multi-layer recurrent neural network is connected to the first classification decision layer;

[0220] Or;

[0221] The neural network model includes a second classification decision layer and a single-layer recurrent neural network, and an output of the single-layer recurrent neural network is connected to the second classification decision layer.

[0222] In an embodiment, the processing module 101 is configured to:

[0223] The single-frame ultrasonic echo signal is spatially multi-point sampled to obtain spatial sampling data corresponding to the single-frame ultrasonic echo signal.

[0224] The spatial sampling data is converted into a first data vector.

[0225] According to the first data vector, model input data of the neural network model is obtained.

[0226] In an embodiment, the processing module 101 is configured to:

[0227] The single-frame ultrasonic echo signal is spatially sampled according to a preset row-column interval to obtain spatial sampling data corresponding to the single-frame ultrasonic echo signal.

[0228] In an embodiment, the processing module 101 is configured to:

[0229] The first data vector is normalized, and the normalized first data vector is the model input data of the neural network model.

[0230] In an embodiment, the processing module 101 is configured to:

[0231] The echo intensity feature value of the single-frame ultrasonic echo signal collected at the current time step and the echo intensity feature values corresponding to other time steps are converted into a second data vector, and the other time steps are time steps before the current time step.

[0232] According to the second data vector, model input data of the neural network model is obtained.

[0233] In an embodiment, the echo intensity feature value includes an echo intensity mean value of the single-frame ultrasonic echo signal, or the echo intensity feature value includes a minimum echo intensity value in the single-frame ultrasonic echo signal.

[0234] In an embodiment, the other time steps are N-1 continuous time steps before the current time step N.

[0235] In an embodiment, the processing module 101 is configured to:

[0236] The second data vector is normalized, and the normalized second data vector is the model input data of the neural network model.

[0237] In an embodiment, the finger pressing state output by the neural network model includes a classification-determined state or a classification-pending state, and the classification-determined state is any pressing state in at least two categories of pressing states.

[0238] In an embodiment: the at least two categories of pressing states include a finger touch state and a finger non-touch state; or; the at least two categories of pressing states include a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, and a non-touch-to-touch critical state.

[0239] In an embodiment, the processing module 101 is further configured to:

[0240] When the output finger pressing state is the classification determination state, the classification determination state is taken as the final finger pressing state, and the output classification determination state is updated to the state memory sequence in time sequence;

[0241] When the output finger pressing state is the classification pending state, the last recorded pressing state in the state memory sequence is taken as the final finger pressing state, or the pressing state with the highest state probability in the state memory sequence is taken as the final output finger pressing state, and is updated to the state memory sequence.

[0242] In summary, the embodiment of the present application provides a finger pressing state detection device based on a neural network model, which brings the following technical effects: without setting a reference threshold comparison method, the differences between different ultrasonic modules can be eliminated, the same network model parameters have better generalization ability between different ultrasonic modules, and the differences in various different scenarios (temperature, frequency, etc.) and different finger states can be effectively eliminated, a variety of special scenarios such as high temperature, low temperature, and wet fingers can be covered, and the device has stronger generalization ability and accuracy; more technical effects can be referred to the corresponding description of the foregoing method embodiments.

[0243] In an embodiment, a neural network model training device is provided, which corresponds to the neural network model training method in the above-mentioned embodiments. As shown in FIG. 14, the neural network model training device includes an acquisition module 201 and a training module 202. The functions of each functional module are described in detail as follows:

[0244] The acquisition module 201 is configured to acquire training sample data, wherein the training sample data includes finger pressing state sample data and corresponding pressing state classification labels, the finger pressing state sample data is constructed based on an ultrasonic echo sample signal, and the ultrasonic echo sample signal is an ultrasonic reflection signal after an ultrasonic wave sample signal is emitted to a pressing area;

[0245] The training module 202 is configured to train a neural network model based on the training sample data until a neural network model satisfying a preset model condition is obtained, and the trained neural network model is used to output a finger pressing state.

[0246] In combination with the previous embodiment, in an embodiment, the neural network model comprises a neural network model constructed based on a recurrent neural network; or a neural network model constructed based on a convolutional neural network.

[0247] In combination with the previous embodiment, in an embodiment, the neural network model comprises a first classification decision layer and a multi-layer recurrent neural network, an output of the multi-layer recurrent neural network being connected to the first classification decision layer.

[0248] Or;

[0249] The neural network model comprises a second classification decision layer and a single-layer recurrent neural network, an output of the single-layer recurrent neural network being connected to the second classification decision layer.

[0250] In an embodiment, the finger pressing state sample data comprises a first sample data vector.

[0251] The first sample data vector is converted from spatial domain sampling sample data, the spatial domain sampling sample data being obtained by spatial domain multi-point sampling based on a single-frame ultrasonic echo sample signal.

[0252] In an embodiment, the spatial domain sampling sample data is obtained by spatial domain sampling of the single-frame ultrasonic echo sample signal at a preset interval.

[0253] In an embodiment, the first sample data vector is a sample data vector after normalization processing.

[0254] In an embodiment, the finger pressing state sample data comprises a second sample data vector.

[0255] The second sample data vector is converted from echo intensity feature values corresponding to multiple single-frame ultrasonic echo sample signals respectively, the multiple single-frame ultrasonic echo sample signals being single-frame ultrasonic echo sample signals corresponding to multiple different time steps respectively.

[0256] In an embodiment, the multiple different time steps are consecutive time steps.

[0257] In an embodiment, the second sample data vector is a sample data vector after normalization processing.

[0258] In an embodiment, the echo intensity feature values comprise an echo intensity mean value of the single-frame ultrasonic echo signal, or the echo intensity feature values comprise a minimum echo intensity value in the single-frame ultrasonic echo signal.

[0259] In an embodiment, the finger pressing state output by the neural network model comprises a classification determined state or a classification pending state, the classification determined state being any pressing state of at least two categories of pressing states.

[0260] In an embodiment, the at least two categories of pressing states include a finger touch state and a finger non-touch state; or the at least two categories of pressing states include a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, and a non-touch-to-touch critical state.

[0261] The specific limitations of the above device can refer to the relevant limitations of the method described above, which will not be repeated here. Each module in the above device can be implemented by software, hardware, and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the electronic device or computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0262] In an embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the neural network model training method of any one of the preceding embodiments when executing the computer program.

[0263] In an embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the finger pressing state detection method of any one of the preceding embodiments when executing the computer program.

[0264] Specifically, in an embodiment, as shown in FIG. 1, an electronic device is provided, including an ultrasonic piezoelectric sensor, an analog-to-digital converter, a data processing center, and a central controller, wherein the ultrasonic piezoelectric sensor array, the analog-to-digital converter, and the data processing center are connected to the central controller;

[0265] The ultrasonic piezoelectric sensor is configured to emit an ultrasonic signal to a pressing area in response to the central controller and receive an ultrasonic echo signal;

[0266] The analog-to-digital converter is configured to perform digital-to-analog conversion on the ultrasonic echo signal in response to the central controller;

[0267] The data processing center is configured to implement the steps of the finger pressing state detection method of the preceding embodiments in response to the central controller.

[0268] In an embodiment, a readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the neural network model training method or the finger pressing state detection method of the preceding embodiments.

[0269] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method for training a neural network model or the method for detecting a finger pressing state as described above.

[0270] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory.

[0271] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is taken as an example for illustration. In actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0272] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features. Such modifications or replacements do not change the essence of the corresponding technical solutions, and should be included in the protection scope of the present application.

Claims

1. A finger pressing state detection method characterized by, The method comprises: preprocessing an ultrasonic echo signal to obtain model input data, the ultrasonic echo signal being an ultrasonic echo signal corresponding to an ultrasonic signal emitted by an ultrasonic array to a pressing region; inputting the model input data into a trained neural network model to enable the neural network model to output a finger pressing state.

2. The finger pressing state detection method according to claim 1, characterized in that: the neural network model comprises a first classification decision layer and a multi-layer recurrent neural network, and an output of the multi-layer recurrent neural network is connected to the first classification decision layer; or; the neural network model comprises a second classification decision layer and a single-layer recurrent neural network, and an output of the single-layer recurrent neural network is connected to the second classification decision layer.

3. The finger pressing state detection method according to claim 1, characterized by, The preprocessing of the ultrasonic echo signal to obtain the model input data comprises: spatially multi-point sampling a single frame of the ultrasonic echo signal to obtain spatial sampling data corresponding to the single frame of the ultrasonic echo signal, converting the spatial sampling data into a first data vector, and obtaining the model input data of the neural network model according to the first data vector; or; converting echo intensity feature values of a single frame of the ultrasonic echo signal collected at a current time step and echo intensity feature values corresponding to other time steps into a second data vector, obtaining the model input data of the neural network model according to the second data vector, and the other time steps being time steps before the current time step.

4. The finger pressing state detection method according to claim 3, characterized by, The spatial sampling of the single frame of the ultrasonic echo signal to obtain the spatial sampling data corresponding to the single frame of the ultrasonic echo signal comprises: spatially sampling the single frame of the ultrasonic echo signal according to a preset row-column interval to obtain the spatial sampling data corresponding to the single frame of the ultrasonic echo signal.

5. The finger pressing state detection method according to claim 3, characterized by, The other time steps are N-1 continuous time steps before the current time step N.

6. The finger pressing state detection method according to claim 3, characterized by, The echo intensity feature values include an echo intensity mean value of the single frame of the ultrasonic echo signal, or the echo intensity feature values include a minimum echo intensity value in the single frame of the ultrasonic echo signal.

7. The finger pressing state detection method according to any one of claims 1 to 6, characterized by, The finger pressing state output by the neural network model includes a classification determined state or a classification pending state, and the classification determined state is any pressing state of at least two categories of pressing states.

8. The finger pressing state detection method according to claim 7, characterized in that: the at least two categories of pressing states include a finger touch state and a finger non-touch state; or; the at least two categories of pressing states include a finger touch state, a finger non-touch state, a critical state from touch to non-touch, and a critical state from non-touch to touch.

9. The finger pressing state detection method according to claim 7, characterized by, After the inputting of the model input data into the trained neural network model to enable the neural network model to output the finger pressing state, the method further comprises: when the output finger pressing state is the classification determined state, taking the classification determined state as a final finger pressing state and updating the output classification determined state to the state memory sequence in time sequence. When the output finger pressing state is the classification pending state, the last recorded pressing state in the state memory sequence is taken as the final finger pressing state, or the pressing state with the highest state probability in the state memory sequence is taken as the final output finger pressing state, and is correspondingly updated to the state memory sequence.

10. A neural network model training method, characterized in that, The method comprises: obtaining training sample data, the training sample data comprising finger pressing state sample data and corresponding pressing state classification labels, the finger pressing state sample data being constructed based on an ultrasonic echo sample signal, the ultrasonic echo sample signal being an ultrasonic echo signal corresponding to an ultrasonic wave sample signal emitted by an ultrasonic array to a pressing area; training a neural network model based on the training sample data until a neural network model satisfying a preset model condition is obtained; the trained neural network model is used to output a finger pressing state.

11. The neural network model training method of claim 10, wherein: the neural network model comprises a first classification decision layer and a multi-layer recurrent neural network, and the output of the multi-layer recurrent neural network is connected to the first classification decision layer; or; the neural network model comprises a second classification decision layer and a single-layer recurrent neural network, and the output of the single-layer recurrent neural network is connected to the second classification decision layer.

12. The neural network model training method of claim 10, wherein, The finger pressing state sample data comprises a first sample data vector, which is obtained by converting spatial domain sampling sample data, and the spatial domain sampling sample data is obtained by spatial domain multi-point sampling based on a single-frame ultrasonic echo sample signal; or; The finger pressing state sample data comprises a second sample data vector, which is obtained by converting echo intensity feature values corresponding to multiple single-frame ultrasonic echo sample signals, and the multiple single-frame ultrasonic echo sample signals are single-frame ultrasonic echo sample signals corresponding to multiple different time steps.

13. The neural network model training method of claim 12, wherein, The spatial domain sampling sample data is obtained by spatial domain sampling a single-frame ultrasonic echo sample signal at a preset interval.

14. The neural network model training method of claim 12, wherein, The multiple different time steps are consecutive time steps.

15. The method of claim 12, wherein, The echo intensity feature values comprise a mean value of echo intensity of a single-frame ultrasonic echo signal, or the echo intensity feature values comprise a minimum echo intensity value in a single-frame ultrasonic echo signal. 16.The method of any one of claims 10-15, wherein, The finger pressing state output by the neural network model comprises a classification determined state or a classification pending state, and the classification determined state is any pressing state of at least two categories of pressing states.

17. The neural network model training method of claim 16, wherein: the at least two categories of pressing states comprise a finger touch state and a finger non-touch state; or; the at least two categories of pressing states comprise a finger touch state, a finger non-touch state, a touch-to-non-touch critical state, and a non-touch-to-touch critical state.

18. An electronic device, comprising: The electronic device comprises an ultrasonic piezoelectric sensor, an analog-to-digital converter, a data processing center, and a central controller, wherein the ultrasonic piezoelectric sensor array, the analog-to-digital converter, and the data processing center are connected to the central controller. The ultrasonic piezoelectric sensor is configured to transmit ultrasonic signals to the pressing area and receive ultrasonic echo signals in response to the central controller; The analog-digital converter is configured to convert the ultrasonic echo signals from analog signals to digital signals in response to the central controller; The data processing center is configured to implement the steps of the finger pressing state detection method according to any one of claims 1-9 in response to the central controller.

19. An electronic device, comprising: A computer program product, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the neural network model training method according to any one of claims 10-17 when executing the computer program.

20. A readable storage medium, the readable storage medium storing a computer program, characterized in that, The computer program is executable on the processor to implement the steps of the method according to any one of claims 1-17.

Images