Press area detection method and apparatus, and electronic device, storage medium and program product

Abstract

Provided in the embodiments of the present application are a press area detection method and apparatus, and an electronic device, a storage medium and a program product. The press area detection method comprises: acquiring press data obtained by a user performing a press operation on a touch-control screen; inputting the press data into a machine learning model to perform center point prediction, so as to obtain predicted center coordinates of a press area; and performing shape prediction on the basis of the press data and the predicted center coordinates, so as to obtain predicted shape parameters of the press area. The present solution achieves a better effect in terms of detecting a press area of a touch-control screen.

Description

Methods, devices, electronic equipment, storage media, and program products for detecting pressing areas. Technical Field

[0001] This application relates to the field of touch technology, and in particular to a method, apparatus, electronic device, storage medium, and program product for detecting a pressed area. Background Technology

[0002] With the rapid popularization and continuous development of smartphones and other mobile devices, touchscreens have become an indispensable part of them, and pressing on the touchscreen to perform touch operations has become one of the core methods of user interaction. The detection of the pressed area is crucial to the user experience of touchscreens; for example, touch control can be achieved by detecting parameters of the pressed area. However, the accuracy of pressing area detection in related technologies is relatively poor. Therefore, how to improve the accuracy of pressing area detection on touchscreens has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of this application provide a pressing area detection method, apparatus, electronic device, storage medium, and program product.

[0004] According to a first aspect of the embodiments of this application, a method for detecting a pressing area is provided, comprising: acquiring pressing data obtained by a user pressing a touch screen; inputting the pressing data into a machine learning model to predict the center point and obtain the predicted center coordinates of the pressing area; and performing shape prediction based on the pressing data and the predicted center coordinates to obtain the predicted shape parameters of the pressing area.

[0005] In some optional embodiments, the step of inputting the pressure data into a machine learning model for center point prediction to obtain the predicted center coordinates of the pressure area includes: inputting the pressure data into a machine learning model for feature extraction to obtain feature information of the pressure data, and performing center point prediction based on the feature information of the pressure data to obtain the predicted center coordinates of the pressure area; the step of performing shape prediction based on the pressure data and the predicted center coordinates to obtain the predicted shape parameters of the pressure area includes: performing shape prediction based on the feature information of the pressure data and the preset center coordinates to obtain the predicted shape parameters of the pressure area.

[0006] In some optional embodiments, the predicted morphological parameters include at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the shape corresponding to the pressing area.

[0007] In some optional embodiments, obtaining the pressing data obtained by the user pressing the touch screen includes: obtaining the original capacitance data obtained by the user pressing the touch screen; expanding the capacitance data corresponding to the pressing area in the original capacitance data to obtain target capacitance data, and determining the target capacitance data as the pressing data.

[0008] In some optional embodiments, the method further includes: displaying a given pressing area on the touch screen to prompt the user to press the given pressing area; and acquiring a second pressing data sample obtained by the user pressing the given pressing area on the touch screen, wherein the second pressing data sample is used to fine-tune the machine learning model to obtain a machine learning model for the user.

[0009] In some optional embodiments, the method further includes controlling the terminal device on which the touchscreen is installed based on the predicted center coordinates of the pressing area and / or the predicted morphological parameters.

[0010] According to a second aspect of the embodiments of this application, a model training method is provided, comprising: acquiring a first training sample for training a machine learning model, wherein the first training sample includes a first pressing data sample obtained by a user pressing a touch screen, a first center point ground value corresponding to the first pressing data sample, and a first morphological parameter ground value corresponding to the first pressing data sample; inputting the first training sample into the machine learning model to perform center point prediction to obtain a first predicted center coordinate sample of the pressing area, and performing morphological prediction based on the first pressing data sample and the first predicted center coordinate sample to obtain a first predicted morphological parameter sample of the pressing area; and training the machine learning model based on the first predicted center coordinate sample, the first center point ground value, the first predicted morphological parameter sample, and the first morphological parameter ground value.

[0011] In some optional embodiments, the step of inputting the first training sample into the machine learning model to predict the center point and obtain the first predicted center coordinate sample of the pressing area includes: inputting the first training sample into the machine learning model, extracting features from the first pressing data sample in the first training sample to obtain feature information of the first pressing data sample, and predicting the center point based on the feature information of the first pressing data sample to obtain the first predicted center coordinate sample of the pressing area; the step of performing shape prediction based on the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted shape parameter sample of the pressing area includes: performing shape prediction based on the feature information of the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted shape parameter sample of the pressing area.

[0012] In some optional embodiments, the machine learning model includes: a feature extraction part, a first perception layer part, and a second perception layer part; the step of extracting features from the first pressing data sample in the first training sample to obtain feature information of the first pressing data sample, and predicting the center point based on the feature information of the first pressing data sample to obtain the first predicted center coordinate sample of the pressing area includes: extracting features from the first pressing data sample in the first training sample through the feature extraction part to obtain feature information of the first pressing data sample, and predicting the center point based on the feature information of the first pressing data sample through the first perception layer part to obtain the first predicted center coordinate sample of the pressing area; the step of predicting the shape based on the feature information of the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted shape parameter sample of the pressing area includes: predicting the shape based on the feature information of the first pressing data sample and the first predicted center coordinate sample through the second perception layer part to obtain the first predicted shape parameter sample of the pressing area.

[0013] In some optional embodiments, the feature extraction part includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer; the step of extracting features from the first pressing data sample in the first training sample through the feature extraction part to obtain the feature information of the first pressing data sample includes: sequentially performing corresponding linear transformation processing, convolution processing, and pooling processing on the first pressing data sample in the first training sample through the linear transformation layer, at least one convolutional layer, and at least one pooling layer of the feature extraction part to obtain the feature information of the first pressing data sample.

[0014] In some optional embodiments, the first predicted morphological parameter sample includes at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample of the shape corresponding to the pressing area.

[0015] In some optional embodiments, the first press data sample obtained by the user pressing the touch screen is obtained by: acquiring an original capacitance data sample, which is a capacitance data sample used to characterize the user pressing the touch screen; expanding the capacitance data corresponding to the pressing area in the original capacitance data sample to obtain a target capacitance data sample, and determining the target capacitance data sample as the first press data sample.

[0016] In some optional embodiments, the method further includes: obtaining a second training sample, wherein the second training sample includes: a second pressing data sample obtained by the user pressing a given pressing area on the touch screen, a second center point ground value corresponding to the second pressing data sample, and a second morphological parameter ground value corresponding to the second pressing data sample; and using the second training sample to fine-tune the machine learning model to obtain a machine learning model for the user.

[0017] In some optional embodiments, the true value of the second center point is the coordinates of the center point of the shape corresponding to the given pressing area, and the true value of the second morphological parameter is the morphological parameter of the shape corresponding to the given pressing area.

[0018] In some optional embodiments, the step of fine-tuning the machine learning model using the second training samples includes: inputting the second training samples into the machine learning model to predict the center point, obtaining a second predicted center coordinate sample of the given pressing area; and performing morphological prediction based on the second pressing data sample and the second predicted center coordinate sample to obtain a second predicted morphological parameter sample of the given pressing area; and fine-tuning the machine learning model based on the second predicted center coordinate sample, the second center point ground truth, the second predicted morphological parameter sample, and the second morphological parameter ground truth.

[0019] According to a third aspect of the embodiments of this application, a pressing area detection device is provided, comprising: an acquisition module for acquiring pressing data obtained by a user pressing a touch screen; a prediction module for inputting the pressing data into a machine learning model to predict the center point and obtain the predicted center coordinates of the pressing area; and, based on the pressing data and the predicted center coordinates, performing morphological prediction to obtain the predicted morphological parameters of the pressing area.

[0020] According to a fourth aspect of the embodiments of this application, a model training apparatus is provided, comprising: a sample acquisition module, configured to acquire a first training sample for training a machine learning model, wherein the first training sample includes a first pressing data sample obtained by a user pressing a touch screen, a first center point ground value corresponding to the first pressing data sample, and a first morphological parameter ground value corresponding to the first pressing data sample; a parameter prediction module, configured to input the first training sample into the machine learning model to perform center point prediction, thereby obtaining a first predicted center coordinate sample of the pressing area, and to perform morphological prediction based on the first pressing data sample and the first predicted center coordinate sample, thereby obtaining a first predicted morphological parameter sample of the pressing area; and a model training module, configured to train the machine learning model based on the first predicted center coordinate sample, the first center point ground value, the first predicted morphological parameter sample, and the first morphological parameter ground value.

[0021] According to a fifth aspect of the present application, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is used to store a computer program; and the processor is used to execute the method described in the first or second aspect by running the computer program stored in the memory.

[0022] According to a sixth aspect of the embodiments of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method described in the first or second aspect.

[0023] According to a seventh aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in the first or second aspect.

[0024] According to the technical solution provided in the embodiments of this application, since the pressing data obtained by the user pressing the touch screen can be acquired, and then the pressing data is input into the machine learning model for center point prediction to obtain the predicted center coordinates of the pressing area, and the shape prediction is performed based on the pressing data and the predicted center coordinates to obtain the predicted shape parameters of the pressing area, on the one hand, the center point coordinates and shape parameters of the user pressing the touch screen can be effectively predicted through the machine learning model, thereby effectively realizing the pressing area detection; and on the other hand, since the accurate center point prediction is first performed through the machine learning model to obtain a relatively accurate predicted center coordinate, and then the shape prediction is performed based on the pressing data and the accurate predicted center point coordinates using the machine learning model, the accuracy of the shape parameter prediction of the pressing area can be effectively improved, thus effectively ensuring the accuracy of the pressing area detection results. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0026] Figure 1A is a flowchart of a technical solution for detecting the pressing area of ​​a touch screen in the related technology.

[0027] Figure 1B is a schematic diagram of the elliptical equipotential lines of the pressing area calculated based on pressing data in related technologies.

[0028] Figure 1C is a schematic diagram of an example capacitive sensing circuit network for a touch screen.

[0029] Figure 2 is a flowchart of a pressing area detection method according to an embodiment of this application.

[0030] Figure 3 is a flowchart of some optional steps in step S102 of an embodiment of this application.

[0031] Figure 4 shows a schematic diagram of obtaining press data from raw capacitance data.

[0032] Figure 5 shows a flowchart of an example of a press area detection using a machine learning model.

[0033] Figure 6 is a flowchart of an example application of the pressure detection scheme according to an embodiment of this application.

[0034] Figure 7 is a flowchart of a model training method according to an embodiment of this application.

[0035] Figure 8 is a flowchart of an optional method for obtaining the first press data sample according to an embodiment of this application.

[0036] Figure 9 is a schematic diagram of the prompt area of ​​an example embodiment of this application.

[0037] Figure 10 shows an example workflow diagram for training a machine learning model.

[0038] Figure 11 is a flowchart of an optional method for fine-tuning a machine learning model using a second training sample.

[0039] Figure 12 is a schematic diagram of a pressing area detection device according to an embodiment of this application.

[0040] Figure 13 is a schematic diagram of a model training device according to an embodiment of this application.

[0041] Figure 14 is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0042] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0043] With the rapid popularization and continuous development of smartphones and other mobile devices, touchscreens have become an indispensable part of them, and pressing on the touchscreen to perform touch operations has become one of the core ways for users to interact. Detecting the pressing area is quite important for the user experience of touchscreens; for example, touch control can be achieved by detecting the parameters of the pressing area.

[0044] The accuracy of detecting touchscreen pressing areas in related technologies is relatively poor. For example, when a user presses the touchscreen using different fingers (such as thumb or index finger) and different parts of the hand (such as fingertip, finger pad, or side of the finger), at different angles, each different pressing method may result in different pressing area shapes on the touchscreen. For instance, when a user presses with their fingertip, the corresponding pressing data usually approximates a circle (it should be understood that a circle can be considered a special type of ellipse), while when the user presses with their finger pad or side of the finger, the pressing data may approximate various elliptical shapes. Therefore, the pressing area can usually be treated as an elliptical region to calculate various parameters of the ellipse. However, related technologies use fixed algorithms to calculate the parameters of the pressing area, while the actual pressing data of users has such varied shapes that the pressing area parameters calculated by the fixed algorithm will deviate. Even if a user tries to press the same position on the touchscreen, the calculated pressing area parameters (for an ellipse, this may include, but is not limited to, the coordinates of the ellipse's center) may deviate from the expected values. Therefore, the pressing area detection effect of the touchscreen is poor, affecting the user experience. In addition, different users have individual differences. Different users have different finger shapes (including differences in length, width, and thickness), and their usage habits such as pressing on the touchscreen are also different. This means that even when pressing the same position, the calculated pressing area parameters under the fixed algorithm will not match the expectations due to individual differences between users, thus further improving the pressing area detection effect of the touchscreen and further affecting the user experience. The reduced detection performance of the press area not only affects the accuracy of touch operation, but also leads to a decrease in user experience and satisfaction (for example, some examples of user experience problems include users frequently making mistakes when pressing to input text on the touch screen and having to re-enter the text, etc.).

[0045] For example, Figure 1A shows a flowchart of a technical solution for detecting the pressing area of ​​a touch screen in the related art. As shown in Figure 1A, taking the case where the pressing area of ​​the touch screen is regarded as an elliptical area and the touch screen is a capacitive touch screen as an example, the technical solution in this related art can first obtain the pressing data (capacitance data, such as mutual capacitance data) at a certain moment to determine whether there is a pressing event on the touch screen. If so, the pressing data corresponding to the pressing area is determined from the above pressing data, and the elliptical equipotential line is calculated. Then, based on the calculation result of the elliptical equipotential line, the parameters of the pressing area (such as including but not limited to the coordinates of the center point of the elliptical area) are calculated and output.

[0046] Figure 1B shows a schematic diagram of calculating the elliptical equipotential line of the pressing area based on the pressing data. The elliptical equipotential line can be considered as the fitted elliptical contour line, which is roughly determined by a set of equipotential points. The search result of the equipotential point set is mainly determined by the equipotential line threshold. When calculating the elliptical equipotential line, the equipotential point set is usually calculated according to the principle of linear interpolation (for example, assuming the equipotential line threshold = 75, then through linear interpolation, the "×" in Figure 1B represents the equipotential points, which constitute the equipotential point set, and this set approximates the elliptical curve (shown by the dashed line in Figure 1B). In the figure, CD is the major axis of the ellipse, and AB is the minor axis of the ellipse).

[0047] However, on the one hand, the equipotential line threshold in the above-mentioned schemes is difficult to determine. The equipotential line threshold is usually proportional to the peak value within the current pressing area, and this fixed proportion needs to be preset by engineers based on debugging datasets and personal experience. However, debugging datasets may be insufficient, and personal experience is often subjective, which may lead to a suboptimal choice of the equipotential line threshold, thus affecting the performance of ellipse recognition and reducing the accuracy of pressing area detection in the aforementioned related technologies. On the other hand, linear interpolation also introduces errors when calculating the elliptical equipotential lines in the above-mentioned schemes. As shown in Figure 1C, the touch sensing unit of the touch screen can include multiple drivers (TX) and receivers (RX). The capacitance detection circuit between TX and RX can be approximately simplified as a "grid". Under normal circumstances, there are approximately equal spacings between TX and RX. (As shown in Figure 1C, the multiple drivers TX are TX0 to TXend, the multiple receivers RX are RX0 to RXend, each driver TX is connected to a drive signal line (also called a drive electrode), and each receiver RX is connected to a receive signal line (also called a receive electrode). The multiple drive signal lines and multiple receive signal lines form a capacitance detection circuit network (mutual capacitance). Taking a total of 5 drive signal lines and 5 receive signal lines in the figure as an example (for ease of understanding), the changes in mutual capacitance data C11 to C55 (for example only) caused by touch in each area of ​​the touch screen 100 can be detected and collected.) When an actual press event occurs, it may occur at the boundary between TX and RX, or it may occur between the gaps. Due to the uneven electric field distribution of the touch sensor layer, the press capacitance data (assuming it is Touch) at the gap (as shown at position a in Figure 1C, indicated by "×") will vary. mid The data is not necessarily equal to the capacitive touch data at the four edges (b1, b2, b3, and b4 in Figure 1C, all marked with "×") (assuming it is Touch). i The weighted summation of (i ∈ {1, 2, 3, 4}) (i.e. In other words, the capacitive data at a certain location is not linearly related to the surrounding data. Because of this non-linear relationship, the equipotential point set obtained using linear interpolation in the above scheme may not be accurate. Therefore, the technical solutions described above cannot guarantee the accuracy of the pressing area detection. It should be noted that the technical solutions shown in Figures 1A and 1B are not necessarily existing technologies.

[0048] It should be understood that this application provides a new pressing area detection scheme, which can realize the detection of the user's pressing area and the determination of related parameters through a machine learning model, and can at least improve the accuracy of pressing area detection of the touch screen.

[0049] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0050] Figure 2 is a flowchart of a pressure area detection method according to an embodiment of the present application. According to a first aspect of the present application, a pressure area detection method is provided. Referring to Figure 2, the method includes steps S102, S104, and S106, specifically:

[0051] S102: Obtain the press data obtained by the user pressing the touch screen.

[0052] It should be noted that the press area detection method in this embodiment can be applied to the press area detection of any touch screen. For example, the touch screen can be a capacitive touch screen, a resistive touch screen, etc. For ease of explanation, a capacitive touch screen will be used as an example below.

[0053] Optionally, the pressing area detection method in this application embodiment can be used in the use stage of the touch screen, that is, in the stage after the touch screen has been manufactured and installed on the terminal device.

[0054] In this embodiment, the type of press data can be determined according to the type of touchscreen. For example, taking a capacitive touchscreen as an example, the press data can be capacitance data. Here, capacitance data can be mutual capacitance data or self-capacitance data, depending on whether the touchscreen is based on the mutual capacitance principle or the self-capacitance principle. Taking mutual capacitance data as an example, mutual capacitance data can be collected at various locations on the touchscreen (this can be understood in conjunction with the example in Figure 1C, where the dashed box roughly shows the shape of the touchscreen; this is just an example). When the user does not touch the touchscreen, the fluctuation of the mutual capacitance data collected at various locations on the touchscreen is very small. However, when the user presses a portion of the touchscreen to achieve touch control, the mutual capacitance data collected through that portion changes, causing a significant increase in the difference between the mutual capacitance data and the value when no pressure is applied. Therefore, by detecting the change in the difference in mutual capacitance data (e.g., the difference from a reference value (e.g., 0)), it can be determined whether the touchscreen is being pressed or touched, thus facilitating accurate detection of the press area.

[0055] In some optional embodiments, referring to the flowchart shown in FIG3, step S102 may include steps S1022 and S1024, specifically:

[0056] S1022: Obtain raw capacitance data obtained from the user's pressing operation on the touch screen.

[0057] When collecting training samples, the user presses on the touchscreen, and the touchscreen can detect raw capacitance data based on the press operation. The raw capacitance data can include at least the capacitance data corresponding to the pressed area.

[0058] S1024: Extend the capacitance data corresponding to the pressing area in the original capacitance data to obtain the target capacitance data, and determine the target capacitance data as the pressing data.

[0059] Optionally, a minimum bounding box containing the capacitance data corresponding to the pressed area can be determined from the original capacitance data. This minimum bounding box can then be expanded to extend the capacitance data corresponding to the pressed area in the original capacitance data. The capacitance data within the expanded bounding box is then extracted to obtain the target capacitance data, which is then designated as the pressed data.

[0060] It should be understood that the above-mentioned expansion processing can be achieved by expanding the minimum rectangle outwards from at least one of its top, bottom, left, or right sides. For example, if one side of the minimum rectangle coincides with a corresponding position on the edge of the touchscreen, the other three sides can be expanded outwards to achieve expansion processing; if two sides of the minimum rectangle coincide with corresponding positions on the edge of the touchscreen, the other two sides can be expanded outwards to achieve expansion processing; if the minimum rectangle is located in the middle, all four sides can be expanded outwards to achieve expansion processing. The outward expansion distance of each side can be set as needed, and the outward expansion distances of each side can be the same or different. No restrictions are imposed as long as the actual needs are met.

[0061] For example, Figure 4 shows a schematic diagram of obtaining press data from raw capacitance data. In Figure 4, the P box is the smallest rectangle containing the capacitance data corresponding to the press area, while the Q box is the rectangle obtained by expanding the four sides (p1, p2, p3, p4) of the smallest rectangle (P box) outward. Since the capacitance data in the Q box is more than that in the P box, it can effectively avoid missing the capacitance data corresponding to the press area. Using the target capacitance data obtained by this expansion process as press data can improve the accuracy of subsequent press area detection based on the press data.

[0062] Based on this, the optional implementation of steps S1022 to S1024 in this application embodiment can, on the one hand, effectively avoid missing the capacitance data corresponding to the pressing area. Using the target capacitance data obtained through such extended processing as pressing data can improve the accuracy of subsequent pressing area detection based on pressing data. On the other hand, since the target capacitance data obtained through extended processing is used as pressing data and processed by the machine learning model, instead of inputting all the original capacitance data of the touch screen into the machine learning model for processing, the computational load can be effectively reduced, thereby improving the pressing detection efficiency and reducing the interference of too much useless data, further improving the detection accuracy.

[0063] S104: Input the compression data into a machine learning model to predict the center point, obtaining the predicted center coordinates of the compression area; and...

[0064] S106: Based on the pressing data and the predicted center coordinates, shape prediction is performed to obtain the predicted shape parameters of the pressing area.

[0065] It should be noted that step S106, like step S104, is also implemented through a machine learning model. That is, the machine learning model performs shape prediction based on the compression data and the predicted center coordinates to obtain the predicted shape parameters of the compression area. This machine learning model can be considered a "two-stage prediction model," which can separately predict the predicted center coordinates and the predicted shape parameters of the compression area.

[0066] Based on this, the pressing area detection scheme in this embodiment, through the above steps S102 to S106, can obtain pressing data obtained by the user pressing the touch screen, and then input the pressing data into a machine learning model to predict the center point, thereby obtaining the predicted center coordinates of the pressing area. Furthermore, based on the pressing data and the predicted center coordinates, shape prediction is performed to obtain the predicted shape parameters of the pressing area. Therefore, on the one hand, the machine learning model can effectively predict the center point coordinates and shape parameters of the user's pressing area on the touch screen, thus effectively achieving pressing area detection. On the other hand, since the machine learning model first performs accurate center point prediction to obtain relatively accurate predicted center coordinates, and then uses the machine learning model to perform shape prediction based on the pressing data and accurate predicted center point coordinates, the accuracy of the shape parameter prediction of the pressing area can be effectively improved, thus effectively ensuring the accuracy of the pressing area detection results.

[0067] Furthermore, the use of machine learning models not only makes it more convenient to use, but also improves development and debugging efficiency, facilitates maintenance and optimization, enhances the versatility of the solution, and helps reduce the extra time and effort that R&D personnel need to invest before and after development is completed.

[0068] It should be understood that in the technical solution of this application embodiment, predicting the center point coordinates first is crucial for the detection of the pressing area. This is because predicting the center coordinates first, and then performing shape prediction based on the pressing data and the predicted center coordinates to obtain the predicted shape parameters, helps to reduce the negative impact of the nonlinearity of the pressing data on the prediction parameters (which can be understood in conjunction with the relevant content of Figure 1C above). This can effectively improve the accuracy of the prediction of the shape parameters of the pressing area and improve the accuracy of the pressing area detection.

[0069] The machine learning model in this embodiment can be a pre-trained machine learning model. The machine learning model is implemented based on artificial intelligence technology. This machine learning model can be of any type. For example, it can be a convolutional neural network model (including but not limited to AlexNet, Inception, VGG, ResNet, MobileNet, FCN, U-Net, DeepLab, BiSeNet, R-CNN, YOLO, SSD, RetinaNet, etc.), a deep language model (including but not limited to LSTM, GRU, BiLSTM, Transformer, MarianMT, BERT, RoBERTa, XLNet, GPT, T5, S4, Mamba, etc.), or a hybrid variant model based on the above two (including but not limited to ViT, DeiT, ConViT, Swing Transformer, TransUNet, Swing Unet, DETR, Deformable DETR, ConvS2S, ConvBERT, etc.). This embodiment does not impose any limitations on the training process of the machine learning model. For example, it can be trained using the model training method described in the second aspect below. The model training part will be explained in detail below and will not be repeated here.

[0070] Optionally, when the pressing area is taken as a circular area, the predicted center coordinates can be the center coordinates of the shape corresponding to the pressing area, and the predicted shape parameters can be the circle diameter parameter, radius parameter, etc. It can be understood that a circle can also be considered as a special type of ellipse with equal major and minor axes.

[0071] When treating the pressing area as an elliptical region, in some optional embodiments, the predicted center coordinates can be the center coordinates of an ellipse corresponding to the shape of the pressing area. The center of the ellipse is the intersection of the major and minor axes of the ellipse.

[0072] It should be understood that by determining the above predicted center coordinates, the center coordinates of the pressing area can be accurately detected, so as to improve the user experience of the touch screen by utilizing the center coordinates.

[0073] In some optional embodiments, the predicted morphological parameters include at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the shape corresponding to the pressing area.

[0074] It should be understood that by determining at least one of the above predicted morphological parameters, it is easier to accurately detect the morphological parameters of the pressing area, so as to improve the user experience of the touch screen by utilizing these morphological parameters.

[0075] Optionally, when treating the pressing area as an elliptical region, the predicted morphological parameters may include at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the ellipse.

[0076] The aforementioned major axis tilt angle parameter can be the angle between the major axis and the vertical or horizontal axis of the coordinate system. Generally speaking, the horizontal and vertical axes of the coordinate system correspond to the length or width direction of the touch screen, or the width or length direction of the touch screen, respectively. Therefore, the major axis tilt angle parameter can also be considered as the angle between the major axis and the length or width direction of the touch screen.

[0077] The minor axis tilt angle parameter mentioned above can be the angle between the minor axis and the vertical or horizontal axis of the coordinate system. Similarly, the minor axis tilt angle parameter can also be considered as the angle between the minor axis and the length or width direction of the touch screen.

[0078] For example, if the vertical axis of the coordinate system corresponds to the length direction of the touch screen, then the major axis tilt angle parameter can be determined as the angle between the major axis and the vertical axis of the coordinate system (or it can be considered as the angle between the major axis and the length direction of the touch screen); the minor axis tilt angle parameter can be determined as the angle between the minor axis and the vertical axis of the coordinate system (or it can be considered as the angle between the minor axis and the length direction of the touch screen).

[0079] In some optional embodiments, the pressing area detection method of this application further includes: controlling the terminal device with a touch screen installed based on the predicted center coordinates and / or predicted morphological parameters of the pressing area.

[0080] Therefore, after completing the pressing area detection and obtaining accurate predicted center coordinates and predicted shape parameters in this embodiment of the application, the predicted center coordinates and predicted shape parameters can be utilized, for example, to control the terminal device, so as to improve the user's experience of using the touch screen and the terminal device.

[0081] It should be understood that the control methods for the terminal device with the touch screen installed based on the predicted center coordinates and / or predicted shape parameters of the pressing area in the embodiments of this application can be diverse, and appropriate control can be achieved as needed. For example, it may include, but is not limited to: display control of the touch screen of the terminal device (for example, in some examples of display control, the shape of a certain interface element displayed on the touch screen can be adjusted, scaled, rotated, etc., the touch screen can be controlled to display different light and shadow effects, the display brightness of the touch screen can be controlled, the up and down swiping of the touch screen interface can be controlled, etc.), and control of the volume of the terminal device.

[0082] For example, the following are some possible application methods:

[0083] Some optional applications allow the results of pressure area detection to be combined with fingerprint unlocking to achieve dynamic feedback. For example, in some under-display fingerprint unlocking systems, a fixed fingerprint enrollment prompt area (such as a circle or other shape) is typically displayed on the screen (touchscreen). Using the aforementioned predicted center coordinates and predicted shape parameters, a more dynamic and visually appealing unlocking experience can be achieved. For instance, the shape of the fingerprint enrollment prompt area can be adjusted using the predicted center coordinates and predicted shape parameters, allowing the area to deform as the finger presses to match the actual press shape. This enables display control during touchscreen fingerprint unlocking, not only making the visuals more intuitive for user operation but also enhancing the overall appeal of the product. Furthermore, in some implementations, during fingerprint unlocking, the touchscreen can be controlled to display corresponding light and shadow effects based on the predicted center coordinates and / or predicted shape parameters, making the unlocking effect more visually appealing and improving the user experience. (For example, taking the predicted shape parameter as the major axis tilt parameter, when the major axis tilt parameter is in the range aa1, it corresponds to the light and shadow effect bb1; when the major axis tilt parameter is in the range aa2, it corresponds to the light and shadow effect bb2, and so on. For another example, taking the predicted center point coordinates as an example, when the predicted center point coordinates are in the coordinate range cc1, it corresponds to...) The lighting effect dd1 corresponds to lighting effect dd2 when the predicted center point coordinates are within the coordinate range cc2, and so on. If a combination of multiple parameters (e.g., the major axis tilt angle parameter and preset center point coordinates) are used to correspond to lighting effects, then when the major axis tilt angle parameter is within the range aa1 and the predicted center point coordinates are within the coordinate range cc1, it corresponds to lighting effect ee1; when the major axis tilt angle parameter is within the range aa2 and the predicted center point coordinates are within the coordinate range cc2, it corresponds to lighting effect ee2, and so on. Of course, the above are merely examples and do not constitute any limitation on the embodiments of this application.

[0084] Other optional applications can combine the results of pressure area detection for posture estimation. For example, by predicting the center coordinates and shape parameters, the specific pressure point and posture of the finger can be further inferred. This means that touchscreens and terminal devices (such as mobile phones) may be able to support richer gesture operations, thereby further optimizing the human-computer interaction experience. For example, some implementation methods may include: controlling the shape adjustment, scaling, rotation, etc. of a certain interface element displayed on the touchscreen; controlling the display brightness of the touchscreen on the terminal device; controlling the volume of the terminal device; controlling the up and down swiping of the touchscreen interface, etc. Of course, the above are only examples and are not intended to limit the embodiments of this application.

[0085] Of course, there are many other applications, but I won't go into detail here.

[0086] In some optional embodiments, step S104 may include: inputting the pressing data into a machine learning model for feature extraction to obtain feature information of the pressing data, and performing center point prediction based on the feature information of the pressing data to obtain the predicted center coordinates of the pressing area; step S106 may include: performing shape prediction based on the feature information of the pressing data and the preset center coordinates to obtain the predicted shape parameters of the pressing area.

[0087] Based on this, in this embodiment, the compression data is input into a machine learning model for feature extraction to obtain the feature information of the compression data. This allows the machine learning model to accurately predict the center point based on the feature information of the compression data, effectively obtaining accurate predicted center coordinates. Furthermore, the machine learning model can accurately predict the shape based on the feature information of the compression data and the accurate predicted center coordinates, effectively obtaining accurate predicted shape parameters. In addition, since the machine learning model first accurately predicts the center point based on the feature information of the compression data to obtain relatively accurate predicted center coordinates, and then uses the machine learning model to predict the shape based on the feature information of the compression data and the accurate predicted center point coordinates, the accuracy of the shape parameter prediction for the compression area can be effectively improved, thus ensuring the accuracy of the results obtained from the compression area detection.

[0088] Optionally, the above-mentioned optional implementation methods can be implemented based on different parts included in the machine learning model. It should be understood that the specific composition structure of the machine learning model is not limited in any way in the embodiments of this application, as long as it can meet the needs.

[0089] Optionally, as shown in Figure 5, the machine learning model may include: a feature extraction part, a first perception layer part, and a second perception layer part. The feature extraction part can be used to predict the center point of the pressure data to obtain the feature information of the pressure area. This feature information is then used in the first perception layer part to predict the center point of the pressure area, resulting in the predicted center coordinates of the pressure area. The feature information and predicted center coordinates of the pressure data can be used in the second perception layer part to predict the shape, obtaining the predicted shape parameters of the pressure area.

[0090] It should be understood that with such a model structure, feature extraction can be accurately achieved through the feature extraction part, and center point prediction and shape prediction can be accurately performed through the first perception layer and the second perception layer, thereby ensuring the accuracy of the results obtained from the pressing area detection.

[0091] The feature extraction part in this application embodiment can be composed of any structure, as long as it meets the requirements. For example, in some optional embodiments, referring to FIG5, the feature extraction part includes a linear transformation layer, at least one convolutional layer and at least one pooling layer; the pressing data can be sequentially processed by the linear transformation layer, at least one convolutional layer and at least one pooling layer of the feature extraction part to obtain the feature information of the pressing data.

[0092] It should be understood that the feature extraction part, which includes the above optional structure, enables the machine learning model to accurately and effectively extract feature information from the pressure data, so as to realize the detection of the pressure area.

[0093] In this embodiment, the linear transformation layer can perform a certain linear transformation on the press data, which can slightly rotate and / or translate the press data. At least one convolutional layer and at least one pooling layer perform convolution and pooling processing respectively, which can extract high-level features of the press data (which can be considered as a sampled representation of the press data).

[0094] Optionally, the feature extraction part of the machine learning model may include multiple convolutional layers and multiple pooling layers to better achieve feature extraction. It should be understood that the specific number can be set according to actual needs and is not limited here.

[0095] Optionally, the first perception layer may include a multilayer perceptron. Optionally, the second perception layer may include a multilayer perceptron. Optionally, the perceptron may be a universal function approximator, which can be used to summarize the features of the first half of the model and output the prediction result. For example, in the detection of the pressing area, the multilayer perceptron in the first perception layer may output the prediction result of the center point (i.e., the predicted center coordinates), and the multilayer perceptron in the first perception layer may output the prediction result of the shape (i.e., the predicted shape parameters, including at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the shape corresponding to the pressing area).

[0096] For example, referring to the workflow of the pressing area detection shown in Figure 5, the following steps are taken: First, the original capacitance data is obtained. Then, the capacitance data corresponding to the pressing area is determined from the original capacitance data. Next, the capacitance data corresponding to the pressing area is expanded to obtain the target capacitance data. The target capacitance data is determined as the pressing data. The pressing data is input into the machine learning model. Through the feature extraction part, which includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer, linear transformation, convolution, and pooling processes are performed sequentially to achieve feature extraction and obtain the feature information of the pressing data. The first perception layer of the machine learning model predicts the center point of the pressing area using the feature information of the pressing data to obtain the predicted center coordinates of the pressing area. The second perception layer of the machine learning model predicts the shape of the pressing area using the feature information of the pressing data and the predicted center coordinates to obtain the predicted shape parameters of the pressing area, which may include at least one of the major axis parameters, minor axis parameters, major axis tilt angle parameters, and minor axis tilt angle parameters.

[0097] Optionally, as shown in the flowchart of the example application of the pressure detection scheme in Figure 6, when actually using this scheme, the original capacitance data at a certain moment can be obtained first. After confirming that there is a pressure event, the pressure data can be determined from the original capacitance data. Then, a machine learning model can be used to predict the center point and shape of the pressure area to obtain the predicted center coordinates and predicted shape parameters of the pressure area (such as an elliptical area), thereby achieving accurate detection of the pressure area.

[0098] It should be understood that Figures 5 and 6 above are not intended to limit the embodiments of this application.

[0099] In some optional embodiments, the pressing area detection method in this application further includes: displaying a given pressing area on a touch screen to prompt the user to perform a pressing operation in the given pressing area; obtaining a second pressing data sample obtained by the user performing a pressing operation in the given pressing area on the touch screen, wherein the second pressing data sample is used to fine-tune the training of the machine learning model to obtain a machine learning model for the user.

[0100] Optionally, the second press data sample can be a capacitance data sample. This capacitance data sample can be either mutual capacitance data or self-capacitance data, depending on whether the touchscreen is based on the mutual capacitance principle or the self-capacitance principle.

[0101] Optionally, the press area detection method can be used during the touchscreen usage phase. At this time, the machine learning model is in a trained state and has a certain degree of universality and generalization. However, due to differences in finger shapes among different users, in practice, to adapt to the user's finger shape and further improve the user experience of the touchscreen user, this embodiment can implement "registration" for specific users. By obtaining the second press data sample obtained by the user pressing a given press area on the touchscreen, it is convenient to use it to fine-tune the machine learning model and obtain a machine learning model for that user. This allows the fine-tuned machine learning model to better achieve accurate press area detection for that user.

[0102] The second press data sample, the given press area, and the fine-tuning training are described in detail in the model training method embodiments below, and can be understood by referring to the embodiments below, and will not be repeated here.

[0103] Optionally, the second press data sample can be obtained in the following way: obtaining a second original capacitance data sample, which is a capacitance data sample used to characterize the user's press operation in a given press area of ​​the touch screen; expanding the capacitance data corresponding to the press area in the second original capacitance data sample to obtain a second target capacitance data sample, and determining the second target capacitance data sample as the second press data sample (this part can be understood by referring to the content in the model training method embodiment below).

[0104] For example, a finger registration system can be added to terminal devices with touchscreens (such as mobile phones). Users of this device can register through a registration interface displayed on the touchscreen. The registration interface can prompt users to press a given pressing area (e.g., an elliptical shape, which can also be understood as the "prompt area" described below) displayed on the touchscreen. Multiple different shapes of the given pressing area can be used during registration. Users can select a given pressing technique (e.g., the pressing techniques in Table 1 below) to completely cover the given pressing area, so as to obtain a second original capacitance data sample for subsequent processing to obtain the second pressing data sample for model fine-tuning training. It should be understood that this part is described in detail in the model training method embodiments below, and can be referred to below for understanding. It will not be repeated here.

[0105] It is understood that the foregoing description of the pressing area detection scheme is merely an exemplary description of the embodiments of this application and is not intended to limit the embodiments of this application in any way.

[0106] In summary, the pressure area detection scheme provided in this application can acquire pressure data obtained from the user's pressing operation on the touch screen, input the pressure data into a machine learning model for center point prediction to obtain the predicted center coordinates of the pressure area, and perform shape prediction based on the pressure data and predicted center coordinates to obtain the predicted shape parameters of the pressure area. Therefore, on the one hand, the machine learning model can effectively predict the center point coordinates and shape parameters of the user's pressing area on the touch screen, thereby effectively realizing pressure area detection; and on the other hand, since the machine learning model first performs accurate center point prediction to obtain relatively accurate predicted center coordinates, and then uses the machine learning model to perform shape prediction based on the pressure data and accurate predicted center point coordinates, the accuracy of the shape parameter prediction of the pressure area can be effectively improved, thus effectively ensuring the accuracy of the results obtained from pressure area detection.

[0107] According to a second aspect of the embodiments of this application, a model training method is provided. Referring to the flowchart shown in FIG7, the model training method includes the following steps S202, S204 and S206, specifically:

[0108] S202: Obtain a first training sample for training the machine learning model, wherein the first training sample includes a first press data sample obtained by the user pressing the touch screen, a first center point ground value corresponding to the first press data sample, and a first morphological parameter ground value corresponding to the first press data sample.

[0109] Optionally, if the touchscreen is a capacitive touchscreen, then the first press data sample in the first training sample can be a capacitance data sample. Here, the capacitance data sample can be mutual capacitance data or self-capacitance data, depending on whether the touchscreen is based on the mutual capacitance principle or the self-capacitance principle. Optionally, the ground truth value of the first center point corresponding to the first press data sample can include labeled data annotating the center point of the first press data sample, or it can be in other forms. Optionally, the ground truth value of the first morphological parameter corresponding to the first press data sample can include labeled data annotating the morphological parameters of the first press data sample (e.g., including at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter), or it can be in other forms.

[0110] In this embodiment of the application, a machine learning model is trained by acquiring at least a first press data sample, the ground truth value of the first center point corresponding to the first press data sample, and the ground truth value of the first morphological parameter. This enables the trained machine learning model to accurately predict the center coordinates and morphological parameters of the press area according to the user's actual press data, thereby realizing press area detection and improving the user's experience of using the touch screen.

[0111] During model training, the machine learning model can be trained multiple times using multiple initial training samples to achieve better training results.

[0112] In some optional embodiments, referring to the flowchart shown in FIG8, the first press data sample can be obtained by the following steps S2022 to S2024, specifically:

[0113] S2022: Obtain raw capacitance data samples. Raw capacitance data samples are used to characterize the capacitance data obtained when the user performs a pressing operation on the touch screen.

[0114] During model training, raw capacitance data samples obtained from different users pressing the touchscreen with different pressing techniques can be acquired to determine the first training sample.

[0115] Optionally, when collecting data, try to cover users from different groups (e.g., men, women, the elderly, and children). Data can also be manually reviewed to filter out obviously erroneous data (e.g., a child's thumb is small and may not fully cover the larger elliptical area of ​​the prompt, so it needs to be removed).

[0116] For example, in one optional implementation, a prompt area can be displayed on the touchscreen of a terminal device (such as a mobile phone). This prompt area can be an elliptical region, used to prompt the user to press the elliptical region at different positions, with different major axes, different minor axes, different major axis tilt angles, and different minor axis tilt angles (at least one of these). After displaying the prompt area, the user selects a pressing method according to the prompt and presses the area completely, then the collected capacitance data is used as the raw capacitance data sample for subsequent processing to obtain the first pressing data sample for model training. Since the relevant parameters of the prompt area displayed on the touchscreen are all known, they can be used as truth values. That is, if the prompt area is an elliptical region, the known center point coordinates of the prompt area can be used as the first center point truth value, and the known morphological parameters of the prompt area (e.g., at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter) can be determined as the first morphological parameter truth values.

[0117] For example, some sample pressing techniques can be shown in Table 1 below:

[0118] Table 1

[0119] As shown in Table 1 above, the available pressing techniques can include thumb tip pressing, thumb pad pressing, thumb side pressing, index finger tip pressing, index finger pad pressing, index finger side pressing, etc. There are other finger or other types of finger pressing techniques, which will not be elaborated here. You can set them as needed.

[0120] For example, taking an elliptical area as the prompt area displayed on the touchscreen, and referring to the example prompt area shown in Figure 9, its parameters can be set as follows:

[0121] ① As shown in Figure 9, the center coordinates (x, y) of the prompt area should be set according to the following formula: (x, y) = (Tx i +a×△Tx,Rx i +b×△Rx)

[0122] Where 'a' is in the range [0,1), for example, it can be selected from the preset values ​​{0.00,0.25,0.50,0.75}; and 'b' is in the range [0,1), for example, it can be selected from the preset values ​​{0.00,0.25,0.50,0.75}.

[0123] The above △Tx can be the unit interval of the horizontal axis, and △Rx can be the unit interval of the vertical axis. Both should be set smaller as needed. These two intervals can be understood as follows: For example, suppose a certain driver TX i Signal lines, receiver RX i The coordinates of the intersection of the signal lines are (Tx i Rx i Adjacent driver TX i+1 Receiver RX i+1 The coordinates of the signal line intersection are (Tx) i+1 Rx i+1 If ), then △Tx=Tx i+1 -Tx i , △Rx=Rx i+1 -Rx i .

[0124] ②As shown in Figure 9, the major axis parameter CD in the prompt area can be set arbitrarily according to actual needs. For example, it can be selected from the preset values ​​{12mm, 14mm, 16mm, 18mm, 20mm}.

[0125] ③ As shown in Figure 9, the major axis parameter AB of the prompt area can be set arbitrarily according to actual needs. For example, it can be selected from the preset values ​​{10mm, 12mm, 14mm, 16mm}.

[0126] ④ As shown in Figure 9, the major axis tilt angle parameter θ of the prompt area is used in this example. Here, θ is the angle between the major axis and the vertical axis of the coordinate system (for example, it can correspond to the length direction of the terminal device (such as a mobile phone)). This θ can be set arbitrarily according to actual needs. For example, it can be selected from the preset values ​​{-90°,-75°,-60°,-45°,-30°,-15°,0°,15°,30°,45°,60°,75°,90°}. A negative angle can be considered to correspond to the case where the slope of the major axis is less than 0 (as shown in the example θ in Figure 9, it is a negative value, and the slope of the major axis is less than 0), and a positive angle can be considered to correspond to the case where the slope of the major axis is greater than 0.

[0127] Regarding the minor axis inclination parameter, it can be deduced by referring to the explanation of the major axis inclination parameter θ. In some examples, the minor axis inclination parameter may not be used, which will not be explained further here.

[0128] It should be understood that the above description is merely an example and is not intended to limit the embodiments of this application.

[0129] S2024: Expand the capacitance data corresponding to the pressing area in the original capacitance data sample to obtain the target capacitance data sample, and determine the target capacitance data sample as the first pressing data sample.

[0130] Optionally, a minimum bounding box containing the capacitance data corresponding to the pressed area can be determined in the original capacitance data sample. This minimum bounding box can then be expanded to extend the capacitance data corresponding to the pressed area in the original capacitance data sample. The capacitance data within the expanded bounding box is then extracted to obtain the target capacitance data sample, which is designated as the first pressed data sample.

[0131] It should be understood that the scope of the above-mentioned expansion processing can be set as needed. For example, at least one of the top, bottom, left, and right sides of the minimum rectangle can be expanded outwards to achieve the expansion processing. For example, if one side of the minimum rectangle coincides with the corresponding edge of the touch screen, the other three sides can be expanded outwards to achieve the expansion processing; if two sides of the minimum rectangle coincide with the corresponding edge of the touch screen, the other two sides can be expanded outwards to achieve the expansion processing; if the minimum rectangle is located in the middle, all four sides can be expanded outwards to achieve the expansion processing. The outward expansion distance of each side can be set as needed, for example, from 1mm to 10mm. Furthermore, the outward expansion distances of each side can be the same or different for each pair of sides. No limitations are imposed as long as the actual needs are met.

[0132] It should be understood that the extension processing principle of step S2024 is similar to that of step S1024 above. The difference is that step S1024 above obtains the pressing data from the original capacitance data, while step S2024 obtains the first pressing data sample from the original capacitance data sample. Therefore, it can also be understood by analogy with the previous explanation of Figure 4. That is, the P box in Figure 4 is the smallest rectangular box that includes the capacitance data corresponding to the pressing area, while the Q box is the rectangular box after expanding the four sides (p1, p2, p3, p4 sides) of the smallest rectangular box (P box) outward. Since the capacitance data in the Q box is more than the capacitance data in the P box, it can effectively avoid the omission of capacitance data corresponding to the pressing area. Using the target capacitance data sample obtained by this extension processing as the first pressing data sample can improve the effect of model training using the first pressing data sample, so that the trained machine learning model can accurately detect the pressing area.

[0133] Based on this, the optional implementation of steps S2022 to S2024 in this application embodiment can, on the one hand, effectively avoid missing the capacitance data corresponding to the pressing area. Using the target capacitance data sample obtained through such extended processing as the first pressing data sample can improve the effect of model training using the first pressing data sample, so that the trained machine learning model can accurately detect the pressing area. On the other hand, since the target capacitance data sample obtained through extended processing is used as the first pressing data sample to train the machine learning model, instead of inputting all the original capacitance data samples of the touch screen into the machine learning model for training, the computational load can be effectively reduced, thereby improving the model training efficiency and reducing the interference of too much useless data, further improving the model training effect, and thus further ensuring the accuracy of the trained machine learning model in detecting the pressing area.

[0134] S204: Input the first training sample into the machine learning model to predict the center point, and obtain the first predicted center coordinate sample of the pressing area. Then, perform morphological prediction based on the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted morphological parameter sample of the pressing area.

[0135] The machine learning model in this embodiment can be a pre-trained machine learning model. The machine learning model is implemented based on artificial intelligence technology. This machine learning model can be of any type. For example, it can be a convolutional neural network model (including but not limited to AlexNet, Inception, VGG, ResNet, MobileNet, FCN, U-Net, DeepLab, BiSeNet, R-CNN, YOLO, SSD, RetinaNet, etc.), a deep language model (including but not limited to LSTM, GRU, BiLSTM, Transformer, MarianMT, BERT, RoBERTa, XLNet, GPT, T5, S4, Mamba, etc.), or a hybrid variant model based on the above two (including but not limited to ViT, DeiT, ConViT, Swing Transformer, TransUNet, Swing Unet, DETR, Deformable DETR, ConvS2S, ConvBERT, etc.). This embodiment does not limit the training process of the machine learning model. For example, it can be trained using the model training method described in the second aspect below.

[0136] Optionally, when the pressing area is taken as a circular area, the first predicted center coordinate sample can be the center coordinate sample of the shape corresponding to the pressing area, and the first predicted morphological parameter sample can be a circular diameter parameter sample, radius parameter sample, etc. It can be understood that a circle can also be considered as a special type of ellipse with equal major and minor axes.

[0137] When treating the pressed area as an elliptical region, in some optional embodiments, the first predicted center coordinate sample can be the center coordinate sample of an ellipse corresponding to the shape of the pressed area. The center of the ellipse is the intersection of the major and minor axes of the ellipse.

[0138] It should be understood that by determining the first predicted center coordinate sample above to train the machine learning model, the trained machine learning model can accurately detect the center coordinate of the pressing area, so as to improve the user experience of the touch screen by utilizing the center coordinate.

[0139] In some optional embodiments, the first predicted morphological parameter sample includes at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample of the shape corresponding to the pressing area.

[0140] It should be understood that by determining at least one of the above first predicted morphological parameter samples, the machine learning model is trained so that the trained machine learning model can accurately detect the morphological parameters of the pressing area, so as to improve the user experience of the touch screen by utilizing these morphological parameters.

[0141] Optionally, when treating the pressing area as an elliptical region, the first predicted morphological parameter sample may include at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample of the ellipse. The relevant content regarding the major axis tilt angle parameter and the minor axis tilt angle parameter has been explained above and will not be repeated here.

[0142] In some optional embodiments, the method of obtaining the first predicted center coordinate sample in step S104 may include: inputting the first training sample into the machine learning model, extracting features from the first pressing data sample in the first training sample to obtain the feature information of the first pressing data sample, and predicting the center point based on the feature information of the first pressing data sample to obtain the first predicted center coordinate sample of the pressing area; the method of obtaining the first predicted morphological parameter sample in step S104 may include: performing morphological prediction based on the feature information of the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted morphological parameter sample of the pressing area.

[0143] Based on this, in this embodiment, by inputting the first training sample into the machine learning model, feature extraction is performed on the first press data sample in the first training sample to obtain the feature information of the first press data sample. This allows the machine learning model to accurately predict the center point based on the feature information of the first press data sample, effectively obtaining an accurate first predicted center coordinate sample. Furthermore, the machine learning model can accurately predict the shape based on the feature information of the first press data sample and the accurate first predicted center coordinate sample, effectively obtaining an accurate first predicted shape parameter sample. In addition, since the machine learning model first accurately predicts the center point based on the feature information of the first press data sample to obtain a relatively accurate first predicted center coordinate sample, and then uses the machine learning model to perform shape prediction based on the feature information of the first press data sample and the accurate first predicted center coordinate sample, the accuracy of shape prediction can be effectively improved. Therefore, training the machine learning model with such a relatively accurate first predicted center coordinate sample and a relatively accurate first predicted shape parameter sample results in better training performance, enabling the trained machine learning model to more accurately detect the press area of ​​the touch screen.

[0144] Optionally, the above-mentioned optional implementation methods can be implemented based on different parts included in the machine learning model. It should be understood that the specific composition structure of the machine learning model is not limited in any way in the embodiments of this application, as long as it can meet the needs.

[0145] Optionally, as shown in Figure 10, the machine learning model may include: a feature extraction part, a first perception layer part, and a second perception layer part. The first pressing data sample in the first training sample can be used to extract features from the feature extraction part to obtain the feature information of the first pressing data sample. The feature information of the first pressing data sample can then be used by the first perception layer part to predict the center point, obtaining the first predicted center coordinate sample of the pressing area. The feature information of the first pressing data sample and the first predicted center coordinate sample can then be used by the second perception layer part to predict the shape, obtaining the first predicted shape parameter sample of the pressing area.

[0146] It should be understood that with such a model structure, the feature extraction part can accurately extract features from the first press data sample, and the first perception layer and the second perception layer can accurately predict the center point and shape, thereby ensuring the accuracy of the first predicted center coordinate sample and the first predicted shape parameter sample. The machine learning model is trained with such accurate first predicted center coordinate sample and accurate first predicted shape parameter sample, and the training effect is good, so that the machine learning model after training can more accurately detect the press area of ​​the touch screen.

[0147] The feature extraction part in this application embodiment can be composed of any structure, as long as it meets the requirements. For example, in some optional embodiments, referring to FIG5, the feature extraction part includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer; the first pressing data sample in the first training sample can be subjected to corresponding linear transformation processing, convolution processing, and pooling processing in sequence through the linear transformation layer, at least one convolutional layer, and at least one pooling layer of the feature extraction part to obtain the feature information of the first pressing data sample.

[0148] It should be understood that by including the feature extraction part of the above optional structure, the machine learning model can accurately and effectively extract the feature information of the first press data sample, so as to accurately predict the center point and shape, thereby ensuring the accuracy of the first predicted center coordinate sample and the first predicted shape parameter sample. By training the machine learning model with such a more accurate first predicted center coordinate sample and a more accurate first predicted shape parameter sample, the training effect is better, and the machine learning model after training can more accurately detect the press area of ​​the touch screen.

[0149] Regarding the optional model structures mentioned above, the feature extraction part, linear transformation layer, convolutional layer, pooling layer, first perception layer part, and second perception layer part have been explained in detail in the touch area detection method in the first aspect above, and the relevant content can also be understood by referring to the previous text.

[0150] For example, during model training, a linear transformation layer can perform a linear transformation on the first press data sample, slightly rotating and / or translating the press data. At least one convolutional layer and at least one pooling layer perform convolution and pooling operations respectively, enabling the extraction of high-level features from the first press data sample (which can be considered a sampled representation of the first press data sample). Optionally, the feature extraction part of the machine learning model may include multiple convolutional layers and multiple pooling layers to better achieve feature extraction. It should be understood that the specific number can be set according to actual needs and is not limited here. Optionally, the first perception layer may include a multilayer perceptron. Optionally, the second perception layer may include a multilayer perceptron. Optionally, the perceptron can be a universal function approximator, which can be used to summarize the features of the first half of the model and output the prediction result. For example, the multilayer sensor in the first sensing layer can output the prediction result of the center point prediction (i.e., the first predicted center coordinate sample), and the multilayer sensor in the first sensing layer can output the prediction result of the shape prediction (i.e., the first predicted shape parameter sample, including at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample of the shape corresponding to the pressing area).

[0151] S206: Train the machine learning model based on the first prediction center coordinate sample, the first center point true value, the first prediction morphological parameter sample, and the first morphological parameter true value.

[0152] After obtaining the first predicted center coordinate sample, the first center point ground truth, the first predicted morphological parameter sample, and the first morphological parameter ground truth, these can be input into the machine learning model to train the model until the training termination condition is met, thus completing the training of the machine learning model. Training can be supervised training. The training termination condition can be set as needed; for example, it can be ending training after a predetermined number of training epochs (e.g., 1000, 5000, 10000, etc.), or it can be ending training when the loss value of the machine learning model's loss function is less than a certain threshold.

[0153] Based on this, the model training scheme of steps S202 to S206 in this embodiment of the application can obtain a first training sample for training the machine learning model. The first training sample is then input into the machine learning model to predict the center point, obtaining a first predicted center coordinate sample of the pressing area. Furthermore, based on the first pressing data sample and the first predicted center coordinate sample in the first training sample, shape prediction is performed to obtain a first predicted shape parameter sample of the pressing area. Finally, based on the first predicted center coordinate sample, the first center point ground truth, the first predicted shape parameter sample, and the first shape parameter ground truth, the machine learning model is trained. Therefore, on the one hand, a model capable of responding to touch screens can be trained. The machine learning model accurately detects the press area. Furthermore, since the machine learning model first performs accurate center point prediction to obtain a relatively accurate first predicted center coordinate sample, and then uses the machine learning model to perform shape prediction based on the first press data sample and the accurate first predicted center point coordinate sample, the accuracy of the shape parameter prediction of the press area can be effectively improved, and a relatively accurate first predicted shape parameter sample can be obtained. Therefore, by training the machine learning model with such a relatively accurate first predicted center coordinate sample and a relatively accurate first predicted shape parameter sample, the training effect is good, so that the machine learning model after training can more accurately detect the press area of ​​the touch screen.

[0154] Furthermore, since the machine learning model trained using the above-mentioned model training method can be used subsequently, this machine learning model is not only more convenient to use, but also more efficient in development and debugging, easier to maintain and optimize, has better versatility, and helps reduce the extra time and effort that R&D personnel need to invest before and after development is completed.

[0155] It should be understood that in the technical solution of this application embodiment, predicting the first center point coordinate sample is crucial for model training. This is because predicting the first center point coordinate sample first, and then performing shape prediction based on the first press data sample and the first center point coordinate sample to obtain the first predicted shape parameter sample, helps to reduce the negative impact of the nonlinearity of the press data on the prediction parameters, thereby better ensuring the model training effect, enabling the model to effectively improve the accuracy of shape parameter prediction, and enabling the subsequently trained machine learning model to more accurately detect the press area of ​​the touch screen.

[0156] For example, using the model structure shown in Figure 10, the feature extraction part (linear transformation layer, at least one convolutional layer, at least one pooling layer) can be trained to adjust the model parameters to a suitable state. When training the first perception layer and the second perception layer, the first perception layer can be trained separately under supervision to adjust its parameters so that the first perception layer of the machine learning model can achieve a suitable effect in center point prediction. This can effectively improve the negative impact of nonlinearity of the pressing data on the prediction of the pressing area. Then, the second perception layer can be trained under supervision to adjust its parameters so that the second perception layer of the machine learning model can achieve a suitable effect in morphological parameter prediction.

[0157] For example, referring to the workflow of training a machine learning model as shown in Figure 10, the initial process involves obtaining raw capacitance data samples (e.g., obtained by the user pressing on a prompt area on the touchscreen during training sample acquisition, as explained earlier). The capacitance data corresponding to the pressed area is then determined from the raw capacitance data samples. This data is then expanded to obtain target capacitance data samples, which are designated as the first pressed data samples. These first pressed data samples are input into the machine learning model. Through a feature extraction component including a linear transformation layer, at least one convolutional layer, and at least one pooling layer, linear transformation, convolution, and pooling processes are performed sequentially to extract the feature information of the first pressed data sample. The first perception layer of the learning model predicts the center point of the pressing area using the feature information of the first pressing data sample, obtaining the first predicted center coordinate sample of the pressing area. The second perception layer of the machine learning model predicts the shape of the pressing area using the feature information of the first pressing data sample and the first predicted center coordinate sample, obtaining the first predicted shape parameter sample of the pressing area (which may include at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter). The information of the prompt area (such as the center point coordinates and shape parameters of the prompt area of ​​an ellipse) is used as the ground truth (i.e., the ground truth of the first center point and the ground truth of the first shape parameter). The machine learning model is trained using the first pressing data sample, the first predicted shape parameter sample, the ground truth of the first center point, and the ground truth of the first shape parameter. It should be understood that Figure 10 above is not intended to limit any embodiment of this application.

[0158] In some optional embodiments, the model training method of this application embodiment further includes the steps of: obtaining a second training sample, wherein the second training sample includes: a second pressing data sample obtained by the user pressing a given pressing area on the touch screen, a second center point ground value corresponding to the second pressing data sample, and a second morphological parameter ground value corresponding to the second pressing data sample; and using the second training sample to fine-tune the machine learning model to obtain a machine learning model for the user.

[0159] It should be understood that although the machine learning model has a certain degree of universality and generalization after training in step S206, due to the differences in finger shapes among different users, in practice, in order to adapt to the finger shapes of users and further improve the user experience of users using touch screens, this application embodiment can implement "registration" for specific users. By obtaining the second training sample corresponding to the user, the machine learning model is then fine-tuned and trained to obtain a machine learning model for that user, so that the machine learning model after fine-tuning and training can better achieve accurate pressing area detection for that user.

[0160] Optionally, a given pressing area can be displayed on the touch screen to prompt the user to press the given pressing area; a second pressing data sample obtained by the user pressing the given pressing area on the touch screen can be acquired.

[0161] Optionally, the second press data sample can be a capacitance data sample. This capacitance data sample can be either mutual capacitance data or self-capacitance data, depending on whether the touchscreen is based on the mutual capacitance principle or the self-capacitance principle.

[0162] Optionally, the second press data sample in the second training sample can be obtained in the following way: obtaining the second original capacitance data sample, which is a capacitance data sample used to characterize the user's press operation in a given press area of ​​the touch screen; expanding the capacitance data corresponding to the press area in the second original capacitance data sample to obtain the second target capacitance data sample, and determining the second target capacitance data sample as the second press data sample.

[0163] The extended processing here is similar in principle to the original capacitance data sample and the first press data sample in steps S2022 to S2024 above. You can refer to the previous text for understanding, so it will not be repeated here.

[0164] In some optional embodiments, the true value of the second center point is the coordinates of the center point of the shape corresponding to the given pressing area, and the true value of the second morphological parameter is the morphological parameter of the shape corresponding to the given pressing area.

[0165] Given that the center point coordinates and morphological parameters of the corresponding shape of the pressed area can be known, obtaining them as the true values ​​of the second center point and the second morphological parameters helps to ensure the effectiveness of model fine-tuning and makes it easier and more accurate to achieve model fine-tuning in order to obtain a machine learning model for the user.

[0166] Optionally, if the given pressing area is an elliptical region, then the coordinates of the center point of the shape corresponding to the given pressing area are the center coordinates of the ellipse. The shape parameters of the shape corresponding to the given pressing area include, for example, at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the ellipse. The relevant content regarding the major axis tilt angle parameter and the minor axis tilt angle parameter has been explained above and will not be repeated here.

[0167] The settings of the relevant parameters for the given pressing area in this embodiment can be understood by referring to the relevant parameters of the "prompt area" above, and will not be repeated here.

[0168] For example, a finger registration system can be added to terminal devices with touchscreens (such as mobile phones). Users of this device can register through a registration interface displayed on the touchscreen. The registration interface can prompt users to press a given pressing area (e.g., an ellipse, which can also be understood as the "prompt area" mentioned above) displayed on the touchscreen. Multiple different shapes of the given pressing area can be used during registration. Users can choose a given pressing technique (e.g., the pressing techniques in Table 1 above) to completely cover the given pressing area, so as to obtain a second original capacitance data sample for subsequent processing to obtain a second pressing data sample for model fine-tuning training. Since the relevant parameters of the given pressing area displayed on the touchscreen are all known, they can be used as the truth values. That is, if the given pressing area is an elliptical area, the known center point coordinates of the given pressing area can be used as the second center point truth value. The known shape parameters of the given pressing area (e.g., at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter) can be determined as the second shape parameter truth values. It should be understood that the above description is merely an example and is not intended to limit the embodiments of this application.

[0169] In some optional embodiments, referring to the flowchart shown in FIG11, the fine-tuning training of the machine learning model using the second training sample in this embodiment can be achieved through the following steps S302 to S304, specifically:

[0170] S302: Input the second training sample into the machine learning model to predict the center point, and obtain the second predicted center coordinate sample of the given pressing area. Then, perform morphological prediction based on the second pressing data sample and the second predicted center coordinate sample to obtain the second predicted morphological parameter sample of the given pressing area.

[0171] The processing method for center point prediction and shape prediction in step S302 is similar to that in step S204. You can refer to step S204 for understanding. The difference is that step S204 takes the first training sample as input and outputs the first predicted center coordinate sample and the first predicted shape parameter sample of the pressing area, while step S302 takes the second training sample as input and outputs the second predicted center coordinate sample and the second predicted shape parameter sample of the pressing area.

[0172] For example, in some optional embodiments, the second predicted center coordinate sample may be an ellipse center coordinate sample corresponding to the shape of the pressing area.

[0173] For example, in some optional embodiments, the second predicted morphological parameter sample includes at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample corresponding to the shape of the given pressing area.

[0174] For example, in some optional embodiments, the method of obtaining the second predicted center coordinate sample in step S302 may include: inputting the first training sample into the machine learning model, extracting features from the second pressing data sample in the second training sample to obtain the feature information of the second pressing data sample, and predicting the center point based on the feature information of the second pressing data sample to obtain the second predicted center coordinate sample of the given pressing area; the method of obtaining the second predicted morphological parameter sample in step S302 may include: performing morphological prediction based on the feature information of the second pressing data sample and the second predicted center coordinate sample to obtain the second predicted morphological parameter sample of the given pressing area.

[0175] For example, in some optional embodiments, the machine learning model may include: a feature extraction part, a first perception layer part, and a second perception layer part. The feature extraction part can extract features from the second pressing data sample in the second training samples to obtain the feature information of the second pressing data sample. The feature information of the second pressing data sample can then be used by the second perception layer part to predict the center point, obtaining a second predicted center coordinate sample for a given pressing area. The feature information of the second pressing data sample and the second predicted center coordinate sample can then be used by the second perception layer part to predict the shape, obtaining a second predicted shape parameter sample for a given pressing area.

[0176] For example, in some optional embodiments, the feature extraction part includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer; the second pressing data sample in the second training sample can be subjected to corresponding linear transformation processing, convolution processing, and pooling processing in sequence through the linear transformation layer, at least one convolutional layer, and at least one pooling layer of the feature extraction part to obtain the feature information of the second pressing data sample.

[0177] S304: Fine-tune the machine learning model based on the second prediction center coordinate sample, the second center point true value, the second prediction morphological parameter sample, and the second morphological parameter true value.

[0178] After obtaining the second prediction center coordinate samples, the second center point ground truth, the second prediction morphological parameter samples, and the second morphological parameter ground truth, these can be input into the machine learning model for fine-tuning training until the training termination condition is met, thus completing the fine-tuning training of the machine learning model. Fine-tuning training can be supervised training. The termination condition for fine-tuning training can be set as needed. For example, it can be that training ends after a predetermined number of training epochs (e.g., 10, 20, 30, etc.), or it can be that the loss value of the machine learning model's loss function is less than a certain threshold.

[0179] Based on this, through the optional implementation of steps S302 to S304 above, the machine learning model can be effectively fine-tuned using the second training sample, thereby effectively obtaining a machine learning model for the user, so that the machine learning model after fine-tuning can better achieve accurate pressure area detection for a specific user.

[0180] It should be understood that in the technical solution of this application embodiment, predicting the second center point coordinate sample first is crucial for model fine-tuning training. This is because predicting the second center point coordinate sample first, and then performing shape prediction based on the second press data sample and the second center point coordinate sample to obtain the second predicted shape parameter sample, helps to reduce the negative impact of the nonlinearity of the press data on the prediction parameters, thereby better ensuring the model fine-tuning training effect, enabling the model to effectively improve the accuracy of shape parameter prediction, and enabling the machine learning model that has completed subsequent fine-tuning training to more accurately detect the press area of ​​the touch screen for specific users.

[0181] Optionally, for a machine learning model comprising a feature extraction part, a first perceptron part, and a second perceptron part, where the feature extraction part includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer, during the fine-tuning training in step S304, the at least one convolutional layer and at least one pooling layer of the feature extraction part may not be fine-tuned. This is because convolutional layers and pooling layers are used to extract high-level features from the data and have a certain degree of robustness and versatility, thus eliminating the need for further fine-tuning and improving the efficiency of model fine-tuning. However, the linear transformation layer, the first perceptron part, and the second perceptron part of the feature extraction part all require fine-tuning. This is because the linear transformation layer can approximate the user's pressing data during the training phase through a certain linear transformation, so fine-tuning is required. The first and second perception layers can summarize the features of the first half of the model and output the prediction results (i.e., center coordinates, morphological parameters, etc.). There will be some errors between the prediction results and the true values ​​prompted by the system (i.e., true values ​​of center coordinates, true values ​​of morphological parameters, etc.) (because different users' fingers are different, and each user presses the same area differently, etc.), so fine-tuning is required to eliminate the error.

[0182] Of course, in other alternative embodiments, the linear transformation layer, at least one convolutional layer and at least one pooling layer, the first perceptron part and the second perceptron part of the feature extraction part can all be fine-tuned, as long as the requirements are met.

[0183] It is understood that the foregoing description of the model training scheme is merely an exemplary description of the embodiments of this application and is not intended to limit the embodiments of this application in any way.

[0184] As can be seen from the technical solutions of the embodiments of this application above, this technical solution aims to utilize a machine learning model based on artificial intelligence technology to achieve press area detection. This model can accurately determine the center coordinates and shape parameters (e.g., ellipse center coordinates and ellipse shape parameters) of the press area corresponding to the press data, thereby facilitating the optimization of the user's touchscreen and terminal device (e.g., mobile phone) operating experience when needed using accurate detection results. Some technical solutions in the embodiments of this application may include, but are not limited to, the following key points:

[0185] First, the machine learning model in this technical solution can search for the global optimum on a large amount of training data, achieving good generalization and versatility (able to adapt to different users' finger and pressing states to a certain extent). This avoids the problem of traditional solutions where engineers adjust thresholds based on experience, leading to cross-generalization (potentially only adapting to the finger and pressing states of some users). Furthermore, this technical solution pre-trains the machine learning model by collecting diverse training data (covering different pressing positions, different elliptical shapes, and finger characteristics of different groups of people), resulting in good training effects. The pressing area parameters obtained by using this model for pressing area detection are relatively accurate.

[0186] Secondly, to further adapt to the finger shapes of different users, some optional implementations of this technical solution also introduce a finger registration function. This function can collect pressing data samples of a specific user on a given pressing area (elliptical area) on the touchscreen, and then feed them back to the machine learning model for fine-tuning. This allows the machine learning model to better adapt to the pressing area detection needs of specific users. Using the fine-tuned machine learning model for real-time pressing area detection can adapt to the characteristics of different users and achieve high performance.

[0187] Furthermore, in the technical solution of this application embodiment, the machine learning model adopts "two-stage prediction", first predicting the coordinates of the prediction center, and then performing shape prediction based on the pressing data and the coordinates of the prediction center to obtain the predicted shape parameters. This helps to reduce the negative impact of the nonlinearity of the pressing data on the prediction parameters, thereby effectively improving the accuracy of the prediction of the shape parameters of the pressing area and improving the accuracy of the pressing area detection.

[0188] Referring to FIG12, according to a third aspect of the embodiments of this application, a pressing area detection device 1200 is provided, comprising:

[0189] The acquisition module 1202 is used to acquire the press data obtained by the user pressing the touch screen;

[0190] The prediction module 1204 is used to input the pressing data into a machine learning model to predict the center point and obtain the predicted center coordinates of the pressing area; and to perform shape prediction based on the pressing data and the predicted center coordinates to obtain the predicted shape parameters of the pressing area.

[0191] According to the technical solution provided in the embodiments of this application, since the acquisition module 1202 can acquire the pressing data obtained by the user pressing the touch screen, the prediction module 1204 then inputs the pressing data into the machine learning model to predict the center point, obtains the predicted center coordinates of the pressing area, and performs shape prediction based on the pressing data and the predicted center coordinates to obtain the predicted shape parameters of the pressing area. Therefore, on the one hand, the machine learning model can effectively predict the center point coordinates and shape parameters of the user's pressing area on the touch screen, thereby effectively realizing the pressing area detection. On the other hand, since the machine learning model first performs accurate center point prediction to obtain relatively accurate predicted center coordinates, and then uses the machine learning model to perform shape prediction based on the pressing data and accurate predicted center point coordinates, the accuracy of the shape parameter prediction of the pressing area can be effectively improved, thus effectively ensuring the accuracy of the pressing area detection results.

[0192] In some optional embodiments, the prediction module 1204 is specifically used to: input the pressing data into a machine learning model for feature extraction to obtain feature information of the pressing data; perform center point prediction based on the feature information of the pressing data to obtain the predicted center coordinates of the pressing area; and perform shape prediction based on the feature information of the pressing data and the preset center coordinates to obtain the predicted shape parameters of the pressing area.

[0193] In some optional embodiments, the predicted morphological parameters include at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the shape corresponding to the pressing area.

[0194] In some optional embodiments, the acquisition module 1202 is specifically used to: acquire the original capacitance data obtained by the user pressing the touch screen; expand the capacitance data corresponding to the pressed area in the original capacitance data to obtain target capacitance data, and determine the target capacitance data as the press data.

[0195] In some optional embodiments, the press area detection device 1200 is further configured to: display a given press area on the touch screen to prompt the user to press the given press area; and acquire a second press data sample obtained by the user pressing the given press area on the touch screen, wherein the second press data sample is used to fine-tune the machine learning model to obtain a machine learning model for the user.

[0196] In some optional embodiments, the pressing area detection device 1200 is further configured to: control the terminal device on which the touch screen is installed based on the predicted center coordinates of the pressing area and / or the predicted morphological parameters.

[0197] The pressing area detection device 1200 and its optional embodiments in this application have been described in detail in the foregoing pressing area detection method embodiments. Therefore, its related content and beneficial effects can be understood by referring to the content of the optional embodiments of the above method, and will not be repeated here.

[0198] Referring to FIG13, according to a fourth aspect of an embodiment of the present application, a model training apparatus 1300 is provided, comprising:

[0199] The sample acquisition module 1302 is used to acquire a first training sample for training a machine learning model. The first training sample includes a first press data sample obtained by the user pressing the touch screen, a first center point truth value corresponding to the first press data sample, and a first morphological parameter truth value corresponding to the first press data sample.

[0200] The parameter prediction module 1304 is used to input the first training sample into the machine learning model to predict the center point, obtain the first predicted center coordinate sample of the pressing area, and perform morphological prediction based on the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted morphological parameter sample of the pressing area.

[0201] The model training module 1306 is used to train the machine learning model based on the first predicted center coordinate sample, the first center point ground value, the first predicted morphological parameter sample, and the first morphological parameter ground value.

[0202] According to the technical solution provided in the embodiments of this application, since the sample acquisition module 1302 can obtain the first training sample for training the machine learning model, the parameter prediction module 1304 then inputs the first training sample into the machine learning model to predict the center point, obtaining the first predicted center coordinate sample of the pressing area, and performs morphological prediction based on the first pressing data sample and the first predicted center coordinate sample in the first training sample, obtaining the first predicted morphological parameter sample of the pressing area. The model training module 1306 then trains the machine learning model based on the first predicted center coordinate sample, the first center point ground truth, the first predicted morphological parameter sample, and the first morphological parameter ground truth. Therefore, on the one hand, a model capable of practical application can be trained. The present invention utilizes a machine learning model to accurately detect the pressing area of ​​a touchscreen. Furthermore, since the model first performs accurate center point prediction to obtain a relatively accurate first predicted center coordinate sample, and then uses the machine learning model to perform shape prediction based on the first pressing data sample and the accurate first predicted center point coordinate sample, the accuracy of the shape parameter prediction of the pressing area can be effectively improved, resulting in a relatively accurate first predicted shape parameter sample. Therefore, training the machine learning model with such a relatively accurate first predicted center coordinate sample and a relatively accurate first predicted shape parameter sample yields good training results, enabling the trained machine learning model to more accurately detect the pressing area of ​​the touchscreen.

[0203] In some optional embodiments, the parameter prediction module 1304 is specifically used to: input the first training sample into the machine learning model; extract features from the first pressing data sample in the first training sample to obtain feature information of the first pressing data sample; predict the center point based on the feature information of the first pressing data sample to obtain the first predicted center coordinate sample of the pressing area; and predict the shape based on the feature information of the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted shape parameter sample of the pressing area.

[0204] In some optional embodiments, the machine learning model includes: a feature extraction part, a first perception layer part, and a second perception layer part; the parameter prediction module 1304 is specifically used to: extract features from the first pressing data sample in the first training sample through the feature extraction part to obtain feature information of the first pressing data sample; predict the center point of the first pressing data sample through the first perception layer part using the feature information of the first pressing data sample to obtain a first predicted center coordinate sample of the pressing area; and predict the shape of the first pressing data sample and the first predicted center coordinate sample through the second perception layer part to obtain a first predicted shape parameter sample of the pressing area.

[0205] In some optional embodiments, the feature extraction part includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer; the parameter prediction module 1304 is specifically used to: perform corresponding linear transformation processing, convolution processing, and pooling processing on the first pressing data sample in the first training sample through the linear transformation layer, at least one convolutional layer, and at least one pooling layer of the feature extraction part in sequence to obtain the feature information of the first pressing data sample.

[0206] In some optional embodiments, the first predicted morphological parameter sample includes at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample of the shape corresponding to the pressing area.

[0207] In some optional embodiments, the sample acquisition module 1302 is specifically used to: acquire an original capacitance data sample, wherein the original capacitance data sample is a capacitance data sample used to characterize the user's pressing operation on the touch screen; perform extended processing on the capacitance data corresponding to the pressing area in the original capacitance data sample to obtain a target capacitance data sample, and determine the target capacitance data sample as the first pressing data sample.

[0208] In some optional embodiments, the model training device 1300 is further configured to: obtain a second training sample, wherein the second training sample includes: a second pressing data sample obtained by the user pressing a given pressing area on the touch screen, a second center point ground value corresponding to the second pressing data sample, and a second morphological parameter ground value corresponding to the second pressing data sample; and use the second training sample to fine-tune the machine learning model to obtain a machine learning model for the user.

[0209] In some optional embodiments, the true value of the second center point is the coordinates of the center point of the shape corresponding to the given pressing area, and the true value of the second morphological parameter is the morphological parameter of the shape corresponding to the given pressing area.

[0210] In some optional embodiments, the model training device 1300 is specifically used to: input the second training sample into the machine learning model to predict the center point, obtain a second predicted center coordinate sample of the given pressing area, and perform morphological prediction based on the second pressing data sample and the second predicted center coordinate sample to obtain a second predicted morphological parameter sample of the given pressing area; and fine-tune the machine learning model based on the second predicted center coordinate sample, the second center point ground truth, the second predicted morphological parameter sample, and the second morphological parameter ground truth.

[0211] The model training device 1300 and its optional embodiments in this application have been described in detail in the foregoing model training method embodiments. Therefore, their related content and beneficial effects can be understood by referring to the content of the optional embodiments of the above method, and will not be repeated here.

[0212] According to a fifth aspect of the embodiments of this application, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to execute the pressing area detection method described in the first aspect or the model training method described in the third aspect by running the computer program stored in the memory.

[0213] Figure 14 shows a structural block diagram of an optional electronic device according to an embodiment of this application. This embodiment does not limit the specific implementation of the electronic device 1400. As an example, referring to Figure 14, the electronic device 1400 provided in this embodiment includes: a processor 1402, a communications interface 1404, a memory 1406, and a communication bus 1408. Wherein:

[0214] The processor 1402, communication interface 1404, and memory 1406 communicate with each other via communication bus 1408.

[0215] Communication interface 1404 is used to communicate with other electronic devices or servers.

[0216] The processor 1402 is used to execute the computer program 1410, specifically the relevant steps in any of the aforementioned press area detection method or model training method embodiments.

[0217] Specifically, computer program 1410 may include program code that includes computer operation instructions.

[0218] The processor 1402 may be a CPU, a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0219] Memory 1406 is used to store computer program 1410. Memory 1406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0220] Specifically, computer program 1410 can be used to cause processor 1402 to execute the pressing area detection method or model training method in any of the foregoing embodiments.

[0221] The specific implementation of each step in computer program 1410 can be found in the corresponding steps and units described in any of the foregoing embodiments of the pressing area detection method or model training method, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device and module described above can be referred to the corresponding process description in the foregoing method embodiments, and will not be repeated here.

[0222] The electronic device 1400 in this application embodiment has been described in detail in the foregoing embodiments of the pressing area detection method or model training method. Therefore, its related content and beneficial effects can be understood by referring to the above method embodiments, and will not be repeated here.

[0223] According to a sixth aspect of the embodiments of this application, the embodiments of this application also provide a computer storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the pressing area detection method or model training method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk, etc.

[0224] According to a seventh aspect of the embodiments of this application, the embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the pressing area detection method or model training method as described in any of the embodiments of the plurality of methods described above.

[0225] The pressing area detection device 1200 / model training device 1300 / electronic device 1400 / computer storage medium / computer program product embodiments in this application have been described in detail in the foregoing pressing area detection method and model training method embodiments. Therefore, their related content and beneficial effects can be understood by referring to the above method embodiments, and will not be repeated here.

[0226] It should be understood that the various figures in the embodiments of this application are for the purpose of illustrating the structure, and the dimensions of each structure may not be drawn according to the actual scale.

[0227] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0228] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0229] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0230] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of the embodiments of this application.

[0231] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for detecting a pressure area, comprising: Obtain the pressure data obtained from the user's press operation on the touch screen; The pressing data is input into a machine learning model to predict the center point, thereby obtaining the predicted center coordinates of the pressing area. Furthermore, based on the pressing data and the predicted center coordinates, morphological prediction is performed to obtain the predicted morphological parameters of the pressing area.

2. The method according to claim 1, wherein, The step of inputting the pressure data into a machine learning model for center point prediction to obtain the predicted center coordinates of the pressure domain includes: The compression data is input into a machine learning model for feature extraction to obtain the feature information of the compression data. Based on the feature information of the compression data, the center point is predicted to obtain the predicted center coordinates of the compression area. The step of predicting the shape of the pressed area based on the pressure data and the predicted center coordinates to obtain the predicted shape parameters of the pressed area includes: Based on the feature information of the pressing data and the preset center coordinates, shape prediction is performed to obtain the predicted shape parameters of the pressing area.

3. The method according to claim 1, wherein, The predicted morphological parameters include at least one of the major axis parameter, minor axis parameter, major axis tilt angle parameter, and minor axis tilt angle parameter of the shape corresponding to the pressing area.

4. The method according to any one of claims 1-3, wherein, The acquisition of press data obtained by the user pressing the touchscreen includes: Obtain raw capacitance data obtained by the user pressing the touch screen; The capacitance data corresponding to the pressing area in the original capacitance data is expanded to obtain the target capacitance data, and the target capacitance data is determined as the pressing data.

5. The method according to any one of claims 1-3, wherein, The method further includes: A given pressing area is displayed on the touch screen to prompt the user to press the given pressing area; A second press data sample is obtained by the user pressing a given pressing area on the touch screen, wherein the second press data sample is used to fine-tune the machine learning model to obtain a machine learning model for the user.

6. The method according to any one of claims 1-3, wherein, The method further includes: The terminal device with the touch screen installed is controlled based on the predicted center coordinates of the pressing area and / or the predicted shape parameters.

7. A model training method, comprising: Obtain a first training sample for training a machine learning model, wherein the first training sample includes a first press data sample obtained by the user pressing the touch screen, a first center point truth value corresponding to the first press data sample, and a first morphological parameter truth value corresponding to the first press data sample. The first training sample is input into the machine learning model to predict the center point, thereby obtaining the first predicted center coordinate sample of the pressing area. Furthermore, based on the first pressing data sample and the first predicted center coordinate sample, morphological prediction is performed to obtain the first predicted morphological parameter sample of the pressing area. The machine learning model is trained based on the first predicted center coordinate sample, the first center point ground value, the first predicted morphological parameter sample, and the first morphological parameter ground value.

8. The method according to claim 7, wherein, The step of inputting the first training sample into the machine learning model to predict the center point and obtain the first predicted center coordinate sample of the pressing area includes: The first training sample is input into the machine learning model, and the first pressing data sample in the first training sample is subjected to feature extraction to obtain the feature information of the first pressing data sample. Based on the feature information of the first pressing data sample, the center point is predicted to obtain the first predicted center coordinate sample of the pressing area. The step of performing morphological prediction based on the first pressure data sample and the first prediction center coordinate sample to obtain the first predicted morphological parameter sample of the pressure area includes: Based on the feature information of the first pressing data sample and the first prediction center coordinate sample, morphological prediction is performed to obtain the first predicted morphological parameter sample of the pressing area.

9. The method according to claim 8, wherein, The machine learning model includes: a feature extraction part, a first perceptual layer part, and a second perceptual layer part; The step of extracting features from the first pressure data sample in the first training sample to obtain feature information of the first pressure data sample, and predicting the center point based on the feature information of the first pressure data sample to obtain the first predicted center coordinate sample of the pressure area includes: The first pressing data sample in the first training sample is subjected to feature extraction by the feature extraction part to obtain the feature information of the first pressing data sample. The feature information of the first pressing data sample is then used to predict the center point by the first perception layer part to obtain the first predicted center coordinate sample of the pressing area. The step of performing morphological prediction based on the feature information of the first pressure data sample and the first prediction center coordinate sample to obtain the first predicted morphological parameter sample of the pressure area includes: The feature information of the first pressing data sample and the first prediction center coordinate sample are used to perform morphological prediction through the second perception layer to obtain the first predicted morphological parameter sample of the pressing area.

10. The method according to claim 9, wherein, The feature extraction part includes a linear transformation layer, at least one convolutional layer, and at least one pooling layer; The step of extracting features from the first pressure data sample in the first training sample through the feature extraction part to obtain the feature information of the first pressure data sample includes: The first pressing data sample in the first training sample is subjected to corresponding linear transformation, convolution and pooling processes in the linear transformation layer, at least one convolutional layer and at least one pooling layer of the feature extraction part in sequence to obtain the feature information of the first pressing data sample.

11. The method according to claim 7, wherein, The first predicted morphological parameter sample includes at least one of the major axis parameter sample, minor axis parameter sample, major axis tilt angle parameter sample, and minor axis tilt angle parameter sample of the shape corresponding to the pressing area.

12. The method according to any one of claims 7-11, wherein, The first press data sample obtained by the user pressing the touchscreen is obtained in the following way: Obtain raw capacitance data samples, which are capacitance data samples obtained to characterize the user's pressing operation on the touch screen; The capacitance data corresponding to the pressing area in the original capacitance data sample is expanded to obtain the target capacitance data sample, and the target capacitance data sample is determined as the first pressing data sample.

13. The method according to any one of claims 7-11, wherein, The method further includes: Obtain a second training sample, wherein the second training sample includes: a second pressing data sample obtained by the user pressing a given pressing area on the touch screen, a second center point true value corresponding to the second pressing data sample, and a second morphological parameter true value corresponding to the second pressing data sample; The machine learning model is fine-tuned using the second training sample to obtain a machine learning model for the user.

14. The method according to claim 13, wherein, The true value of the second center point is the coordinates of the center point of the shape corresponding to the given pressing area, and the true value of the second morphological parameter is the morphological parameter of the shape corresponding to the given pressing area.

15. The method according to claim 13, wherein, The step of fine-tuning the machine learning model using the second training samples includes: The second training sample is input into the machine learning model to predict the center point, thereby obtaining the second predicted center coordinate sample of the given pressing area. Furthermore, based on the second pressing data sample and the second predicted center coordinate sample, morphological prediction is performed to obtain the second predicted morphological parameter sample of the given pressing area. The machine learning model is fine-tuned based on the second predicted center coordinate sample, the second center point true value, the second predicted morphological parameter sample, and the second morphological parameter true value.

16. A pressure detection device, comprising: The acquisition module is used to acquire the press data obtained by the user pressing the touch screen; The prediction module is used to input the pressure data into a machine learning model to predict the center point and obtain the predicted center coordinates of the pressure area; and to perform shape prediction based on the pressure data and the predicted center coordinates to obtain the predicted shape parameters of the pressure area.

17. A model training device, comprising: The sample acquisition module is used to acquire the first training sample for training the machine learning model. The first training sample includes the first press data sample obtained by the user pressing the touch screen, the ground truth value of the first center point corresponding to the first press data sample, and the ground truth value of the first morphological parameter corresponding to the first press data sample. The parameter prediction module is used to input the first training sample into the machine learning model to predict the center point, obtain the first predicted center coordinate sample of the pressing area, and perform morphological prediction based on the first pressing data sample and the first predicted center coordinate sample to obtain the first predicted morphological parameter sample of the pressing area. The model training module is used to train the machine learning model based on the first predicted center coordinate sample, the first center point ground truth, the first predicted morphological parameter sample, and the first morphological parameter ground truth.

18. An electronic device comprising: The processor, the communication interface, the memory, and the communication bus are provided, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The memory is used to store computer programs; The processor is configured to execute the computer program stored on the memory to perform the method of any one of claims 1-15.

19. A computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1-15.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-15.

Images