Sensitivity threshold value determination method for capacitance value sensor, and electronic device

Abstract

Provided in the present application are a sensitivity threshold value determination method for a capacitance value sensor, and an electronic device. The method comprises: when a sensing object is in contact with a screen of a second electronic device, acquiring target touch-control data, wherein the target touch-control data at least comprises target capacitance value information, and the target capacitance value information indicates a capacitance value of a capacitance value sensor in the second electronic device; on the basis of the target touch-control data, determining the protection state of the second electronic device, such that the protection state of the second electronic device itself is automatically sensed; and on the basis of the determined protection state, determining a sensitivity threshold value corresponding to the capacitance value sensor, so that the capacitance value sensor operates on the basis of the determined sensitivity threshold value. By means of the present application, automatic matching between a sensitivity threshold value of a capacitance value sensor and the protection state of a second electronic device is realized, and the problems of touch interruption, false touch, etc., that occur after the protection state of the second electronic device changes can be effectively solved, thereby improving the touch-control experience of a user using the electronic device.

Description

Method for determining sensitivity threshold of capacitance sensor and electronic device

[0001] This application is based on the Chinese patent application with application number 202410045827.8, application date January 10, 2024, and invention name “A method for determining the sensitivity threshold of a capacitance sensor and an electronic device”, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a method for determining a sensitivity threshold of a capacitance sensor and an electronic device. Background Art

[0003] In order to protect the electronic devices during use, users often put protective films on the screens of the electronic devices and put on protective cases. However, usually, the sensitivity threshold of the capacitance sensor in the electronic device is set according to the state of the electronic device without the film or the case. The sensitivity threshold is used to indicate the minimum input value for the sensor to generate an output, that is, the capacitance sensor will only generate an output when the input value exceeds the sensitivity threshold. When the electronic device is covered with a film or a protective case, the sensing value sensed by the capacitance sensor in the electronic device will deviate. For example, when the user touches the screen of the electronic device before and after the film is applied with the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the electronic device after the film is applied will be lower than that before the film is applied, and the capacitance amplitude sensed after the film is applied may not reach the sensitivity threshold of the capacitance sensor, causing problems such as disconnection and false touch when the user uses the electronic device. Summary of the Invention

[0004] In order to solve the above technical problems, the present application provides a method for determining the sensitivity threshold of a capacitance sensor and an electronic device, so as to realize automatic adaptation of the sensitivity threshold of the capacitance sensor to the protection state of the electronic device, solve the problems of disconnection and false touch that occur after the electronic device is covered with a film or a case, and improve the touch experience of users when using the electronic device.

[0005] In a first aspect, the present application provides a method for determining a sensitivity threshold of a capacitance sensor, which is applied to a second electronic device, the method comprising: obtaining target touch data when a sensing object is in contact with a screen of the second electronic device; the target touch data comprises at least target capacitance information; the target capacitance information indicates the capacitance of a capacitance sensor in the second electronic device; determining a protection state of the second electronic device based on the target touch data; the protection state comprises one of a state without a film and without a case, a state with a film but not in a case, a state with a case but not a film, and a state with a film and a case; and determining a sensitivity threshold corresponding to the capacitance sensor based on the protection state.

[0006] In an embodiment of the present application, the second electronic device is an electronic device that can automatically sense the protection state it is in. When the sensing object contacts the screen of the second electronic device, the second electronic device obtains target touch data and determines the protection state of the second electronic device based on the target touch data, so as to realize automatic perception of the protection state it is in. The second electronic device determines the sensitivity threshold of the capacitance sensor based on the sensed protection state, so that the capacitance sensor can operate based on the determined sensitivity threshold, thereby realizing automatic adaptation of the sensitivity threshold of the capacitance sensor to the protection state of the second electronic device, which can effectively solve the problems of disconnection, false touch, etc. that occur after the protection state of the second electronic device is changed, and improve the touch experience of users when using electronic devices.

[0007] According to the first aspect, when the sensing object is in contact with the screen of the second electronic device, target touch data is obtained, including: when the sensing object is in contact with the screen of the second electronic device, target capacitance information is obtained based on a preset touch sampling rate, and multiple frames of target capacitance information constitute target capacitance information; wherein the target capacitance information includes the capacitance of the capacitance sensor in the second electronic device at the corresponding acquisition moment.

[0008] In an embodiment of the present application, the target capacitance information in the target touch data includes multi-frame target capacitance information, which can effectively characterize the contact characteristics when the sensing object contacts the screen of the second electronic device, and provide effective data support for subsequent perception of the protection status of the second electronic device based on the target capacitance information.

[0009] According to the first aspect, or any implementation of the first aspect above, the protection state of the second electronic device is determined according to the target capacitance information, including: determining touch feature data according to the target touch data; inputting the touch feature data into the first model to obtain the protection state output by the first model.

[0010] In an embodiment of the present application, the second electronic device only needs to input the touch feature data into the first model to obtain the protection status output by the first model, which simplifies the way the second electronic device determines its own protection status and improves the speed at which the second electronic device perceives its own protection status.

[0011] According to the first aspect, or any implementation of the first aspect above, touch feature data is determined based on the target touch data, including: determining the target capacitance feature data based on the target capacitance information; using the target capacitance feature data as the touch feature data; or, the target touch data also includes target auxiliary data; performing data fusion processing on the target capacitance feature data and the target auxiliary data to obtain the touch feature data; wherein the target auxiliary data includes one or more of target charging status information, target posture information, and target touch pressure information.

[0012] In the embodiment of the present application, the target auxiliary data are all data that will affect the capacitance of the capacitance sensor. Therefore, involving the target auxiliary data in the process of predicting the protection status of the second electronic device can reduce the error in predicting the protection status of the second electronic device and improve the accuracy of the determined protection status of the second device.

[0013] According to the first aspect, or any implementation of the first aspect above, determining target capacitance characteristic data based on target capacitance information includes: selecting multiple frames of capacitance information from the target capacitance information according to a preset selection rule; and extracting a capacitance matrix of a preset size from each frame of the selected capacitance information, and combining the data into capacitance characteristic data. In an embodiment of the present application, selecting multiple frames of capacitance information according to the preset selection rule and then extracting a capacitance matrix of a preset size from each frame of capacitance information can reduce data redundancy and extract valid data to be combined into capacitance characteristic data.

[0014] According to the first aspect, or any implementation of the first aspect above, the matrix center of the capacitance matrix is ​​the maximum capacitance in the corresponding capacitance information.

[0015] In an embodiment of the present application, the maximum capacitance value in the capacitance information corresponds to the contact point when the sensing object touches the screen of the second electronic device. Therefore, using the maximum capacitance value in the capacitance information as the matrix center of the capacitance matrix can make the extracted capacitance matrix include the capacitance value of the capacitance sensor near the contact point, thereby improving the validity of the capacitance characteristic data.

[0016] According to the first aspect, or any implementation method of the first aspect above, the preset selection rule includes: in the target capacitance information, with the target capacitance information where the maximum capacitance is located as the center, selecting the center, the preset frame target capacitance information sorted before the center, and the preset frame target capacitance information sorted after the center.

[0017] In an embodiment of the present application, the target capacitance information at the maximum capacitance value can effectively characterize the contact characteristics between the sensing object and the second electronic device. Therefore, with the target capacitance information at the maximum capacitance value as the center, the center, the preset frame target capacitance information sorted before the center, and the preset frame target capacitance information sorted after the center are selected to improve the validity of the capacitance characteristic data.

[0018] According to the first aspect, or any implementation method of the first aspect above, the preset selection rule also includes: discarding the target capacitance information of the first N frames from the target capacitance information of the preset frames sorted before the center; wherein N frames are greater than 1 frame and less than the preset frame; and discarding the target capacitance information of the last N frames from the target capacitance information of the preset frames sorted after the center.

[0019] In an embodiment of the present application, since the capacitance sensed by the capacitance sensor is unstable when the finger of the sensing object approaches and leaves the screen of the second electronic device when a touch operation is performed on the second electronic device, it is necessary to filter out capacitance information with higher data stability from the initially selected capacitance information, thereby further improving the validity of the capacitance characteristic data determined based on the filtered capacitance information.

[0020] According to the first aspect, or any implementation of the first aspect above, based on the protection status, the sensitivity threshold corresponding to the capacitance sensor is determined, including: when the protection status indicates a state without a film and a case, determining the sensitivity threshold corresponding to the capacitance sensor to be a first threshold; when the protection status indicates a state with a film but not in a case, determining the sensitivity threshold corresponding to the capacitance sensor to be a second threshold; the second threshold is less than the first threshold; when the protection status indicates a state without a film on the case, determining the sensitivity threshold corresponding to the capacitance sensor to be a third threshold; the third threshold is less than the first threshold; when the protection status indicates a state with a film and a case, determining the sensitivity threshold corresponding to the capacitance sensor to be a fourth threshold; the fourth threshold is less than the second threshold and the third threshold.

[0021] In an embodiment of the present application, the second electronic device pre-stores a correspondence between the protection state and the sensitivity threshold of the capacitance sensor. Therefore, after the second electronic device senses its own protection state, it can automatically adapt the sensitivity threshold of the corresponding capacitance sensor according to the sensed protection state.

[0022] According to the first aspect, after determining the sensitivity threshold corresponding to the capacitance sensor based on the protection state, the method further includes: adjusting the sensitivity parameter of the capacitance sensor to the determined sensitivity threshold, so that the touch chip connected to the capacitance sensor outputs a sensing signal based on the determined sensitivity threshold.

[0023] In an embodiment of the present application, the second electronic device adjusts the sensitivity parameter of the capacitance sensor to a determined sensitivity threshold, enabling the capacitance sensor to operate based on the determined sensitivity threshold, thereby realizing automatic adjustment of the sensitivity threshold of the capacitance sensor according to the protection state of the second electronic device. This can effectively solve the problems of disconnection, false touch, etc. that occur after the protection state of the second electronic device is changed, thereby improving the user's touch experience when using the electronic device.

[0024] In a second aspect, the present application provides a model training method, which is characterized in that it is applied to a server, and the server is connected to a first electronic device; the method includes: obtaining touch sample data of the first electronic device in various protection states; the protection state indicates the state without film and without case, the state with film but not in case, the state with case but not film, and the state with film and case; the touch sample data includes capacitance information and charging status information; performing first data processing on the touch sample data to obtain training feature data; training an initial first model based on the training feature data to obtain a trained first model; the trained first model is used to be configured in a second electronic device to determine the protection state of the second electronic device.

[0025] In an embodiment of the present application, the first model is trained using touch sample data of the first electronic device in various protection states, so that the trained first model has the function of automatically sensing the protection state of the electronic device and can determine the protection state of the corresponding electronic device based on the touch data.

[0026] According to the second aspect, touch sample data of the first electronic device in various protection states is obtained, including: receiving first capacitance information sent by the first electronic device; the first capacitance information indicates the capacitance of the capacitance sensor in the first electronic device during the process of the tester touching the screen of the first electronic device in the first protection state; the first protection state indicates any one of a state without film and without case, a state with film but not in case, a state with case but not film, and a state with film and case.

[0027] In the embodiment of the present application, the capacitance of the capacitance sensor can effectively represent the contact characteristics between the tester and the first electronic device. Therefore, using the capacitance of the capacitance sensor as the touch sample data can improve the validity of the touch sample data.

[0028] According to the second aspect, or any implementation method of the second aspect above, the touch sample data is subjected to a first data processing to obtain training feature data, including: extracting the first capacitance feature data based on the first capacitance information to extract effective information from the first capacitance information and reduce data redundancy; labeling the first capacitance feature data according to the first protection state of the first electronic device to obtain the first training feature data; or, the touch sample data also includes auxiliary parameters; performing data fusion processing on the first capacitance feature data and the auxiliary parameters to obtain the first fused feature data; wherein the auxiliary parameters include one or more of the first charging state information, the first posture information and the first touch pressure information; labeling the fused feature data according to the first protection state of the first electronic device to obtain the first training feature data, and the protection state label on the training feature data can be used as a true result to measure the accuracy of the first model output.

[0029] In this embodiment of the present application, incorporating this auxiliary parameter into the training process of the first model can improve the accuracy of the trained first model. The first fused feature data is labeled based on the first protection state of the first electronic device to obtain first training feature data. The protection state label on this training feature data can be used as a true result to measure the accuracy of the first model output.

[0030] According to the second aspect, or any implementation method of the second aspect above, an initial first model is trained based on training feature data to obtain a trained first model, including: inputting the training feature data into the initial first model to obtain an output result; based on the output result and the loss function, updating the weight of the initial first model, so that the error between the protection status predicted by the first model (output result) and the protection status indicated by the input data label (actual result) is less than a preset error, thereby obtaining the trained first model.

[0031] According to the second aspect, or any implementation of the second aspect above, the first model indicates a convolutional neural network model; the loss function indicates a cross-entropy loss function.

[0032] According to the second aspect, after obtaining the trained first model, the method also includes: performing a preset format conversion on the trained first model so that the trained first model is converted into a lightweight model that can be adapted to a mobile terminal; deploying the first model converted into the preset format on a second electronic device so that the second electronic device has the function of automatically sensing its own protection status. For example, when a user is using the second electronic device and performs a touch operation on the screen of the second electronic device, the second electronic device can obtain target touch data based on the user's touch operation, and input the processed target touch data into the pre-deployed first model to obtain protection status information output by the first model.

[0033] In a third aspect, the present application provides an electronic device, comprising: one or more processors; a memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by one or more processors, the electronic device executes the method for determining the sensitivity threshold of a capacitance sensor as described in the first aspect, or any one of the implementations of the first aspect above.

[0034] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0035] In a fourth aspect, the present application provides a server comprising: one or more processors; a memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by one or more processors, the electronic device executes the model training method as in the second aspect.

[0036] The technical effects corresponding to the fourth aspect can be found in the technical effects corresponding to the above-mentioned second aspect, and will not be repeated here.

[0037] In a fifth aspect, the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method for determining the sensitivity threshold of a capacitance sensor as in the first aspect, or any one of the implementations of the first aspect above.

[0038] The fifth aspect corresponds to the first aspect and any implementation of the first aspect. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0039] In a sixth aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the model training method of the second aspect.

[0040] The technical effects corresponding to the sixth aspect can be found in the technical effects corresponding to the above-mentioned second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a schematic diagram showing the distribution of capacitance values ​​of a capacitance sensor collected when an electronic device is in a film-applied state and a film-unapplied state, provided by an embodiment of the present application;

[0042] FIG2 is a schematic diagram of a scenario provided by an embodiment of the present application;

[0043] FIG3 is a schematic diagram of a scenario provided by an embodiment of the present application;

[0044] FIG4 is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0045] FIG5a is a schematic diagram of a flow chart of a first model training method provided in an embodiment of the present application;

[0046] FIG5 b is a flow chart of a method for a server to obtain touch sample data according to an embodiment of the present application;

[0047] FIG5c is a flow chart of a method for obtaining training feature data according to an embodiment of the present application;

[0048] FIG5 d is a flow chart of another method for obtaining training feature data provided in an embodiment of the present application;

[0049] FIG6 is a schematic diagram of an electronic device obtaining capacitance information according to an embodiment of the present application;

[0050] FIG7 is an example diagram of an electronic device posture provided by an embodiment of the present application;

[0051] FIG8 is an example diagram of a preprocessing module for extracting capacitance characteristic data according to an embodiment of the present application;

[0052] FIG9 is a schematic diagram of a model training provided in an embodiment of the present application;

[0053] FIG10 is a flowchart of a first model deployment method provided in an embodiment of the present application;

[0054] FIG11 is a flow chart of a method for determining a sensitivity threshold of a capacitance sensor provided in an embodiment of the present application;

[0055] FIG12 is a schematic diagram showing the distribution of capacitance values ​​of a capacitance sensor collected when an electronic device is in a case state and an uncase state provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0058] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0059] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0060] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0061] FIG1 is a schematic diagram showing the distribution of capacitance values ​​of a capacitance sensor collected when an electronic device is in a film-applied state and a film-unapplied state, provided by an embodiment of the present application. FIG2 is a schematic diagram showing a scenario provided by an embodiment of the present application. Before introducing the embodiment of the present application, the application scenario of the embodiment of the present application is first described based on FIG1 and FIG2. In both FIG1 and FIG2, a mobile phone is used as an example of the electronic device 100. The unit of capacitance in the following embodiments is picofarad (pF).

[0062] In Figure 1, a user performs a touch operation on the same position of an electronic device 100 without a film attached and on the same position of the electronic device 100 with a film attached, respectively, as an example. A capacitance sensor array 101 is provided under the screen of the electronic device 100. The capacitance sensor array 101 includes a plurality of capacitance sensors arranged in rows and columns, where each small square represents a capacitance sensor. It should be noted that the capacitance sensor array in Figure 1 is for illustrative purposes only. The number of rows and columns of the capacitance sensor array and the number of capacitance sensors contained in each row or column can be set according to actual application conditions and are not specifically limited in the embodiments of this application.

[0063] Generally speaking, the process of a capacitance sensor sensing a finger is related to the distance between the finger and the screen of the electronic device 100. As the finger continues to approach the screen of the electronic device 100, the capacitance of the capacitance sensor that is closer to the finger is more likely to change, and the capacitance of the capacitance sensor that is farther away from the finger is less likely to change.

[0064] As shown in (1) of FIG1 , in a scenario where the electronic device 100 is not filmed, the user touches the screen of the electronic device 100 with a finger. The capacitance of the capacitance sensor near the user's finger is shown as 101-1, where each block is used to represent a capacitance sensor, and the number in the block is used to represent the capacitance value of the capacitance sensor. For example, the capacitance value of capacitance sensor 101-01 is 800. It should be noted that a capacitance value less than the sensitivity threshold of the capacitance sensor can be considered as a noise value. The sensitivity threshold is, for example, 120.

[0065] As shown in (2) of FIG1 , in a scenario where the electronic device 100 is in a film-applied state, a protective film 102 is applied to the electronic device 100. The user touches the screen of the electronic device 100 with a finger, and the capacitance of the capacitance sensor near the user's finger is shown in 101-2, where each block represents a capacitance sensor, and the number in the block represents the capacitance value of the capacitance sensor. For example, the capacitance value detected by the capacitance sensor 101-01 is 100.

[0066] Comparing the capacitance distribution shown in 101-1 with that shown in 101-2, the capacitance value sensed by capacitance sensor 101-01 when electronic device 100 is not in the film-applied state is 800, while the capacitance value sensed when the film-applied state is 100. In other words, for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the film-applied state is lower than the capacitance amplitude sensed when the film-applied state is not, and the capacitance value sensed by the capacitance sensor in the film-applied state may be lower than the sensitivity threshold.

[0067] Referring to the side view of the electronic device 100 shown in (1) of FIG2 , a protective film 102 is attached to the electronic device 100. In the first display interface 103 of the electronic device 100 shown in (2) of FIG2 , icons of multiple applications are included, such as icons of applications such as clock, calendar, gallery, memo, camera, address book, information and phone. In some embodiments, the user clicks on the phone icon 1031 in the first display interface 103, and the electronic device 100 should respond to the click operation and display the second display interface 104 shown in (3) of FIG2 . However, when the electronic device 100 is affixed with the protective film 102, as shown in FIG1 , the capacitance amplitude sensed by the capacitance sensor in the electronic device 100 will be reduced. In some cases, the capacitance value of the capacitance sensor may not reach the sensitivity threshold of the capacitance sensor, causing the touch module connected to the capacitance sensor to regard the capacitance value as noise, and the touch module will not generate output. The electronic device 100 cannot sense the click operation, let alone respond to the click operation and display the second display interface 103 shown in (3) of FIG2 .

[0068] To this end, an embodiment of the present application provides a method for determining a sensor sensitivity threshold. When a user touches the screen of an electronic device, the electronic device obtains target touch data and determines the protection status of the electronic device based on the target touch data. That is, the electronic device can automatically sense the film and case of the electronic device, and then determine the sensitivity threshold of the capacitance sensor based on the protection status of the electronic device, so that the capacitance sensor can operate based on the determined sensitivity threshold. Automatic adjustment of the sensitivity threshold of the capacitance sensor is achieved, which can effectively improve the situations of broken touch, false touch, etc. that occur after the electronic device is in the film or case state, and improve the user's touch experience when using the electronic device.

[0069] The sensor sensitivity threshold determination method provided in the embodiments of the present application can be applied to electronic devices, which may be wearable electronic devices (such as watches), portable computers (such as mobile phones), tablet computers, laptop computers, personal computers (PCs), augmented reality (AR) and virtual reality (VR) devices, in-vehicle computers, and the like. The following embodiments do not impose any special restrictions on the specific form of the electronic device.

[0070] Before describing the technical solutions of the embodiments of the present application, the electronic device of the embodiments of the present application will first be described with reference to the accompanying drawings. Figure 3 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application. It should be understood that the electronic device 100 shown in Figure 3 is only an example of an electronic device, and the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in Figure 3 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0071] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0072] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0073] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0074] In an embodiment of the present application, when a user touches the screen of an electronic device, the controller may control the processor to obtain the capacitance value of the capacitance sensor and to obtain charging status information from the charging management module. In some embodiments, when a user touches the screen of an electronic device, the controller may also control the processor to obtain posture information collected by a gyroscope sensor and touch pressure information collected by a pressure sensor.

[0075] The processor 110 may further include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0076] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, a pulse code modulation (PCM) interface, a general-purpose input / output (GPIO) interface, and / or a universal serial bus (USB) interface.

[0077] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0078] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0079] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0080] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0081] In an embodiment of the present application, when the server performs the first model training, the server can achieve a wired connection with the first electronic device through the above-mentioned GPIO interface or USB interface. The server and the first electronic device can achieve data transmission through the wired connection.

[0082] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142, it can also power the electronic device through the power management module 141. In an embodiment of the present application, the charging management module 140 is also used to transmit charging status information of the electronic device to the processor 110, and the charging status information is, for example, not charged or charged.

[0083] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0084] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0085] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0086] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the electronic device 100.

[0087] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology. In an embodiment of the present application, when the server performs the first model training, the server can achieve a wired connection with the first electronic device through a wireless connection. The server and the first electronic device can achieve data transmission through this wireless connection.

[0088] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0089] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0090] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0091] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, in an embodiment of the present application, after configuring the first model in the electronic device, when a user touches the electronic device, the digital signal processor can be used to process touch data obtained by the electronic device based on the touch operation, such as capacitance information, charging status information, etc.

[0092] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0093] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0094] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0095] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device comprising at least two parallel plates having a conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A.

[0096] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B.

[0097] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0098] Capacitance sensor 180M is used to generate a change in capacitance based on a touch operation performed by a finger on or near it. A touch control module (chip) in an electronic device can determine the location of the touch point by detecting the change in capacitance (or capacitance value) of the capacitance sensor. The touch control module (chip) can transmit this touch point information to the application processor, which controls the display screen 194 to provide visual output related to the touch operation.

[0099] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0100] FIG4 is a software structure block diagram of an electronic device 100 provided in an embodiment of the present application.

[0101] The layered architecture of electronic device 100 divides the software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into five layers: application layer, framework layer, Android runtime, hardware abstraction layer (HAL), system library, and kernel layer.

[0102] As shown in FIG4 , the application package may include applications such as a camera, a gallery, a calendar, and a call. It is understood that the applications included in the application layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer applications than the applications included in the application layer shown in FIG4 , or may include completely different applications.

[0103] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0104] As shown in FIG4 , the application framework layer may include a content provider, a view system, a resource manager, and the like.

[0105] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0106] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0107] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0108] The Hardware Abstraction Layer (HAL) is an interface layer between the operating system kernel and the hardware circuits. The HAL provides a virtual hardware platform for the operating system. Both the operating system kernel and the hardware driver can call the HAL. As shown in Figure 4, in this embodiment of the present application, the HAL includes a data processing module, a first model, and a parameter configuration module.

[0109] The data processing module is used to perform corresponding data processing operations on the received data, such as data extraction processing, data fusion processing, etc.

[0110] The first model is used to determine the protection state of the electronic device, which includes any one of: a film-applied but not cased state, a cased but not film-applied state, a film-applied and cased state, and a film-unapplied and not cased state.

[0111] The parameter configuration module is used to determine the sensitivity threshold of the capacitance sensor according to the protection state of the electronic device. The parameter configuration module pre-stores the corresponding relationship between the protection state of the electronic device and the sensitivity threshold of the capacitance sensor.

[0112] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0113] It is understood that the layers in the software structure shown in FIG4 and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not limit this.

[0114] Figure 5a is a schematic flow chart of a first model training method provided in an embodiment of the present application. This first model training method is applied to a server, which includes a preprocessing module and a training module. This first model is used to predict the film application status and casing status of an electronic device. As shown in Figure 5a, this first model training method includes steps S51 through S54.

[0115] Step S51: The pre-processing module obtains touch sample data of the first electronic device in various protection states.

[0116] The first electronic device is an electronic device used to collect touch sample data based on touch operations performed on the screen by a test subject (person). The first electronic device can be, for example, a mobile phone or tablet. The protection status indicates the film and / or case status of the first electronic device. The protection status includes: film-applied but not in a case, case-covered but not film-applied, film-applied and case-covered, and neither film-applied nor case-covered.

[0117] The touch sample data is used to indicate data such as capacitance information collected by the first electronic device during the process of the test subject touching the screen of the first electronic device. In some embodiments, the touch sample data also includes at least one of charging status information, touch pressure information, and posture information of the first electronic device. The charging status information is used to indicate whether the first electronic device is in a charging state, and the charging status information includes a charging state or an uncharged state. The touch pressure information is used to indicate the pressure value when the test subject touches the screen of the first electronic device. The posture information is used to indicate the posture of the first electronic device, and the posture is represented by data collected by a gyroscope sensor.

[0118] In one implementation scenario, the first electronic device establishes a connection with a server, where the connection may be a wired connection or a wireless connection. The server receives the touch sample data reported by the first electronic device through the connection.

[0119] FIG5b is a flow chart of a method for a server to obtain touch sample data provided by an embodiment of the present application. As shown in FIG5b, the server includes a pre-processing module; the first electronic device includes a capacitance sensor, a touch module, a gyroscope sensor, a charging management module, and a communication module. The touch module is connected to a plurality of capacitance sensors and can obtain the capacitance values ​​of the plurality of capacitance sensors; the touch module is also connected to a pressure sensor and can obtain the pressure value of the pressure sensor. In one embodiment, taking the electronic device in the first protection state as an example, the first protection state indicates any one of a film-applied but not cased state, a case-applied but not film-applied state, a film-applied and case-applied state, and a film-unapplied and not cased state, the pre-processing module in the server obtains the touch sample data of the electronic device in the first protection state (the above-mentioned step S51), which includes: steps S511 to S513.

[0120] Step S511: The pre-processing module receives first capacitance value information sent by the communication module in the electronic device.

[0121] In this embodiment of the present application, when a test subject (person) touches the screen of a first electronic device with their finger, the capacitance of a capacitance sensor in the electronic device changes. The first capacitance information indicates information acquired by the touch control module in the first electronic device during the test subject's touch operation on the first electronic device in the first protection state. The first capacitance information includes multiple frames of capacitance information, with each frame of capacitance information including the capacitance of the capacitance sensor in the first electronic device at the corresponding acquisition moment.

[0122] FIG6 is a schematic diagram of an electronic device obtaining capacitance information according to an embodiment of the present application. As shown in FIG6 , the electronic device 100 includes multiple capacitance sensors, which are arranged in rows and columns to form a capacitance sensor array 101. During a touch operation on the electronic device, as time t passes, the electronic device 100 can obtain capacitance information frame by frame according to a set touch sampling rate. The touch sampling rate is the frequency at which the touch module in the electronic device obtains capacitance information. The capacitance sampling rate can be set according to actual application conditions, for example, to 120 Hz, 240 Hz, or other values, which are not limited in the embodiments of the present application. Each time the electronic device obtains the capacitance of the multiple capacitance sensors, a frame of capacitance information can be formed. Each frame of capacitance information includes the capacitance of each capacitance sensor in the capacitance sensor array 108 at the corresponding acquisition time. The capacitance information obtained by the electronic device, for example, is shown in FIG6 as capacitance information 600 obtained at time t0, capacitance information 601 obtained at time t1, and capacitance information 602 obtained at time t2. In the embodiment of the present application, the acquired multi-frame capacitance information constitutes capacitance information. As shown in FIG6 , capacitance information 61 includes the above-mentioned capacitance information 600 , capacitance information 601 , and capacitance information 602 .

[0123] Taking a frame of capacitance information 601 collected at time t1 in FIG6 as an example, the distribution of capacitance values ​​within a frame of capacitance information is exemplified. Capacitance information 601 includes the capacitance values ​​of capacitance sensors in electronic device 100 at time t1. In capacitance information 601, the capacitance values ​​collected by some capacitance sensors near the contact point are shown in 601-1, where each square represents a capacitance sensor, and the number in each square represents the capacitance value of the capacitance sensor. For example, the capacitance value of sensor 601-10 is 995, and the capacitance value of sensor 601-11 is 178. The capacitance values ​​of some capacitance sensors farther from the contact point are shown in 601-2, for example, the capacitance value of capacitance sensor 601-20 is 2, and the capacitance value of sensor 601-21 is 1. Comparing capacitance value distribution 601-1 near the contact point with capacitance value distribution 601-2 farther from the contact point, it can be seen that the closer the capacitance sensor is to the contact point, the greater the capacitance value of the capacitance sensor.

[0124] As can be seen from the capacitance distributions shown in 601-1 and 601-2, in the capacitance distribution shown in 601-2, the capacitance values ​​of each capacitance sensor are all less than the sensor threshold (e.g., 120). These capacitance values ​​less than the sensor threshold are considered noise. In contrast, in the capacitance distribution shown in 601-1, the capacitance values ​​of each capacitance sensor are all greater than the sensor threshold (e.g., 120), effectively characterizing the contact between the stylus and the electronic device screen and can be considered valid information. It should be understood that the specific capacitance values ​​in Figure 6 are for illustrative purposes only.

[0125] In the embodiment of the present application, the first electronic device can obtain the first capacitance information by using the method for obtaining capacitance information provided in FIG. 6 .

[0126] In one embodiment, during the process of performing a touch operation on a first electronic device in a first protection state, the touch control module in the first electronic device sends the acquired first capacitance value information to the communication module, so that the communication module sends the first capacitance value information to the server, and the preprocessing module in the server obtains the first capacitance value information.

[0127] Step S512: The pre-processing module receives the first posture information sent by the communication module.

[0128] The first posture information indicates posture information collected by a gyroscope sensor in the first electronic device during a touch operation performed on the first electronic device in the first protection state. During a touch operation performed on the first electronic device with the film applied but not in the case, the gyroscope sensor in the first electronic device transmits the first posture information to the communication module, which transmits the first posture information to the server, and the pre-processing module in the server obtains the first posture information.

[0129] FIG7 is an example diagram of an electronic device posture provided by an embodiment of the present application. Referring to FIG7 , in one example, the posture information of the electronic device indicates the angle between the electronic device and the horizontal plane. As shown in FIG7 (1), the angle between the electronic device and the horizontal plane is 0°, and the electronic device is in a horizontal posture; as shown in FIG7 (2), the angle between the electronic device and the horizontal plane is 45°, and the electronic device is in an inclined posture; as shown in FIG7 (3), the angle between the electronic device and the horizontal plane is 90°, and the electronic device is in a vertical posture.

[0130] Taking a mobile phone as an example of an electronic device, a mobile phone is often in a horizontal, tilted, and vertical posture during use. Therefore, Figure 7 provides an exemplary illustration of the postures of the electronic device. However, it is understood that the posture information of the electronic device can also include more angle information. For example, the posture information can include information such as the heading angle (Yaw), pitch angle (Pitch), and roll angle (Roll) of the electronic device.

[0131] It should be noted that when a touch object touches the screen of an electronic device, the posture of the electronic device generally affects the capacitance value collected by the capacitance sensor. This is because the posture of the electronic device affects the contact area between the user's finger and the screen, which in turn affects the capacitance value collected by the capacitance sensor. For example, in general, when the electronic device is in a vertical posture, the contact area between the user's finger and the electronic device screen is smaller than when the electronic device is in a horizontal posture. When the touch duration is the same, a smaller contact area will result in a smaller increase in the capacitance value of the capacitance sensor.

[0132] Step S513: The pre-processing module receives the first charging status information sent by the communication module.

[0133] The first charging information indicates whether the first electronic device is charging during a touch operation performed on the first electronic device in the first protection state. During a touch operation performed on the first electronic device with the film applied but not in the case, the charging management module in the first electronic device transmits the first charging status information to the communication module, so that the communication module transmits the first charging status information to the server, and the preprocessing module in the server obtains the first charging status information.

[0134] In one example, during the process of performing a touch operation on the first electronic device without a cover, the first electronic device is charging, and the first charging status information indicates the charging status.

[0135] In another example, during the process of performing a touch operation on the first electronic device without a cover and the first electronic device is not being charged, the first charging state information indicates an uncharged state.

[0136] It's important to note that whether or not a user touches the screen of an electronic device while it's charging affects the capacitance of the capacitance sensor. This is because when a user touches the screen of an uncharged device, the capacitance value (also called the capacitance signal) of the capacitance sensor will contain a floating signal. However, when the device is charging, it's grounded, and when the user touches the screen, the capacitance value of the capacitance sensor will not contain a floating signal. A floating signal is relative to a grounded signal. When the electronic device is grounded, such as when it's charging, when the housing is directly in contact with a hand, or when the housing is directly grounded, the capacitance value collected by the capacitance sensor is a grounded signal. Conversely, a floating signal refers to the capacitance value collected when the housing is not grounded. The maximum capacitance value of a floating signal is smaller than the maximum capacitance value of a grounded signal.

[0137] Step S514: The pre-processing module receives the first touch pressure information sent by the communication module.

[0138] The first touch pressure information indicates the touch pressure applied when a touch operation is performed on the first electronic device in the first protection state. The touch pressure can be represented by a pressure value collected by a capacitance pressure sensor. During the touch operation on the first electronic device in the first protection state, a touch control module in the first electronic device collects the pressure value of the pressure sensor to obtain the first touch pressure information. The touch control module transmits the obtained first touch pressure information to the communication module, which transmits the first touch pressure information to the server. The preprocessing module in the server then obtains the first capacitance information.

[0139] It should be noted that steps S512, S513, and S514 are all optional. In some embodiments, the first electronic device may not collect posture information and charging status information, but may only collect capacitance information as touch sample data for subsequent processing. In other embodiments, the first electronic device may collect capacitance information and at least one of charging status information, touch pressure information, and posture information, all of which are collected as touch sample data for subsequent processing.

[0140] The embodiment shown in FIG5b above provides an exemplary description of the pre-processing module in the server obtaining touch sample data when the first electronic device is in the first protection state. It can be understood that when the first electronic device is in other protection states, the manner of obtaining the corresponding touch sample data is similar to that in the above embodiment and will not be repeated here. For example, when the first protection state is a film-applied but not shelled state, the other protection state may be a no-film-applied and no-shelled state, etc. It should be noted that in order to obtain touch sample data, the touch operation performed on the first electronic device in other protection states is exactly the same as the touch operation in the above embodiment, for example, the touch position (or trajectory) is the same and the touch force is the same.

[0141] After acquiring the touch sample data of the first electronic device in each protection state, the preprocessing module in the server obtains training feature data for training the first model based on the acquired touch sample data in each protection state. For detailed description, see the following step S52.

[0142] Step S52: The pre-processing module performs first data processing on the touch sample data to obtain training feature data.

[0143] Among them, the first data processing includes but is not limited to data extraction processing, data fusion processing, data labeling processing, etc.

[0144] The touch sample data is used to indicate data such as capacitance information collected by the first electronic device during the touch screen process. In some embodiments, the touch sample data also includes at least one of charging status information, posture information of the first electronic device, and touch pressure information.

[0145] Figure 5c is a flow chart of a method for obtaining training feature data provided in an embodiment of the present application. In one embodiment, taking touch sample data obtained when the first electronic device is in a first protection state as an example, the step (step S52) of the preprocessing module performing first data processing on the touch sample data to obtain training feature data is described as follows: the step includes the following steps S521-S522.

[0146] Step S521: extract first capacitance characteristic data according to the first capacitance information.

[0147] Among them, the first capacitance information indicates a set of multiple frames of capacitance information obtained by the electronic device during the process of performing a touch operation on the first electronic device in the first protection state (for example, with a film but no case), and each frame of capacitance information includes the capacitance of each capacitance sensor at the corresponding acquisition moment.

[0148] In an embodiment of the present application, when extracting the first capacitance characteristic data based on the first capacitance information, it is necessary to extract a capacitance matrix from the capacitance information contained in the first capacitance information. For each frame of capacitance information in the first capacitance information, the capacitance sensor position corresponding to the maximum capacitance is determined from the capacitance information, and the capacitance sensor position corresponding to the maximum capacitance in the capacitance information is the contact position. It should be noted that when the test subject touches the screen of the first electronic device, the capacitance of the capacitance sensor near the contact position changes significantly, which can be regarded as valid data and can effectively characterize the contact characteristics between the test subject and the screen of the first electronic device. In addition, the input data when training the first model is a matrix, and there are also restrictions on the length and width of the input data. Therefore, for each frame of capacitance information in the first capacitance information, according to the capacitance sensor position corresponding to the maximum capacitance in the capacitance information and the length and width requirements of the input data, the corresponding matrix is ​​extracted from the capacitance information.

[0149] Figure 8 is an example diagram of a preprocessing module provided by an embodiment of the present application to extract capacitance feature data. As shown in Figure 8, the first capacitance information 800 includes multiple frames of capacitance information, which are sorted according to the acquisition time. Taking the capacitance information 601 acquired at time t1 as an example, the input data of the first model is a matrix. When the length and width of the input data are required to be 7*7, the corresponding 7*7 matrix 601-3 is extracted from the capacitance information 601. The center of the matrix 601-3 contains the maximum capacitance value (the position of the contact) in the capacitance information, that is, the capacitance value 995 of the capacitance sensor 601-10.

[0150] In the embodiment of the present application, in order to improve the accuracy of the training data, it is necessary to obtain the corresponding 7*7 matrix from each frame of capacitance information contained in the first capacitance information 800 to form capacitance feature data.

[0151] In one embodiment, the first rule for selecting capacitance characteristic data is: from all the capacitance information contained in the first capacitance information, with the capacitance information containing the maximum capacitance as the center, select the center, the capacitance information of the preset frame (the preset frame is, for example, 5 frames) sorted before the center, and the capacitance information of the preset frame sorted after the center, and extract the corresponding matrix from each frame of the selected capacitance information.

[0152] As shown in FIG8 , taking capacitance information 601 containing a maximum capacitance value (the maximum capacitance value is 3995 of sensor 601-10) as an example, with capacitance information 601 as the center, capacitance information 601, the five frames of capacitance information that precede capacitance information 601 (capacitance information 607, 608, 609, 610, and 611), and the five frames of capacitance information that follow capacitance information 601 (capacitance information 602, 603, 604, 605, and 606) are selected, and the corresponding capacitance matrix is ​​extracted from each selected frame of capacitance information to form first capacitance feature data 801. The size of first capacitance feature data 801 is 11×7×7.

[0153] In another embodiment, when a touch operation is performed on the first electronic device, the capacitance value sensed by the capacitance sensor is unstable during the process of the finger approaching the screen of the electronic device and leaving the screen, and the data validity is poor. Therefore, it is necessary to filter out capacitance information with higher data stability from the initially selected capacitance information. In this embodiment, the second rule for selecting capacitance feature data is as follows: from all the capacitance information contained in the first capacitance information, with the capacitance information containing the maximum capacitance as the center, the capacitance information of the preset frame (the preset frame is, for example, 5 frames) sorted before the center is selected, and the capacitance information of the preset frame sorted after the center is selected. From the capacitance information of the preset frame sorted before the center, the capacitance information sorted in the first N frames (the N is, for example, 2) is discarded, and from the capacitance information of the preset frame sorted after the center, the capacitance information sorted in the last N frames is discarded; for the remaining (preset frame - 2N) capacitance information, the corresponding capacitance matrix is ​​extracted from each frame capacitance information to form the first capacitance feature data.

[0154] Taking Figure 8 as an example, with capacitance information 601 as the center, the capacitance information 601, the 5 frames of capacitance information sorted before the capacitance information 601 (capacitance information 607, 608, 609, 610, 611), and the 5 frames of capacitance information sorted after the capacitance information 601 (capacitance information 602, 603, 604, 605, 606) are selected. Then, from the 5 frames of capacitance information sorted before the capacitance information 601, the capacitance information sorted in the first 2 frames (capacitance information 610 and 611) are discarded; and, from the 5 frames of capacitance information sorted after the capacitance information 601, the capacitance information sorted in the last 2 frames (capacitance information 605 and 606) are discarded. For the remaining 7 frames of capacitance information, the corresponding capacitance matrix is ​​extracted from each frame of capacitance information, and together they constitute the first capacitance feature data. The size of the first capacitance feature data is 7×7×7.

[0155] It should be noted that when the contact time between the test object and the screen of the first electronic device is less than the preset duration, the first electronic device cannot obtain enough frames of capacitance information, that is, the capacitance information contained in the first capacitance information will be less than M frames. When the capacitance information obtained by the electronic device is less than M frames, the stable capacitance information contained in the capacitance information less than M frames (the capacitance information remaining after removing the first N frames and the last N frames) is not sufficient to support the first model training. The preset duration can be determined according to the touch sampling rate of the electronic device, such as 0.2s or 0.3s, which is not specifically limited in this embodiment. The value of M can be set to 2*preset frame + 1 frame.

[0156] For example, it is explained by taking the example that at least 7 frames of stable capacitance information are required to support the training of the first model. When the first capacitance information contains 11 frames of capacitance information, the stable capacitance information (for example, the capacitance information remaining after removing the first 2 frames and the last 2 frames) is 7 frames. When the first capacitance information only contains 7 frames of capacitance information, the stable capacitance information (for example, the capacitance information remaining after removing the first 2 frames and the last 2 frames) is only 3 frames, which cannot support the training of the first model. Therefore, in the above-mentioned second rule, it is necessary to preliminarily select (2*preset frames + 1 frame) capacitance information from the first capacitance information, and then discard the capacitance information sorted in the last N frames after the first N frames from the (2*preset frames + 1 frame) capacitance information. If it is not possible to preliminarily select (2*preset frames + 1 frame) capacitance information from the first capacitance information, the first capacitance information can be discarded.

[0157] Step S522: Mark the first capacitance characteristic data according to the first protection state of the electronic device to obtain first training characteristic data.

[0158] Among them, annotation is the process of labeling data.

[0159] In an embodiment of the present application, the first capacitance characteristic data is data obtained when the first electronic device is in a first protection state. Therefore, the preprocessing module labels the first capacitance characteristic data including: adding a label of the first protection state to the first capacitance characteristic data to obtain first training characteristic data.

[0160] In this embodiment, first training feature data is obtained through steps S521 and S522. This first training feature data is feature data related to the first electronic device in the first protection state. It will be appreciated that similar operations as steps S521 and S522 can be performed on touch sample data in other protection states to obtain training feature data for the corresponding protection states, providing a data foundation for the training process of the first model.

[0161] For example, when the first protection state is the film-applied, uncased state, the other protection state may be the film-applied and uncased state. Based on the touch sample data obtained in the film-applied and uncased state, operations similar to steps S521 and S522 above are performed to obtain second training feature data for the film-applied and uncased state. In this embodiment of the present application, the touch sample data obtained in each protection state is subjected to first data processing, and the obtained training feature data includes training feature data corresponding to each protection state, for example, the first training feature data and the second training feature data described above.

[0162] Figure 5d is a flow chart of another method for obtaining training feature data provided in an embodiment of the present application. In one embodiment, the touch sample data also includes auxiliary parameters. Taking the touch sample data obtained when the first electronic device is in the first protection state as an example, the step (step S52) in which the preprocessing module performs first data processing on the touch sample data to obtain training feature data is described below: it includes the following steps S523-S525.

[0163] Step S523: extract first capacitance characteristic data according to the first capacitance information.

[0164] Here, step S523 is the same as the aforementioned step S521. Please refer to the aforementioned detailed description of step S521 and will not be repeated here.

[0165] Step S524: performing data fusion processing on the first capacitance characteristic data and the auxiliary parameters to obtain first fused characteristic data.

[0166] Among them, the auxiliary parameters include: one or more of: first charging status information, first posture information and first touch pressure information. The first charging status information is used to indicate whether the first electronic device is charging when the tester performs a touch operation on the first electronic device in the first protection state, and the first charging status information includes an uncharged state or a charged state. The first posture information is used to indicate the posture of the first electronic device when the tester performs a touch operation on the first electronic device in the first protection state, and can be represented by data collected by the gyroscope sensor. The first touch pressure information is used to indicate the touch pressure when the tester performs a touch operation on the first electronic device in the first protection state, and can be represented by a pressure value.

[0167] Data fusion processing refers to the multi-level process of processing the association of data and information from single and multiple sources. Data fusion processing methods include, but are not limited to, projection, splicing, addition, etc. In actual applications, the specific data fusion processing method can be selected based on the specific situation and is not limited in the embodiments of this application.

[0168] It should be noted that the auxiliary parameter indicates a parameter that affects the capacitance of the capacitance sensor when the test object touches the screen of the electronic device. Therefore, incorporating this auxiliary parameter into the training process of the first model can improve the accuracy of the trained first model.

[0169] Step S525: Label the first fused feature data according to the first protection state of the electronic device to obtain first training feature data.

[0170] Among them, annotation is the process of labeling data.

[0171] In the embodiment of the present application, the pre-processing module labels the first fused feature data including: adding a label indicating that the first fused feature data is not covered with a film to obtain first training feature data.

[0172] In this embodiment, first training feature data is obtained through steps S521-S523. This first training feature data is feature data related to the first electronic device in the first protection state. It will be appreciated that similar operations as steps S521-S523 can be performed on touch sample data in other protection states to obtain training feature data for the corresponding protection states, providing a data foundation for the training process of the first model.

[0173] In this embodiment of the present application, the protection status labels on the training feature data can be used as true results to measure the accuracy of the first model's output. During subsequent training of the first model, the training feature data is input into the first model to obtain the first model's output. Based on this output and the protection status of the input training feature data, the error between the output and the true result can be determined.

[0174] Step S53: The preprocessing module sends the training feature data to the training module.

[0175] The training feature data includes protection status labels.

[0176] Step S54: The training module trains the initial first model according to the training feature data to obtain the trained first model.

[0177] Figure 9 is a schematic diagram of a model training provided by an embodiment of the present application. As shown in Figure 9, the training module uses the training feature data as input data, inputs it into the initial first model 900, and obtains the corresponding output result. Among them, the training feature data is a plurality of groups, and the plurality of groups of training feature data are different. For example, one group of training feature data may be feature data labeled as film-applied but not in a case, another group of training feature data may be feature data labeled as in a case but not in a film, and another group of training feature data may be feature data labeled as film-applied and in a case, etc. The output result is used to indicate the predicted protection state of the electronic device. The weight of the first model 900 is updated according to the output result and the loss function, and finally a weight parameter that meets the requirements is obtained. The requirement that the weight parameter needs to meet is that when the first model predicts the protection state of the electronic device based on the weight parameter, the error between the predicted protection state (output result) and the protection state indicated by the input data label (actual result) is less than a preset error, and the preset error can be determined according to the actual application situation. The weight parameter that meets the requirements is the weight of the first model after training, wherein the loss function is, for example, a cross entropy loss function.

[0178] In an embodiment of the present application, the first model can be a network model based on deep learning, such as ResNet18 (a residual network containing 18 convolutional layers) in Convolutional Neural Networks (CNN), which is not specifically limited in this embodiment.

[0179] In the embodiment of the present application, a trained first model is obtained through the above steps S51 to S54. The trained first model can determine the protection status of the electronic device based on the touch data generated by the electronic device in response to the touch operation.

[0180] It should be noted that the trained first model can be deployed in a second electronic device. The first model deployed in the second electronic device can determine the protection status of the second electronic device based on the touch data generated by the second electronic device in response to a touch operation. For detailed instructions on deploying the trained first model in the second electronic device, please refer to the first model deployment method provided in Figure 10 below.

[0181] Figure 10 is a flowchart of a first model deployment method provided in an embodiment of the present application. The first model deployment method is applied to a server or to a device capable of terminal deployment. As shown in Figure 10, the first model deployment method includes: Step S1001-Step S1002.

[0182] Step S1001: convert the trained first model into a preset format.

[0183] Among them, the preset format conversion can be the conversion from pyTorch (an open source neural network framework) to NCNN (a deployment framework for deploying models to mobile devices).

[0184] In an embodiment of the present application, during the training of the first model, the framework used by the first model may be pyTorch. PyTorch is a large machine learning framework that requires a large amount of storage space. Therefore, a lightweight framework, such as NCNN, is required when deploying the model. In this embodiment, after the first model training is completed, the first model is converted from pyTorch to NCNN to facilitate subsequent deployment of the first model.

[0185] Step S1002: deploying the first model and other collaborative modules converted into a preset format on a hardware abstraction layer of a second electronic device.

[0186] Among them, as shown in Figure 4, the other collaborative modules are such as a data processing module and a parameter configuration module. The data processing module has the same function as the preprocessing module of the server in the aforementioned embodiment, and is used to process the received touch sample data into input data that can be input into the first model; the parameter configuration module is used to match the corresponding sensitivity threshold parameters for the capacitance sensor according to the output data of the first model. The parameter configuration module can be described in detail in the embodiments described later and will not be described here.

[0187] In an embodiment of the present application, the trained first model is deployed on the second electronic device, so that the second electronic device can automatically sense the film and case status of the electronic device through the first model.

[0188] In an embodiment of the present application, after the first model is deployed on the second electronic device, the second electronic device can automatically sense its own protection state through the first model, and thus adaptively adjust the sensitivity threshold of the capacitance sensor according to its own protection state, so as to effectively sense the user's touch operation in each protection state and avoid disconnection, false touch, and other phenomena caused by changes in protection state. A detailed description of how the second electronic device automatically senses its own protection state according to the first model after deploying the first model and then adaptively adjusts the sensitivity threshold of the capacitance sensor can be found in the capacitance sensor sensitivity threshold determination method provided in FIG. 11 below.

[0189] Figure 11 is a flow chart illustrating a method for determining the sensitivity threshold of a capacitance sensor provided in an embodiment of the present application. This method is applied to a second electronic device capable of automatically sensing its own protection state. The hardware abstraction layer of this second electronic device includes a data processing module, a first model, and a parameter configuration module. This second electronic device can be a touchscreen device such as a tablet, mobile phone, or laptop. As shown in Figure 11, this sensor sensitivity threshold determination method includes steps S111 through S115.

[0190] S111 . During the process of the sensing object contacting the screen of the second electronic device, the data processing module obtains target touch data.

[0191] The sensing object may be a user, and the sensing object is in contact with the screen of the second electronic device, for example, the user touches the screen of the second electronic device with a finger. Target touch data is used to indicate data such as capacitance information collected by the second electronic device during the user's contact with the screen of the second electronic device. In some embodiments, the target touch data also includes one or more of charging status information, touch pressure information, and posture information of the second electronic device.

[0192] In one example, the second electronic device is in a non-film-attached state, and the user's contact with the screen of the second electronic device refers to contact with the screen of the electronic device itself.

[0193] In another example, the second electronic device is in a film-applied state, and the user's contact with the screen of the second electronic device refers to contact with the protective film affixed to the outer screen of the second electronic device.

[0194] In one embodiment, when a user touches the screen of a second electronic device, a data processing module obtains target touch data, including the following step 1.

[0195] Step 1: When the user is in contact with the screen of the second electronic device, the data processing module obtains target capacitance information.

[0196] The target capacitance information indicates the capacitance of each capacitance sensor in the second electronic device.

[0197] As shown in Figure 1, a second electronic device is configured with multiple capacitance sensors distributed in an array. These capacitance sensors are normally open. When a sensing object performs a touch operation on the screen of the second electronic device, the second electronic device obtains the capacitance values ​​of the multiple capacitance sensors at a predetermined touch sampling rate. A touch module in the second electronic device is connected to the multiple capacitance sensors. The capacitance sampling process can be performed by the touch module (or touch chip). The data processing module obtains capacitance information from the touch module, or the touch module sends the capacitance information to the data processing module.

[0198] When the sensing object contacts the screen of the second electronic device, target capacitance information is acquired based on a preset touch sampling rate. Multiple frames of target capacitance information, sorted by acquisition time, form target capacitance information. The target capacitance information includes the capacitance of each capacitance sensor in the second electronic device at the corresponding acquisition time.

[0199] The process of the second electronic device acquiring target capacitance information can be found in the detailed description of the electronic device acquiring capacitance information provided in FIG6 , and will not be repeated here. The target capacitance information includes multiple frames of target capacitance information acquired based on a preset touch sampling rate during a touch operation. The multiple frames of target capacitance information are sorted by acquisition time, and each frame of target capacitance information includes the capacitance value of each capacitance sensor in the second electronic device at the corresponding acquisition time.

[0200] In another embodiment, when the user touches the screen of the second electronic device, the data processing module obtains target touch data, which includes not only the above step 1 but also one or more of the following steps 2 to 4.

[0201] Step 2: The data processing module receives the target charging status information sent by the charging management module.

[0202] The target charging status information is used to indicate whether the second electronic device is charging when the user touches the screen of the second electronic device. For example, if the second electronic device is charging when the user touches the screen of the second electronic device, the target charging status information indicates the charging state; if the second electronic device is not charging when the user touches the screen of the second electronic device, the target charging status information indicates the uncharging state.

[0203] Step 3: The data processing module receives the target attitude information sent by the gyroscope sensor.

[0204] The target posture information is used to indicate the posture of the second electronic device when the user touches the screen of the second electronic device. The posture is described in detail in FIG. 7 and will not be repeated here.

[0205] Step 4: The data processing module receives the touch pressure information sent by the pressure sensor.

[0206] The touch pressure information is used to indicate the pressure value of the pressure sensor in the second electronic device when the user touches the screen of the second electronic device.

[0207] In this embodiment, when the user touches the screen of the second electronic device, the data processing module obtains corresponding data from the capacitance sensor, pressure sensor, gyroscope sensor and charging management module respectively, forming target touch data corresponding to the screen contact, providing a data basis for subsequent analysis.

[0208] S112: The data processing module performs second data processing on the target touch data to obtain touch feature data.

[0209] The second data processing includes but is not limited to data extraction processing, data fusion processing, etc.

[0210] In one embodiment, the data processing module performs second data processing on the target touch data to obtain touch feature data, including: step S1121.

[0211] Step S1121: Extract target capacity characteristic data according to target capacity information.

[0212] The method by which the data processing module extracts target capacitance characteristic data based on the target capacitance information is the same as the method by which the preprocessing module extracts capacitance characteristic data based on the first capacitance information in step S521 and FIG8 . Therefore, the method by which the data processing module extracts target capacitance characteristic data based on the target capacitance information can be found in the detailed description of step S521 and FIG8 in the aforementioned embodiment and will not be further described here.

[0213] In this embodiment, when the target touch data only includes target capacitance information, the extracted target capacitance feature data is the touch feature data.

[0214] In another embodiment, the target touch data further includes target auxiliary data. The data processing module performs a second data processing on the target touch data to obtain touch feature data, which includes the following step S1122 in addition to the above step S1121.

[0215] Step S1122: performing data fusion processing on the target capacitance characteristic data and the target auxiliary data to obtain touch characteristic data.

[0216] Among them, the target auxiliary data includes one or more of target charging status information, target posture information and target touch pressure information. The target charging status information indicates whether the second electronic device is charging when the sensing object is in contact with the screen of the second electronic device, and the target charging status information includes a charging state or an uncharged state. The target posture information indicates the posture of the second electronic device when the sensing object is in contact with the screen of the second electronic device, and the posture can be represented by data collected by a gyroscope sensor. The target touch pressure information indicates the contact pressure when the sensing object is in contact with the screen of the second electronic device, and the target touch pressure information can be represented by a pressure value collected by a pressure sensor in the second electronic device.

[0217] The data fusion processing methods include but are not limited to projection, splicing, addition, etc. In actual application, the specific data fusion processing method can be selected according to the specific situation and is not limited in the embodiments of this application.

[0218] In one example, when the target touch data also includes target touch pressure information, the data processing module can perform data fusion processing on the target capacitance characteristic data, the target touch pressure information, and the target charging state information to obtain touch characteristic data. It should be noted that the target touch pressure information can represent the pressure value when the user touches the screen of the electronic device. When the pressure value is greater, the distance between the user's finger and the capacitance sensor in the electronic device is closer, and the capacitance of the capacitance sensor will change accordingly. Therefore, using the target touch pressure information as one of the target contact data can improve the accuracy of the protection state of the electronic device determined by the subsequent first model.

[0219] In another example, when the target touch data also includes target posture information, the data processing module can perform data fusion processing on the target capacitance characteristic data, the target posture information, and the target charging state information to obtain touch characteristic data. It should be noted that different posture information of the electronic device will affect the capacitance of the capacitance sensor when the user touches the electronic device. Therefore, using this target posture information as a type of target touch data can improve the accuracy of the protection state of the electronic device determined by the subsequent first model.

[0220] In this embodiment, the data processing module performs second data processing on the target touch data to obtain input data (touch feature data) that can be input into the first model.

[0221] S113: The data processing module inputs the touch feature data into the first model.

[0222] The first model is pre-configured in the second electronic device and is used to sense the protection state of the second electronic device, which is specifically the state of the film and the case.

[0223] S114: The first model outputs protection status information to the parameter configuration module based on the input touch feature data.

[0224] The protection status information is used to indicate the current protection status of the second electronic device, which includes any one of the following: film-applied but not cased state, cased but not film-applied state, film-applied and cased state, and neither film-applied nor cased state.

[0225] S115 . The parameter configuration module determines a sensitivity threshold of the capacitance sensor according to the protection status information.

[0226] It's important to note that when a sensor's input slowly increases from zero, its output remains at zero until it reaches a certain minimum value, at which point the output changes. This minimum value is the sensitivity threshold. Taking a capacitance sensor as an example, as a user's finger approaches and touches the screen of an electronic device, the capacitance of the capacitance sensor increases with the user's finger's proximity. Only when this capacitance increases to a value greater than the sensitivity threshold will the capacitance sensor output a capacitance signal, and the electronic device will then detect the user's touch based on this capacitance signal.

[0227] In the embodiment of the present application, a correspondence table between protection status information and sensitivity thresholds of capacitance sensors is pre-stored in the parameter configuration module. The parameter configuration module can determine the sensitivity threshold of the capacitance sensor corresponding to the protection status information from the correspondence table.

[0228] In one example, when the protection status information indicates that the film is not attached and the case is not covered, the sensitivity threshold of the capacitance sensor is determined to be the first threshold.

[0229] The first threshold is a sensitivity threshold of the capacitance sensor pre-stored in the parameter configuration module, corresponding to the protection state of the electronic device without a film and a case. The first threshold is, for example, the sensitivity threshold configured for the capacitance sensor when the electronic device leaves the factory.

[0230] In another example, when the protection status information indicates that the film is not in the case, the sensitivity threshold of the capacitance sensor is determined to be the second threshold.

[0231] The second threshold is a sensitivity threshold of the capacitance sensor pre-stored in the parameter configuration module and corresponding to the protective state of the electronic device with a film but no case. The second threshold is smaller than the first threshold.

[0232] It should be noted that, as shown in Figure 1, for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the film-applied state is lower than the capacitance amplitude sensed in the film-unapplied state. Therefore, when the electronic device is covered with a film, the sensitivity threshold of the capacitance sensor needs to be lowered accordingly so that for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the film-applied state can also be greater than the sensitivity threshold, thereby generating a sensing signal, allowing the electronic device to detect the user's touch operation based on the sensing signal.

[0233] In yet another example, when the protection status information indicates that the case is not in a film-attached state, the sensitivity threshold of the capacitance sensor is determined to be a third threshold.

[0234] The second threshold is a sensitivity threshold of the capacitance sensor pre-stored in the parameter configuration module and corresponding to the protection state of the electronic device case without film. The third threshold is smaller than the first threshold.

[0235] It should be noted that when a user holds an electronic device and it is not in the case, the electronic device is effectively grounded. In this case, when the user touches the screen of the electronic device, the capacitance signal of the capacitance sensor will not contain a floating signal. When a user holds an electronic device but it is in the case, the electronic device is effectively semi-grounded. In this case, when the user touches the screen of the electronic device, the capacitance signal of the capacitance sensor will contain a partial floating signal, causing the capacitance amplitude sensed by the capacitance sensor to decrease.

[0236] FIG12 is a schematic diagram showing the distribution of capacitance values ​​collected by a capacitance sensor when an electronic device is in a case state and an uncased state, provided by an embodiment of the present application. FIG12 illustrates an example of a user performing a touch operation on the same position of the electronic device 100 in both the uncased and cased states.

[0237] The electronic device 100 is equipped with a capacitance sensor array 101, which includes multiple capacitance sensors arranged in rows and columns, where each small square represents a capacitance sensor. It should be noted that the capacitance sensor array in FIG12 is merely illustrative, and the number of rows and columns of the capacitance sensor array, as well as the number of capacitance sensors contained in each row or column, can be set according to actual application conditions and are not specifically limited in the embodiments of the present application.

[0238] As shown in (1) of FIG12 , in a scenario where the electronic device 100 is in an uncased state, a user touches the screen of the electronic device 100 with a finger. The capacitance of the capacitance sensor near the user's finger is shown in 1201-1, where each block is used to represent a capacitance sensor, and the number in the block is used to represent the size of the capacitance value detected by the capacitance sensor. For example, the capacitance value detected by capacitance sensor 1201-01 is 880. It should be noted that a capacitance value less than the sensitivity threshold of the capacitance sensor can be considered as a noise value. The sensitivity threshold of the capacitance sensor is, for example, 120.

[0239] As shown in (2) of FIG12 , in a scenario where the electronic device 100 is in a cased state, the electronic device 100 is covered with a protective case 105. The user touches the screen of the electronic device 100 with a finger, and the capacitance of the capacitance sensor near the user's finger is shown in 1201-2, where each block is used to represent a capacitance sensor, and the number in the block is used to represent the size of the capacitance value of the capacitance sensor. For example, the capacitance value of the capacitance sensor 1201-01 is 100.

[0240] Comparing the capacitance distribution shown in 1201-1 with the capacitance distribution shown in 1201-2, the capacitance of capacitance sensor 1201-01 when the electronic device 100 is not in the case is 880, while the capacitance value collected when it is in the case is 100. That is, for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the case state is lower than the capacitance amplitude sensed when it is not in the case. Therefore, when the electronic device is in the case, the sensitivity threshold of the capacitance sensor needs to be lowered accordingly so that for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the case state is also greater than the sensitivity threshold, thereby outputting a sensing signal, allowing the electronic device to perceive the user's touch operation based on this sensing signal.

[0241] In yet another example, when the protection status information indicates the film-applied and case-covered state, the sensitivity threshold of the capacitance sensor is determined to be the fourth threshold.

[0242] The fourth threshold is a sensitivity threshold of the capacitance sensor pre-stored in the parameter configuration module and corresponding to the protective state of the electronic device with a film and a case. The fourth threshold is smaller than the first, second, and third thresholds.

[0243] In an embodiment of the present application, when a user touches the screen of an electronic device, the second electronic device obtains target touch data and determines the protection status of the second electronic device based on the target touch data. That is, the second electronic device can automatically sense whether the electronic device is in a film or a case, and then determine the sensitivity threshold of the capacitance sensor based on the protection status of the second electronic device, so that the capacitance sensor can operate based on the determined sensitivity threshold. Automatic adjustment of the sensitivity threshold of the capacitance sensor is achieved, which can effectively solve the problems of disconnected touch, false touch, etc. that occur after the second electronic device is in a film or a case, thereby improving the user's touch experience when using the second electronic device.

[0244] The embodiments of the present application also provide an exemplary description of an application scenario. In the application scenario provided in the embodiments of the present application, a server, a first electronic device, and a second electronic device are included. The server is connected to the first electronic device, and the connection can be a wired connection or a wireless connection. Data is transmitted between the server and the first electronic device via the connection.

[0245] In this application scenario, the server is used to train a first model, which is used to sense the protection state of the electronic device. The first electronic device is used to collect touch sample data for training the first model. The second electronic device is used to deploy the trained first model.

[0246] The tester performs touch operations on the screen of the first electronic device in each protection state respectively. The first electronic device obtains touch sample data of itself (the first electronic device) in each protection state based on the touch operation, and sends the touch sample data in each protection state to the server through a pre-established connection.

[0247] The server receives touch sample data sent by the first electronic device in each protection state and trains an initial first model based on the touch sample data in each protection state to obtain a trained first model. The trained first model can determine the protection state of the electronic device based on the touch data collected by the electronic device when a user touches the screen of the electronic device.

[0248] The device manufacturer can deploy the trained first model to the second electronic device, so that the second electronic device has the ability to automatically sense its own protection status.

[0249] When a user uses a second electronic device and performs a touch operation on the screen of the second electronic device, the second electronic device can obtain target touch data based on the user's touch operation, and input the processed target touch data into a pre-deployed first model to obtain protection status information output by the first model, thereby automatically sensing its own protection status. After automatically sensing its own protection status, the second electronic device can adjust the sensitivity threshold of the capacitance sensor in the second electronic device based on the sensed protection status, thereby automatically adapting the sensitivity threshold of the capacitance sensor to the protection status of the second electronic device. This can effectively solve problems such as disconnected touch and false touch that occur after the electronic device is covered with a protective film or case, and improve the user's touch experience when using the second electronic device.

[0250] It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules that perform the corresponding functions. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0251] The steps performed by the electronic device in the first model training method, the first model deployment method, and the capacitance sensor sensitivity threshold determination method provided in the above-mentioned embodiments of the present application can also be performed by a chip system included in the electronic device, wherein the chip system may include a processor and a Bluetooth chip. The chip system can be coupled to a memory so that when the chip system is running, it calls the computer program stored in the memory to implement the steps performed by the above-mentioned electronic device. The processor in the chip system can be an application processor or a processor other than an application processor.

[0252] This embodiment further provides a computer-readable medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0253] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0254] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the methods in the above-mentioned method embodiments.

[0255] Among them, the electronic device, computer-readable medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0256] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0257] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0258] The units described as separate components may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0259] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above contents is within the scope of the present application. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for making a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0260] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining the sensitivity threshold of a capacitance sensor, characterized in that, Applied to a second electronic device, the method includes: When a sensing object contacts the screen of the second electronic device, obtaining target touch data; the target touch data at least includes target capacitance value information; the target capacitance value information indicates the capacitance value of a capacitance sensor in the second electronic device; According to the target touch data, determining the protection state of the second electronic device; the protection state includes one of a state of not being film - pasted and not being cased, a state of being film - pasted and not being cased, a state of being cased and not being film - pasted, and a state of being film - pasted and being cased; Based on the protection state, determining a sensitivity threshold corresponding to the capacitance sensor.

2. The method according to claim 1, wherein The step of obtaining target touch data when the sensing object contacts the screen of the second electronic device includes: When the sensing object contacts the screen of the second electronic device, obtaining target capacitance information based on a preset touch sampling rate, and multiple frames of the target capacitance information constitute the target capacitance value information; wherein, the target capacitance information includes the capacitance value of the capacitance sensor in the second electronic device at the corresponding acquisition moment.

3. The method according to claim 1 or 2, characterized in that, The step of determining the protection state of the second electronic device according to the target touch data includes: According to the target touch data, determining touch characteristic data; Inputting the touch characteristic data into a first model to obtain the protection state output by the first model.

4. The method according to claim 3, characterized in that According to the target touch data, determining touch characteristic data includes: According to the target capacitance value information, determining target capacitance value characteristic data; Taking the target capacitance value characteristic data as the touch characteristic data; Or, The target touch data further includes target auxiliary data; performing data fusion processing on the target capacitance value characteristic data and the target auxiliary data to obtain touch characteristic data; Wherein, the target auxiliary data includes one or more of target charging state information, target attitude information, and target touch pressure information.

5. The method according to claim 4, characterized in that The step of determining target capacitance value characteristic data according to the target capacitance value information includes: From the target capacitance value information, selecting multiple frames of capacitance information according to a preset selection rule; Extracting a capacitance value matrix with a preset size from each selected frame of capacitance information and combining them into the capacitance value characteristic data.

6. The method according to claim 5, wherein The matrix center of the capacitance value matrix is the maximum capacitance value in the corresponding capacitance information.

7. The method according to claim 5, characterized in that, The preset selection rule includes: In the target capacitance value information, taking the target capacitance information where the maximum capacitance value is located as the center, and selecting the center, a preset number of frames of target capacitance information before the center in order, and a preset number of frames of target capacitance information after the center in order.

8. The method according to claim 7, characterized in that, The preset selection rule further includes: Discarding the first N frames of target capacitance information from the preset number of frames of target capacitance information before the center; where N frames is greater than 1 frame and less than the preset number of frames; Discarding the last N frames of target capacitance information from the preset number of frames of target capacitance information after the center.

9. The method according to claim 3, characterized in that Based on the protection state, determining a sensitivity threshold corresponding to the capacitance sensor includes: When the protection state indicates a state of not being film - pasted and not being cased, determining that the sensitivity threshold corresponding to the capacitance sensor is a first threshold; When the protection status indicates the non-cased state of the film, determine that the sensitivity threshold corresponding to the capacitance sensor is the second threshold; the second threshold is less than the first threshold; When the protection status indicates the non-filmed state of the case, determine that the sensitivity threshold corresponding to the capacitance sensor is the third threshold; the third threshold is less than the first threshold; When the protection status indicates the state of both film and case, determine that the sensitivity threshold corresponding to the capacitance sensor is the fourth threshold; the fourth threshold is less than the second threshold and the third threshold.

10. The method according to claim 1, wherein After determining the sensitivity threshold corresponding to the capacitance sensor based on the protection status, the method further includes: Adjust the sensitivity parameter of the capacitance sensor to the determined sensitivity threshold for the touch chip connected to the capacitance sensor to output a sensing signal based on the determined sensitivity threshold.

11. A model training method, characterized in that, Applied to a server, the server is connected to a first electronic device; the method includes: Obtain touch sample data of the first electronic device in each protection state; the protection state indicates the states of non-filmed and non-cased, filmed but non-cased, cased but non-filmed, and both filmed and cased; the touch sample data includes capacitance information and charging state information; Perform first data processing on the touch sample data to obtain training feature data; Train an initial first model according to the training feature data to obtain the trained first model; the trained first model is used to be configured in a second electronic device to determine the protection state of the second electronic device.

12. The method according to claim 11, wherein The obtaining touch sample data of the first electronic device in each protection state includes: Receive first capacitance information sent by the first electronic device; the first capacitance information indicates the capacitance of the capacitance sensor in the first electronic device during the process that a tester touches the screen of the first electronic device in a first protection state; the first protection state indicates any one of the states of non-filmed and non-cased, filmed but non-cased, cased but non-filmed, and both filmed and cased.

13. The method according to claim 11 or 12, characterized in that, The performing first data processing on the touch sample data to obtain training feature data includes: Extract first capacitance feature data according to the first capacitance information; Label the first capacitance feature data according to the first protection state of the first electronic device to obtain first training feature data; Or, The touch sample data further includes auxiliary parameters; perform data fusion processing on the first capacitance feature data and the auxiliary parameters to obtain first fusion feature data; wherein, the auxiliary parameters include one or more of first charging state information, first attitude information, and first touch pressure information; Label the first fusion feature data according to the first protection state of the first electronic device to obtain first training feature data.

14. The method according to claim 13, wherein The training the initial first model according to the training feature data to obtain the trained first model includes: Input the training feature data into the initial first model to obtain an output result; Update the weights of the initial first model based on the output result and the loss function to obtain the trained first model.

15. The method according to claim 14, wherein The first model refers to a convolutional neural network model; the loss function refers to a cross-entropy loss function.

16. The method according to claim 11, wherein After obtaining the trained first model, the method further includes: Perform a preset format conversion on the trained first model; Deploy the first model after the preset format conversion on a second electronic device.

17. An electronic device, characterized in that, The electronic device includes: One or more processors; A memory; And a computer program, where the computer program is stored on the memory, and when the computer program is executed by the one or more processors, the electronic device is caused to execute the method for determining the sensitivity threshold of the capacitance sensor according to any one of claims 1-10.

18. A server, characterized in that, The server includes: One or more processors; A memory; And a computer program, where the computer program is stored on the memory, and when the computer program is executed by the one or more processors, the electronic device is caused to execute the model training method according to any one of claims 11-16.

19. A computer storage medium, characterized in that, Includes computer instructions that, when running on an electronic device, cause the electronic device to execute the method for determining the sensitivity threshold of the capacitance sensor according to any one of claims 1-10.

20. A computer storage medium, characterized in that, Includes computer instructions that, when running on a server, cause the server to execute the model training method according to any one of claims 11-16.

Images