Method for determining sensitivity threshold value of capacitance value sensor and electronic equipment

Abstract

The invention provides a method for determining a sensitivity threshold value of a capacitance value sensor and electronic equipment. The method comprises the following steps: acquiring target touch data under the condition that a sensing object is in contact with a screen of second electronic equipment; the target touch data at least comprises target capacitance value information; the target capacitance value information indicates the capacitance value of a capacitance value sensor in the second electronic equipment; according to the target touch data, determining a protection state of the second electronic equipment so as to realize automatic sensing of the protection state of the second electronic equipment; and determining a sensitivity threshold value corresponding to the capacitance value sensor based on the determined protection state, so that the capacitance value sensor works based on the determined sensitivity threshold value, and automatic adaptation of the sensitivity threshold value of the capacitance value sensor and the protection state of the second electronic equipment is realized. The conditions of broken touch, mistaken touch and the like after the protection state of the second electronic equipment is changed can be effectively solved, and the touch experience of a user using the electronic equipment is improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a method for determining the sensitivity threshold of a capacitance sensor and an electronic device. Background Art

[0002] During the process of using an electronic device, in order to protect the electronic device, users often stick a protective film on the screen of the electronic device and put on a protective case. However, usually, the sensitivity threshold of the capacitance sensor in the electronic device is set according to the state where the electronic device has no film and no case. This sensitivity threshold is used to indicate the minimum input value that causes the sensor to produce an output, that is, only when the input value exceeds this sensitivity threshold, the capacitance sensor will produce an output. After the electronic device is stuck with a film or put on a protective case, the sensed value sensed by the capacitance sensor in the electronic device will deviate. For example, when a user touches the screen of the electronic device before and after sticking the film with the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the electronic device after sticking the film will be correspondingly lower than that before sticking the film, and the sensed capacitance amplitude after sticking the film may not reach the sensitivity threshold of the capacitance sensor, resulting in problems such as touch interruption and accidental touch when the user uses the electronic device. Summary of the Invention

[0003] To solve the above technical problems, this application provides a method for determining the sensitivity threshold of a capacitance sensor and an electronic device, so as to achieve automatic adaptation of the sensitivity threshold of the capacitance sensor to the protection state of the electronic device, solve problems such as touch interruption and accidental touch after the electronic device is stuck with a film or put on a case, and improve the touch experience when the user uses the electronic device.

[0004] In a first aspect, this application provides a method for determining the sensitivity threshold of a capacitance sensor, which is applied to a second electronic device. The method includes: obtaining target touch data when a sensing object is in contact with the screen of the second electronic device; the target touch data at least includes target capacitance information; the target capacitance information indicates the capacitance of the capacitance sensor in the second electronic device; determining the protection state of the second electronic device according to the target touch data; the protection state includes one of the states of no film and no case, film but no case, case but no film, and film and case; determining the sensitivity threshold corresponding to the capacitance sensor based on the protection state.

[0005] In an embodiment of the present application, the second electronic device is an electronic device that can automatically sense its own protection state. When a sensing object contacts the screen of the second electronic device, the second electronic device acquires target touch data, and based on the target touch data, determines the protection state of the second electronic device to realize the automatic sensing of its own protection state. The second electronic device determines the sensitivity threshold of the capacitance sensor according to the sensed protection state, so that the capacitance sensor can work based on the determined sensitivity threshold, realizing the automatic adaptation of the sensitivity threshold of the capacitance sensor to the protection state of the second electronic device, and effectively solving problems such as touch interruption and false touch that occur after the protection state of the second electronic device changes, and improving the touch experience of the user when using the electronic device.

[0006] According to the first aspect, when a sensing object contacts the screen of the second electronic device, acquiring target touch data includes: when a sensing object contacts the screen of the second electronic device, acquiring target capacitance information based on a preset touch sampling rate, and multiple frames of target capacitance information constitute target capacitance value information; wherein, the target capacitance information includes the capacitance value of the capacitance sensor in the second electronic device at the corresponding acquisition moment.

[0007] In an embodiment of the present application, the target capacitance value information in the target touch data includes multiple frames of target capacitance information, and the multiple frames of target capacitance information can effectively characterize the contact characteristics when the sensing object contacts the screen of the second electronic device, providing effective data support for subsequently sensing the protection state of the second electronic device according to the target capacitance value information.

[0008] According to the first aspect, or any implementation manner of the above first aspect, determining the protection state of the second electronic device according to the target capacitance value information includes: determining touch characteristic data according to the target touch data; inputting the touch characteristic data into a first model to obtain the protection state output by the first model.

[0009] In an embodiment of the present application, the second electronic device only needs to input the touch characteristic data into the first model to obtain the protection state output by the first model, simplifying the way for the second electronic device to determine its own protection state and improving the speed of the second electronic device to sense its own protection state.

[0010] According to the first aspect, or any implementation manner of the above first aspect, determining touch characteristic data according to target touch data includes: determining target capacitance characteristic data according to target capacitance information; using the target capacitance characteristic data as the touch characteristic data; or, the target touch data further includes target auxiliary data; performing data fusion processing on the target capacitance characteristic data and the target auxiliary data to obtain the touch characteristic data; wherein the target auxiliary data includes one or more of target charging state information, target attitude information, and target touch pressure information.

[0011] In the embodiments of the present application, the target auxiliary data are all data that can affect the capacitance of the capacitance sensor. Therefore, involving the target auxiliary data in the process of predicting the protection state of the second electronic device can reduce the error in predicting the protection state of the second electronic device and improve the accuracy of the determined protection state of the second device.

[0012] According to the first aspect, or any implementation manner of the above first aspect, determining target capacitance characteristic data according to target capacitance information includes: selecting multiple frames of capacitance information from the target capacitance information according to a preset selection rule; extracting a capacitance matrix with a preset size from each selected frame of capacitance information and combining them into capacitance characteristic data. In the embodiments of the present application, selecting multiple frames of capacitance information according to a preset selection rule and then extracting a capacitance matrix with a preset size from each frame of capacitance information can reduce data redundancy and extract effective data to combine into capacitance characteristic data.

[0013] According to the first aspect, or any implementation manner of the above first aspect, the center of the capacitance matrix is the maximum capacitance in the corresponding capacitance information.

[0014] In the embodiments of the present application, the maximum capacitance in the capacitance information corresponds to the contact point when the sensing object touches the screen of the second electronic device. Therefore, using the maximum capacitance in the capacitance information as the center of the capacitance matrix can make the extracted capacitance matrix include the capacitance of the capacitance sensor near the contact point and improve the effectiveness of the capacitance characteristic data.

[0015] According to the first aspect, or any implementation manner of the above first aspect, the preset selection rule includes: in the target capacitance information, taking the target capacitance information where the maximum capacitance is located as the center, and selecting a preset number of frames of target capacitance information before the center and a preset number of frames of target capacitance information after the center in terms of sorting.

[0016] In the embodiment of the present application, the target capacitance information where the maximum capacitance value is located can effectively characterize the contact feature between the sensing object and the second electronic device. Therefore, with the target capacitance information where the maximum capacitance value is located as the center, the center, the preset frame target capacitance information sorted before the center, and the preset frame target capacitance information sorted after the center are selected to improve the effectiveness of the capacitance feature data.

[0017] According to the first aspect, or any one of the implementation manners of the above first aspect, the preset selection rule further includes: discarding the target capacitance information sorted in the first N frames from the preset frame target capacitance information sorted before the center; where N frames is greater than 1 frame and less than the preset frame; discarding the target capacitance information sorted in the last N frames from the preset frame target capacitance information sorted after the center.

[0018] In the embodiment of the present application, when a touch operation is performed on the second electronic device, during the process of the finger of the sensing object approaching and leaving the screen of the second electronic device, the capacitance value sensed by the capacitance sensor is unstable. Therefore, it is necessary to further screen out the capacitance information with higher data stability from the initially selected capacitance information, so as to further improve the effectiveness of the capacitance feature data determined based on the screened capacitance information.

[0019] According to the first aspect, or any one of the implementation manners of the above first aspect, determining the sensitivity threshold corresponding to the capacitance sensor based on the protection state includes: when the protection state indicates the state of not being pasted with a film and not being covered with a case, determining the sensitivity threshold corresponding to the capacitance sensor as the first threshold; when the protection state indicates the state of being pasted with a film and not being covered with a case, determining the sensitivity threshold corresponding to the capacitance sensor as the second threshold; the second threshold is less than the first threshold; when the protection state indicates the state of being covered with a case and not being pasted with a film, determining the sensitivity threshold corresponding to the capacitance sensor as the third threshold; the third threshold is less than the first threshold; when the protection state indicates the state of being pasted with a film and covered with a case, determining the sensitivity threshold corresponding to the capacitance sensor as the fourth threshold; the fourth threshold is less than the second threshold and the third threshold.

[0020] In the embodiment of the present application, the second electronic device pre-stores the correspondence between the protection state and the sensitivity threshold of the capacitance sensor. Therefore, after the second electronic device senses its own protection state, it can automatically adapt the corresponding sensitivity threshold of the capacitance sensor according to the sensed protection state.

[0021] According to the first aspect, after determining the sensitivity threshold corresponding to the capacitance sensor based on the protection state, the method further includes: adjusting the sensitivity parameter of the capacitance sensor to the determined sensitivity threshold, so that the touch control chip connected to the capacitance sensor outputs a sensing signal based on the determined sensitivity threshold.

[0022] In the embodiment of the present application, the second electronic device adjusts the sensitivity parameter of the capacitance sensor to the determined sensitivity threshold, enabling the capacitance sensor to operate based on the determined sensitivity threshold, realizing the automatic adjustment of the sensitivity threshold of the capacitance sensor according to the protection state of the second electronic device, effectively solving problems such as touch disconnection and false touch that occur after the protection state of the second electronic device changes, and improving the touch experience of users when using the electronic device.

[0023] In a second aspect, the present application provides a model training method, which is characterized in that it is applied to a server, and the server is connected to a first electronic device; the method includes: obtaining touch sample data of the first electronic device in each protection state; the protection state indicates a state of not applying a film and not wearing a case, a state of applying a film and not wearing a case, a state of wearing a case and not applying a film, and a state of applying a film and wearing a case; the touch sample data includes capacitance information and charging state information; performing first data processing on the touch sample data to obtain training feature data; training an initial first model according to the training feature data to obtain a trained first model; the trained first model is used to be configured in the second electronic device to determine the protection state of the second electronic device.

[0024] In the embodiment of the present application, the first model is trained through the touch sample data of the first electronic device in each protection state, so that the trained first model has the function of automatically sensing the protection state of the electronic device and can determine the protection state of the corresponding electronic device according to the touch data.

[0025] According to the second aspect, obtaining the touch sample data of the first electronic device in each protection state includes: receiving the first capacitance information sent by the first electronic device; the first capacitance information indicates the capacitance of the capacitance sensor in the first electronic device during the process that a tester touches the screen of the first electronic device in the first protection state; the first protection state indicates any one of the state of not applying a film and not wearing a case, the state of applying a film and not wearing a case, the state of wearing a case and not applying a film, and the state of applying a film and wearing a case.

[0026] In the embodiment of the present application, the capacitance of the capacitance sensor can effectively represent the contact characteristics between the tester and the first electronic device. Therefore, using the capacitance of the capacitance sensor as the touch sample data can improve the effectiveness of the touch sample data.

[0027] According to a second aspect, or any implementation of the above second aspect, performing first data processing on touch sample data to obtain training feature data, including: extracting first capacitance feature data according to first capacitance information to extract valid information from the first capacitance information and reduce data redundancy; labeling the first capacitance feature data according to the first protection state of the first electronic device to obtain first training feature data; or, the touch sample data further includes auxiliary parameters; performing data fusion processing on the first capacitance feature data and the auxiliary parameters to obtain first fusion feature data; wherein, the auxiliary parameters include one or more of first charging state information, first attitude information, and first touch pressure information; labeling the fusion feature data according to the first protection state of the first electronic device to obtain first training feature data, and the protection state label on the training feature data can be used as a true result to measure the accuracy of the output of the first model.

[0028] In the embodiments of the present application, involving the auxiliary parameters in the training process of the first model can improve the accuracy of the trained first model. Labeling the first fusion feature data according to the first protection state of the first electronic device to obtain first training feature data, and the protection state label on the training feature data can be used as a true result to measure the accuracy of the output of the first model.

[0029] According to a second aspect, or any implementation of the above second aspect, training an initial first model according to the training feature data to obtain a trained first model, including: inputting the training feature data into the initial first model to obtain an output result; updating the weights of the initial first model based on the output result and a loss function, so that the error between the protection state predicted by the first model (output result) and the protection state indicated by the input data label (actual result) is less than a preset error, thereby obtaining a trained first model.

[0030] According to a second aspect, or any implementation of the above second aspect, the first model refers to a convolutional neural network model; the loss function refers to a cross-entropy loss function.

[0031] According to a second aspect, after obtaining the trained first model, the method further includes: performing a preset format conversion on the trained first model so that the trained first model is converted into a lightweight model that can be adapted to a mobile terminal; deploying the first model after the preset format conversion on a second electronic device, such that the second electronic device has the function of automatically sensing its own protection state. For example, when a user is using the second electronic device and during the process of performing a touch operation on the screen of the second electronic device, the second electronic device can obtain target touch data according to the user's touch operation, and input the target touch data after data processing into the pre-deployed first model to obtain the protection state information output by the first model.

[0032] In a third aspect, the present application provides an electronic device, which includes: one or more processors; a memory; and a computer program, where the computer program is stored in the memory, and when the computer program is executed by one or more processors, the electronic device is caused to execute the capacitance sensor sensitivity threshold determination method as described in the first aspect or any one of the implementation manners of the first aspect above.

[0033] The third aspect and any one of the implementation manners of the third aspect respectively correspond to the first aspect and any one of the implementation manners of the first aspect. The technical effects corresponding to the third aspect and any one of the implementation manners of the third aspect can be referred to the technical effects corresponding to the first aspect and any one of the implementation manners of the first aspect above, and will not be elaborated here.

[0034] In a fourth aspect, the present application provides a server, which includes: one or more processors; a memory; and a computer program, where the computer program is stored in the memory, and when the computer program is executed by one or more processors, the electronic device is caused to execute the model training method as described in the second aspect.

[0035] The technical effects corresponding to the fourth aspect can be referred to the technical effects corresponding to the second aspect above, and will not be elaborated here.

[0036] In a fifth aspect, the present application provides a computer storage medium, which includes computer instructions, and when the computer instructions run on an electronic device, the electronic device is caused to execute the capacitance sensor sensitivity threshold determination method as described in the first aspect or any one of the implementation manners of the first aspect above.

[0037] The fifth aspect corresponds to the first aspect and any one of the implementation manners of the first aspect. The technical effects corresponding to the fifth aspect and any one of the implementation manners of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any one of the implementation manners of the first aspect above, and will not be elaborated here.

[0038] In a sixth aspect, the present application provides a computer storage medium, including computer instructions, which, when running on an electronic device, cause the electronic device to execute the model training method as described in the second aspect.

[0039] For the technical effects corresponding to the sixth aspect, reference may be made to the technical effects corresponding to the second aspect above, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the distribution of capacitance values of a capacitance sensor collected when an electronic device provided in an embodiment of the present application is in a film-applied state and a non-film-applied state;

[0041] Figure 2 It is a schematic diagram of a scenario provided in an embodiment of the present application;

[0042] Figure 3 It is a schematic diagram of a scenario provided in an embodiment of the present application;

[0043] Figure 4 It is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0044] Figure 5a It is a schematic flowchart of a first model training method provided in an embodiment of the present application;

[0045] Figure 5b It is a schematic flowchart of a method for a server to obtain touch sample data provided in an embodiment of the present application;

[0046] Figure 5c It is a schematic flowchart of a method for obtaining training feature data provided in an embodiment of the present application;

[0047] Figure 5d It is a schematic flowchart of another method for obtaining training feature data provided in an embodiment of the present application;

[0048] Figure 6 It is a schematic diagram of an electronic device obtaining capacitance information provided in an embodiment of the present application;

[0049] Figure 7 It is an example diagram of the attitude of an electronic device provided in an embodiment of the present application;

[0050] Figure 8 It is an example diagram of a preprocessing module extracting capacitance feature data provided in an embodiment of the present application;

[0051] Figure 9 It is a schematic diagram of model training provided in an embodiment of the present application;

[0052] Figure 10Flowchart of a first model deployment method provided by an embodiment of this application;

[0053] Figure 11 Schematic flowchart of a method for determining the sensitivity threshold of a capacitance sensor provided by an embodiment of this application;

[0054] Figure 12 Schematic diagram of the distribution of capacitance values of a capacitance sensor collected when an electronic device is in a cased state and an uncased state provided by an embodiment of this application. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this application without making creative efforts shall fall within the protection scope of this application.

[0056] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone.

[0057] The terms "first", "second", etc. in the description and claims of the embodiments of this application are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first target object and the second target object are used to distinguish different target objects, rather than to describe the specific order of target objects.

[0058] In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0059] In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality" refers to two or more. For example, a plurality of processing units refers to two or more processing units; a plurality of systems refers to two or more systems.

[0060] Figure 1 Schematic diagram of the distribution of capacitance values of a capacitance sensor collected when an electronic device is in a film-applied state and a non-film-applied state provided by an embodiment of this application. Figure 2 Schematic diagram of a scenario provided by an embodiment of this application. Before introducing the embodiments of this application, first based on Figure 1 、Figure 2 Describe the application scenarios of the embodiments of the present application. Figure 1 and Figure 2 both take a mobile phone as an example of the electronic device 100. In the following embodiments, the unit of capacitance value is picofarad (pF).

[0061] In Figure 1 , taking the user's touch operations at the same position on the electronic device 100 in the non-film-attached state and the film-attached state of the electronic device 100 as examples. Among them, a capacitance sensor array 101 is arranged under the screen of the electronic device 100. The capacitance sensor array 101 includes a plurality of capacitance sensors arranged in rows and columns. Among them, each small square is used to indicate a capacitance sensor. It should be noted that Figure 1 The capacitance sensor array in is only for illustrative purposes. The number of rows and columns of the capacitance sensor array and the number of capacitance sensors included in each row or each column can be set according to the actual application situation, and no specific limitation is made in the embodiments of the present application.

[0062] Generally speaking, the process of the capacitance sensor sensing the finger is related to the distance between the finger and the screen of the electronic device 100. During the process of the finger continuously approaching the screen of the electronic device 100, the capacitance sensor closer to the finger is more likely to generate a capacitance change, and the capacitance of the capacitance sensor farther from the finger is less likely to generate a capacitance change.

[0063] As Figure 1 shown in (1) of , in the scenario where the electronic device 100 is in the non-film-attached state, the user touches the screen of the electronic device 100 with a finger. The capacitance values of some capacitance sensors near the user's finger are as shown in 101-1. Among them, each square is used to indicate a capacitance sensor, and the number in the square is used to represent the magnitude of the capacitance value of the capacitance sensor. For example, the capacitance value of the capacitance sensor 101-01 is 800. It should be noted that the capacitance value less than the sensitivity threshold of the capacitance sensor can be considered as a noise value, and the sensitivity threshold is, for example, 120.

[0064] As Figure 1 shown in (2) of , in the scenario where the electronic device 100 is in the film-attached state, a protective film 102 is attached to the electronic device 100. The user touches the screen of the electronic device 100 with a finger. The capacitance values of some capacitance sensors near the user's finger are as shown in 101-2. Among them, each square is used to indicate a capacitance sensor, and the number in the square is used to represent the capacitance value of the capacitance sensor. For example, the capacitance value detected by the capacitance sensor 101-01 is 100.

[0065] Comparing the capacitance value distributions shown in 101-1 and 101-2, the capacitance value collected by the capacitance sensor 101-01 when the electronic device 100 is in the non-film-attached state is 800, and the capacitance value collected in the film-attached state is 100. That is, for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the film-attached state is lower than that sensed in the non-film-attached state. Moreover, in the film-attached state, the capacitance value sensed by the capacitance sensor may be lower than the sensitivity threshold.

[0066] See Figure 2 the side view of the electronic device 100 shown in (1) in which a protective film 102 is attached to the electronic device 100. In Figure 2 the first display interface 103 of the electronic device 100 shown in (2) in, there are icons of multiple application programs, such as icons of application programs like clock, calendar, gallery, memo, camera, address book, message, and phone. In some embodiments, when the user clicks on the phone icon 1031 in the first display interface 103, the electronic device 100 should originally respond to this click operation and display the second display interface 104 shown in (3) of Figure 2 . However, when the protective film 102 is attached to the electronic device 100, as shown in Figure 1 , the capacitance amplitude sensed by the capacitance sensor in the electronic device 100 will decrease. In some cases, the capacitance value of the capacitance sensor may not reach the sensitivity threshold of the capacitance sensor, resulting in the touch control module connected to the capacitance sensor regarding this capacitance value as noise, and the touch control module will not generate an output. The electronic device 100 cannot sense this click operation, let alone respond to this click operation and display the second display interface 103 shown in (3) of Figure 2 .

[0067] Therefore, the embodiment of the present application provides a method for determining the sensitivity threshold of a sensor. When the user touches the screen of the electronic device, the electronic device acquires target touch data and determines the protection state of the electronic device based on the target touch data, that is, the electronic device can automatically sense the film-attached and case-covered situations of the electronic device. Then, based on the protection state of the electronic device, the sensitivity threshold of the capacitance sensor is determined for the capacitance sensor to work based on the determined sensitivity threshold, realizing the automatic adjustment of the sensitivity threshold of the capacitance sensor, which can effectively improve the situations of touch interruption and false touch after the electronic device is in the film-attached or case-covered state and improve the touch experience of the user when using the electronic device.

[0068] The method for determining the sensor sensitivity threshold provided by the embodiments of the present application can be applied to an electronic device, which can be a wearable electronic device (such as a watch), a portable computer (such as a mobile phone), a tablet computer, a laptop computer, a personal computer (PC), an augmented reality (AR) / virtual reality (VR) device, an in-vehicle computer, or other devices. The following embodiments do not impose special restrictions on the specific form of the electronic device.

[0069] Before describing the technical solutions of the embodiments of the present application, the electronic device of the embodiments of the present application will be described first with reference to the accompanying drawings. Figure 3 FIG. 5 is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application. It should be understood that Figure 3 the illustrated electronic device 100 is only an example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 3 The various components shown in FIG. 5 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0070] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0071] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0072] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0073] In the embodiments of the present application, when the user touches the screen of the electronic device, the controller may control the processor to obtain the capacitance value of the capacitance sensor and obtain the charging status information from the charging management module. In some embodiments, when the user touches the screen of the electronic device, the controller may also control the processor to obtain the attitude information collected by the gyroscope sensor and obtain the touch pressure information collected by the pressure sensor.

[0074] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0075] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, a pulse code modulation (PCM) interface, a general-purpose input / output (GPIO) interface, and / or a universal serial bus (USB) interface, etc.

[0076] The I2C interface is a two-way synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, charger, flash, camera 193, etc. through different I2C bus interfaces. For example, the processor 110 can be coupled to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface to implement the touch function of the electronic device 100.

[0077] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

[0078] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0079] The USB interface 130 is an interface that conforms to the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transmission between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as AR devices, etc.

[0080] In the embodiments of this application, when the server performs the first model training, the server can achieve a wired connection with the first electronic device through the above GPIO interface or USB interface. The server and the first electronic device can achieve data transmission through this wired connection.

[0081] The charging management module 140 is configured to receive a charging input from a charger. The charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive the charging input from the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141. In the embodiments of the present application, the charging management module 140 is further configured to transmit the charging status information of the electronic device to the processor 110, and the charging status information is, for example, not charging or charging.

[0082] The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, the wireless communication module 160, etc.

[0083] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0084] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0085] The mobile communication module 150 may provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The wireless communication module 160 may provide solutions for wireless communications such as wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the electronic device 100.

[0086] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technologies. In the embodiments of the present application, when the server performs the first model training, the server can achieve a wired connection with the first electronic device through a wireless connection. The server and the first electronic device can achieve data transmission through this wireless connection.

[0087] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0088] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0089] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0090] The digital signal processor is used to process digital signals. In addition to being able to process digital image signals, it can also process other digital signals. For example, in the embodiments of the present application, after a first model is configured in the electronic device, when the user touches the electronic device, the digital signal processor can be used to process the touch data obtained by the electronic device according to the touch operation, such as capacitance value information, charging state information, etc.

[0091] The NPU is a neural-network (NN) computing processor. By learning from the biological neural network structure, such as learning from the transmission mode between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0092] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0093] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0094] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.

[0095] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 180B.

[0096] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 together form a touch screen, also known as the "touch control screen". The touch sensor 180K is used to detect touch operations acting on it or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a different position from the display screen 194.

[0097] The capacitance sensor 180M is used to generate a capacitance change according to the touch operation of a finger acting on it or nearby. The touch control module (chip) in the electronic device can determine the position of the contact point by detecting the capacitance change situation (or capacitance value) of the capacitance sensor. The touch control module (chip) can transmit the contact point information to the application processor, and the application processor controls the display screen 194 to provide visual output related to the touch operation.

[0098] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, taking the Android system with a layered architecture as an example, the software structure of the electronic device 100 is exemplarily described.

[0099] Figure 4 This is a software structure block diagram of an electronic device 100 provided in the embodiments of the present application.

[0100] The layered architecture of the electronic device 100 divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom are the application layer, the framework layer, the Android runtime, the Hardware Abstraction Layer (HAL), the system library, and the kernel layer.

[0101] As Figure 4 shown, the application package can include applications such as a camera, a gallery, a calendar, a call, etc. It can be understood that the applications included in the application layer do not constitute a specific limitation on the electronic device 100. In some other embodiments of the present application, compared with Figure 4The applications included in the application layer shown may vary. The electronic device 100 may include more or fewer applications, or may even include completely different applications.

[0102] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0103] As Figure 4 shown, the application framework layer may include a content provider, a view system, a resource manager, etc.

[0104] The content provider is used to store and obtain data, and make this data accessible to applications. The data may include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc.

[0105] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

[0106] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0107] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and the hardware circuit. The hardware abstraction layer provides a virtual hardware platform for the operating system. Either the operating system core or the hardware driver can call the hardware abstraction layer. As Figure 4 shown, in the embodiments of the present application, the hardware abstraction layer includes a data processing module, a first model, and a parameter configuration module.

[0108] The data processing module is used to perform corresponding data processing operations on the received data, such as data extraction processing, data fusion processing, etc.

[0109] The first model is used to determine the protection state of the electronic device. The protection state includes any one of the following: the state of having a film but no case, the state of having a case but no film, the state of having both a film and a case, and the state of having neither a film nor a case.

[0110] The parameter configuration module is used to determine the sensitivity threshold of the capacitance sensor according to the protection state of the electronic device. The corresponding relationship between the protection state of the electronic device and the sensitivity threshold of the capacitance sensor is pre-stored in the parameter configuration module.

[0111] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0112] It can be understood that Figure 4 the layers in the shown software structure and the components included in each layer do not constitute a specific limitation on the electronic device 100. In some other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which are not limited in the present application.

[0113] Figure 5a It is a schematic flowchart of a first model training method provided by an embodiment of the present application. This first model training method is applied to a server, and the server includes a preprocessing module and a training module. This first model is used to predict the film application state and case covering state of an electronic device. As Figure 5a shown, the training method of this first model includes: step S51 - step S54.

[0114] Step S51, the preprocessing module obtains touch sample data of the first electronic device in each protection state.

[0115] Among them, the first electronic device is an electronic device used to collect touch sample data according to the touch operation of a test object (person) on the screen. The first electronic device is, for example, a mobile phone, a tablet, etc. The protection state is used to indicate the film application and / or case covering state of the first electronic device, and the protection state includes: film applied without case covering state, case covered without film application state, film applied and case covered state, and no film applied and no case covered state.

[0116] The touch sample data is used to indicate data such as capacitance values collected by the first electronic device during the process of the test object touching the screen of the first electronic device. In some embodiments, the touch sample data further includes at least one of charging state information, touch pressure information, and the attitude information of the first electronic device. Among them, the charging state information is used to indicate whether the first electronic device is in a charging state, and the charging state information includes a charging state or a non - charging state. The touch pressure information is used to indicate the pressure value when the test person touches the screen of the first electronic device. The attitude information is used to indicate the attitude of the first electronic device, and the attitude is represented by data collected by a gyroscope sensor.

[0117] In an implementation scenario, the first electronic device establishes a connection relationship with the server, and this connection can be a wired connection or a wireless connection. The server receives the touch sample data reported by the first electronic device through this connection relationship.

[0118] Figure 5b It is a schematic flowchart of a method for a server to obtain touch sample data provided by an embodiment of the present application. As Figure 5bAs shown, the server includes a preprocessing module; the first electronic device includes a capacitance sensor, a touch module, a gyroscope sensor, a charging management module, and a communication module. Among them, the touch module is connected to multiple capacitance sensors and can obtain the capacitance values of the multiple capacitance sensors; the touch module is also connected to a pressure sensor and can obtain the pressure value of the pressure sensor. In one embodiment, taking the first protection state of the electronic device as an example, the first protection state indicates any one of the states of not having a film and not having a case, having a case but not having a film, having both a film and a case, and not having a film and not having a case. The preprocessing module in the server obtaining the touch sample data of the electronic device in the first protection state (the above step S51) will be described. This step includes: step S511 - step S513.

[0119] Step S511: The preprocessing module receives the first capacitance information sent by the communication module in the electronic device.

[0120] In the embodiment of the present application, when the finger of the test object (person) touches the screen of the first electronic device, the capacitance value of the capacitance sensor in the electronic device will change. The above first capacitance information indicates the information obtained by the touch module in the first electronic device collecting the capacitance value of the capacitance sensor during the process of the test person performing a touch operation on the first electronic device in the first protection state. The first capacitance information contains multiple frames of capacitance information, and each frame of capacitance information includes the capacitance value of the capacitance sensor in the first electronic device at the corresponding acquisition moment.

[0121] Figure 6 It is a schematic diagram of an electronic device obtaining capacitance information provided by an embodiment of the present application. As Figure 6 shown, the electronic device 100 includes multiple capacitance sensors, and the multiple capacitance sensors are arranged in rows and columns to form a capacitance sensor array 101. During the process of performing a touch operation on the electronic device, as time t progresses, the electronic device 100 can obtain capacitance information frame by frame according to a set touch sampling rate. The touch sampling rate is the frequency at which the touch module in the electronic device obtains capacitance information, and the capacitance sampling rate can be set according to actual application situations, such as being set to 120Hz, 240Hz, or other values, which are not limited in the embodiment of the present application. Each time the electronic device obtains the capacitance values of the multiple capacitance sensors, a frame of capacitance information can be formed. Each frame of capacitance information includes the capacitance values of the capacitance sensors in the capacitance sensor array 108 at the corresponding acquisition moment. The capacitance information obtained by the electronic device, for example Figure 6 the capacitance information 600 obtained at time t0, the capacitance information 601 obtained at time t1, and the capacitance information 602 obtained at time t2 shown in. In the embodiment of the present application, the multiple frames of capacitance information obtained constitute capacitance information, as Figure 6As shown in the figure, the capacitance value information 61 includes the above-mentioned capacitance information 600, capacitance information 601, capacitance information 602, etc.

[0122] Taking Figure 6 A frame of capacitance information 601 collected at time t1 in the figure as an example, the distribution of capacitance values in a frame of capacitance information is exemplarily described. This capacitance information 601 includes the capacitance values of the capacitance sensors in the electronic device 100 at time t1. In this capacitance information 601, the capacitance values collected by some capacitance sensors near the contact point are as shown in 601-1, where each square represents a capacitance sensor, and the number in each square represents the capacitance value of the capacitance sensor. For example, the capacitance value of sensor 601-10 is 995, and the capacitance value of sensor 601-11 is 178. The capacitance values of some capacitance sensors far from the contact point are as shown in 601-2. For example, the capacitance value of capacitance sensor 601-20 is 2, and the capacitance value of sensor 601-21 is 1. Comparing the capacitance value distribution 601-1 near the contact point with the capacitance value distribution 601-2 far from the contact point, it can be seen that the closer the capacitance sensor is to the contact point, the larger the capacitance value of the capacitance sensor.

[0123] From the capacitance value distributions shown by 601-1 and 601-2, it can be seen that in the capacitance value distribution shown by 601-2, the capacitance values of each capacitance sensor are all less than the sensor threshold (the sensor threshold is, for example, 120), and these capacitance values less than the sensor threshold will be regarded as noise values. While in the capacitance value distribution shown by 601-1, the capacitance values of each capacitance sensor are all greater than the sensor threshold (the sensor threshold is, for example, 120), which can effectively characterize the characteristics of the contact between the stylus and the screen of the electronic device and can be regarded as valid information. It can be understood that Figure 6 The specific numerical values of the capacitance values in the figure are only for exemplary illustration.

[0124] In the embodiments of the present application, the first electronic device can obtain the first capacitance value information through Figure 6 the method for obtaining capacitance value information provided in the figure.

[0125] In one embodiment, during the process of performing a touch operation on the first electronic device in the first protection state, the touch control module in the first electronic device sends the obtained first capacitance value information to the communication module for the communication module to send the first capacitance value information to the server, and the preprocessing module in the server obtains the first capacitance value information.

[0126] Step S512: The preprocessing module receives the first attitude information sent by the communication module.

[0127] Among them, the first attitude information indicates the attitude information collected by the gyroscope sensor in the first electronic device during the process of performing a touch operation on the first electronic device in the first protection state. During the process of performing a touch operation on the first electronic device without a film or a case, the gyroscope sensor in the first electronic device sends the first attitude information to the communication module, so that the communication module sends the first attitude information to the server, and the preprocessing module in the server obtains the first attitude information.

[0128] Figure 7 This is an example diagram of the attitude of an electronic device provided by an embodiment of the present application. Refer to Figure 7 , in one example, the attitude information of the electronic device indicates the angle between the electronic device and the horizontal plane. As Figure 7 shown in (1) of Figure 7 , the angle between the electronic device and the horizontal plane is 0°, and it is in a horizontal attitude; as Figure 7 shown in (2) of

[0129] Taking a mobile phone as an example of the electronic device, the mobile phone is often in a horizontal attitude, an inclined attitude, and a vertical attitude during use. Therefore, Figure 7 the above exemplary description of the attitude of the electronic device is provided. However, it can be understood that the attitude information of the electronic device can also include more angle information. For example, the attitude information can include information such as the yaw angle, pitch angle, and roll angle of the electronic device.

[0130] It should be noted that when a touch object touches the screen of the electronic device, the attitude of the electronic device generally affects the capacitance value collected by the capacitance sensor. This is because the attitude of the electronic device affects the contact area when the user's finger touches the screen, thereby affecting the capacitance value collected by the capacitance sensor. For example, generally, when the electronic device is in a vertical attitude, the contact area between the user's finger and the screen of the electronic device is smaller than when the electronic device is in a horizontal attitude. In the case of the same touch duration, a smaller contact area will result in a smaller increase in the capacitance value of the capacitance sensor.

[0131] Step S513: The preprocessing module receives the first charging state information sent by the communication module.

[0132] Among them, the first charging information indicates whether the first electronic device is charging during the process of performing a touch operation on the first electronic device in the first protection state. During the process of performing a touch operation on the first electronic device without a film or a case, the charging management module in the first electronic device sends the first charging status information to the communication module for the communication module to send the first charging status information to the server, and the preprocessing module in the server obtains the first charging status information.

[0133] In one example, during the process of performing a touch operation on the first electronic device without a film or a case, if the first electronic device is charging, the first charging status information indicates the charging status.

[0134] In another example, during the process of performing a touch operation on the first electronic device without a film or a case, if the first electronic device is not charging, the first charging status information indicates the non-charging status.

[0135] It should be noted that when the user touches the screen of the electronic device, whether the electronic device is in the charging state will affect the capacitance value of the capacitance sensor. This is because when the electronic device is not charging, when the user touches the screen of the electronic device, the capacitance value (which can also be called the capacitance signal) of the capacitance sensor in the electronic device will contain a floating ground signal; while when the electronic device is charging, it is equivalent to the electronic device being grounded, and when the user touches the screen of the electronic device, the capacitance value of the capacitance sensor in the electronic device will not contain a floating ground signal. The floating ground signal is relative to the ground signal. When the electronic device is in a grounded state such as charging, the outer shell of the electronic device being directly touched by the hand or the outer shell of the electronic device being directly grounded, the capacitance value collected by the capacitance sensor is the ground signal. On the contrary, the floating ground signal refers to the capacitance value collected when the outer shell of the electronic device is not grounded. The maximum capacitance value of the floating ground signal is smaller than the maximum capacitance value of the ground signal.

[0136] Step S514: The preprocessing module receives the first touch pressure information sent by the communication module.

[0137] Among them, the first touch pressure information indicates the touch pressure when performing a touch operation on the first electronic device in the first protection state, and this touch pressure can be represented by the pressure value collected by the capacitance pressure sensor. During the process of performing a touch operation on the first electronic device in the first protection state, the touch module in the first electronic device collects the pressure value of the pressure sensor to obtain the first touch pressure information. The touch module sends the obtained first touch pressure information to the communication module for the communication module to send the first touch pressure information to the server, and the preprocessing module in the server obtains the first capacitance information.

[0138] It should be noted that the above steps S512, S513, and S514 are all optional steps. In some embodiments, the first electronic device may not collect attitude information and charging status information, but only collect capacitance value information as touch sample data for subsequent processing. In other embodiments, the first electronic device may collect capacitance value information, and at least one of charging status information, touch pressure information, and attitude information, as touch sample data for subsequent processing.

[0139] The above Figure 5b The exemplary description of the preprocessing module in the server obtaining the touch sample data of the first electronic device in the first protection state is provided in the embodiments shown above. It can be understood that when the first electronic device is in other protection states, the method of obtaining the corresponding touch sample data is similar to the above embodiments and will not be elaborated here. For example, when the first protection state is the state of having a film but no case, this other protection state may be the state of having no film and no case, etc. It should be noted that in order to obtain the touch sample data, the touch operation performed on the first electronic device in other protection states is exactly the same as the touch operation in the above embodiments, for example, the touch position (or trajectory) is the same, and the touch force is the same.

[0140] After the preprocessing module in the server obtains the touch sample data of the first electronic device in each protection state, it obtains the training feature data for training the first model based on the obtained touch sample data in each protection state. For a detailed description, see the following step S52.

[0141] Step S52: The preprocessing module performs first data processing on the touch sample data to obtain training feature data.

[0142] Among them, the first data processing includes but is not limited to data extraction processing, data fusion processing, data annotation processing, etc.

[0143] The touch sample data is used to indicate data such as capacitance value information collected by the first electronic device during the process of touching the screen. In some embodiments, the touch sample data further includes at least one of charging status information, the attitude information of the first electronic device, and touch pressure information.

[0144] Figure 5c This is a flowchart of a method for obtaining training feature data provided by an embodiment of the present application. In one embodiment, taking the touch sample data obtained when the first electronic device is in the first protection state as an example, the step (the above step S52) of the preprocessing module performing first data processing on the touch sample data to obtain training feature data is described: including the following steps S521 - S522.

[0145] Step S521: Extract first capacitance feature data according to the first capacitance value information.

[0146] Among them, the first capacitance value information indicates a set of multiple frames of capacitance information obtained by the electronic device during the process of performing a touch operation on the first electronic device in the first protection state (for example, the film is not covered with a case). Each frame of capacitance information includes the capacitance values of each capacitance sensor at the corresponding acquisition moment.

[0147] In the embodiments of the present application, when extracting the first capacitance feature data according to the first capacitance value information, it is necessary to extract a capacitance matrix from the capacitance information included in the first capacitance value information. For each frame of capacitance information in the first capacitance value information, determine the position of the capacitance sensor corresponding to the maximum capacitance value from this capacitance information. The position of the capacitance sensor corresponding to the maximum capacitance value in this capacitance information is the contact position. It should be noted that when the test object touches the screen of the first electronic device, the capacitance values of the capacitance sensors near the contact position change significantly and can be regarded as valid data, which can effectively represent the contact characteristics between the test object and the screen of the first electronic device. In addition, the input data during the training of the first model is a matrix, and there are also restrictions on the length and width of the input data. Therefore, for each frame of capacitance information in the first capacitance value information, according to the position of the capacitance sensor corresponding to the maximum capacitance value in this capacitance information and the requirements for the length and width of the input data, extract the corresponding matrix from this capacitance information.

[0148] Figure 8 It is an example diagram of a preprocessing module provided by the embodiments of the present application for extracting capacitance feature data. As Figure 8 shown, the first capacitance information 800 includes multiple frames of capacitance information, and these multiple frames of capacitance information are sorted according to the acquisition moment. Taking the capacitance information 601 acquired at time t1 as an example, the input data of the first model is a matrix. When the requirements for the length and width of the input data are 7*7, extract the corresponding 7*7 matrix 601-3 from this capacitance information 601. The center of this matrix 601-3 contains the maximum capacitance value (the position of the contact point) in this capacitance information, that is, the capacitance value 995 of the capacitance sensor 601-10.

[0149] In the embodiments of the present application, to improve the accuracy of the training data, it is necessary to obtain the corresponding 7*7 matrices from each frame of capacitance information included in the first capacitance value information 800 and jointly form the capacitance feature data.

[0150] In one implementation manner, the first rule for selecting the capacitance feature data is: from all the capacitance information included in the first capacitance value information, with the capacitance information containing the maximum capacitance value as the center, select the center, a preset number of frames (for example, 5 frames) of capacitance information sorted before the center, and a preset number of frames of capacitance information sorted after the center, and extract the corresponding matrices from each selected frame of capacitance information.

[0151] AsFigure 8 As shown, taking the capacitance information 601 containing the maximum capacitance value (the maximum capacitance value is the capacitance value 3995 of the sensor 601-10) as an example. Centering on the capacitance information 601, select the capacitance information 601, 5 frames of capacitance information sorted before the capacitance information 601 (capacitance information 607, 608, 609, 610, 611), and 5 frames of capacitance information sorted after the capacitance information 601 (capacitance information 602, 603, 604, 605, 606), and extract the corresponding capacitance value matrices from each selected frame of capacitance information to jointly form the first capacitance value feature data 801, and the size of the first capacitance value feature data 801 is 11×7×7.

[0152] In another embodiment, when a touch operation is performed on the first electronic device, during the process of a finger approaching and leaving the screen of the electronic device, the capacitance value sensed by the capacitance sensor is unstable and the data validity is poor. Therefore, it is necessary to further screen out the capacitance information with higher data stability from the initially selected capacitance information. In this embodiment, the second rule for selecting the capacitance value feature data is: from all the capacitance information included in the first capacitance value information, centering on the capacitance information containing the maximum capacitance value, select the center, a preset number of frames (for example, 5 frames) of capacitance information sorted before the center, and a preset number of frames of capacitance information sorted after the center. Then, from the preset number of frames of capacitance information sorted before the center, discard the first N frames (N is, for example, 2) of capacitance information, and from the preset number of frames of capacitance information sorted after the center, discard the last N frames of capacitance information; for the remaining (preset number of frames - 2N) capacitance information, extract the corresponding capacitance value matrices from each frame of capacitance information to jointly form the first capacitance value feature data.

[0153] Taking Figure 8 as an example, after centering on the capacitance information 601 and selecting the capacitance information 601, 5 frames of capacitance information sorted before the capacitance information 601 (capacitance information 607, 608, 609, 610, 611), and 5 frames of capacitance information sorted after the capacitance information 601 (capacitance information 602, 603, 604, 605, 606), discard the first 2 frames of capacitance information (capacitance information 610 and 611) from the 5 frames of capacitance information sorted before the capacitance information 601; and discard the last 2 frames of capacitance information (capacitance information 605 and 606) from the 5 frames of capacitance information sorted after the capacitance information 601; for the remaining 7 frames of capacitance information, extract the corresponding capacitance value matrices from each frame of capacitance information to jointly form the first capacitance value feature data, and the size of the first capacitance value feature data is 7×7×7.

[0154] It should be noted that when the contact time between the test object and the screen of the first electronic device is less than the preset duration, the first electronic device cannot obtain enough frames of capacitance information, that is, the capacitance information included in the first capacitance value information will be less than M frames. When the capacitance information obtained by the electronic device is less than M frames, among the capacitance information less than M frames, the stable capacitance information (the capacitance information remaining after removing the first N frames and the last N frames) is not sufficient to support the training of the first model. The preset duration can be determined according to the touch sampling rate of the electronic device, such as 0.2 s or 0.3 s, which is not specifically limited in this embodiment. The value of M can be set to 2 * preset frames + 1 frame.

[0155] For example, taking the case where at least 7 frames of stable capacitance information are required to support the training of the first model as an example. When the first capacitance value information includes 11 frames of capacitance information, the stable capacitance information (such as the capacitance information remaining after removing the first 2 frames and the last 2 frames) is 7 frames. When the first capacitance value information only includes 7 frames of capacitance value information, the stable capacitance information (such as the capacitance information remaining after removing the first 2 frames and the last 2 frames) is only 3 frames, which cannot support the training of the first model. Therefore, in the above second rule, it is necessary to initially select (2 * preset frames + 1 frame) of capacitance information from the first capacitance value information, and then discard the capacitance information sorted in the first N frames and the last N frames from the (2 * preset frames + 1 frame) of capacitance information. If (2 * preset frames + 1 frame) of capacitance information cannot be initially selected from the first capacitance value information, the first capacitance value information can be discarded.

[0156] Step S522: Label the first capacitance value feature data according to the first protection state in which the electronic device is located to obtain first training feature data.

[0157] Among them, labeling is the process of adding labels to data.

[0158] In the embodiment of the present application, the first capacitance value feature data is the data obtained when the first electronic device is in the first protection state. Therefore, the preprocessing module labels the first capacitance value feature data including: adding a label of the first protection state to the first capacitance value feature data to obtain first training feature data.

[0159] In this embodiment, through the above steps S521 - S522, the first training feature data is obtained. The first training feature data is the feature data related to the first electronic device in the first protection state. It can be understood that for the touch sample data in other protection states, performing operations similar to the above steps S521 - S522 can obtain the training feature data in the corresponding protection state, providing a data basis for the training process of the first model.

[0160] For example, when the first protection state is the state of having a film applied but no case, other protection states can be the state of having no film applied and no case. According to the touch sample data obtained in the state of having no film applied and no case, operations similar to the above steps S521 - step S522 are performed, and the second training feature data in the state of having no film applied and no case can be obtained. In the embodiments of the present application, the touch sample data obtained in each protection state is subjected to first data processing, and the obtained training feature data includes the training feature data corresponding to each protection state. For example, it includes the above-mentioned first training feature data and second training feature data, etc.

[0161] Figure 5d FIG. is a schematic flowchart of another method for obtaining training feature data provided by the embodiments of the present application. In one embodiment, the touch sample data further includes auxiliary parameters. Taking the touch sample data obtained when the first electronic device is in the first protection state as an example, the steps (the above step S52) of the preprocessing module performing first data processing on the touch sample data to obtain training feature data are described as follows: including the following steps S523 - step S525.

[0162] Step S523: Extract first capacitance feature data according to the first capacitance information.

[0163] Among them, this step S523 is the same as the foregoing step S521. For the detailed description of step S521, reference can be made to the foregoing, and details will not be repeated here.

[0164] Step S524: Perform data fusion processing on the first capacitance feature data and the auxiliary parameters to obtain first fusion feature data.

[0165] Among them, the auxiliary parameters include one or more of: first charging state information, first attitude information, and first touch pressure information. The first charging state information is used to indicate whether the first electronic device is charging when a tester performs a touch operation on the first electronic device in the first protection state. The first charging state information includes a non-charging state or a charging state. The first attitude information is used to indicate the attitude of the first electronic device when a tester performs a touch operation on the first electronic device in the first protection state, and can be represented by the data collected by a gyroscope sensor. The first touch pressure information is used to indicate the touch pressure when a tester performs a touch operation on the first electronic device in the first protection state, and can be represented by a pressure value.

[0166] Data fusion processing is used to indicate a multi-level process of processing the association of data and information from single and multiple sources. The methods of data fusion processing include, but are not limited to, projection, splicing, addition, etc. In the actual application process, the specific data fusion processing method to be selected can be selected according to the specific situation, and it is not limited in the embodiments of the present application.

[0167] It should be noted that the auxiliary parameter refers to the parameter that affects the capacitance value of the capacitance sensor when the test object touches the screen of the electronic device. Therefore, involving this auxiliary parameter in the training process of the first model can improve the accuracy of the trained first model.

[0168] Step S525: According to the first protection state of the electronic device, annotate the first fusion feature data to obtain the first training feature data.

[0169] Among them, annotation is the process of adding labels to data.

[0170] In the embodiment of the present application, the preprocessing module annotating the first fusion feature data includes: adding a label of "film applied without case" to the first fusion feature data to obtain the first training feature data.

[0171] In this embodiment, through the above steps S521 - S523, the first training feature data is obtained. The first training feature data is the feature data related to the first electronic device in the first protection state. It can be understood that for the touch sample data in other protection states, performing operations similar to the above steps S521 - S523 can obtain the training feature data in the corresponding protection state, providing a data basis for the training process of the first model.

[0172] In the embodiment of the present application, the protection state label on the training feature data can be used as the true result to measure the accuracy of the output of the first model. In the subsequent training process of the first model, input the training feature data into the first model to obtain the output result of the first model. According to the output result and the protection state on the input training feature data, the error between the output result and the true result can be determined.

[0173] Step S53: The preprocessing module sends the training feature data to the training module.

[0174] Among them, the training feature data contains the protection state label.

[0175] Step S54: The training module trains the initial first model according to the training feature data to obtain the trained first model.

[0176] Figure 9 This is a schematic diagram of model training provided by the embodiment of the present application. As Figure 9As shown, the training module takes the training feature data as input data and inputs it into the initial first model 900 to obtain the corresponding output result. Among them, there are multiple groups of training feature data, and the multiple groups of training feature data are different from each other. For example, one group of training feature data can be the feature data with the label of film applied without case, another group of training feature data is the feature data with the label of case applied without film, and another group of training feature data is the feature data with the label of film applied and case applied, etc. The output result is used to indicate the protection state of the predicted electronic device. Update the weights of the first model 900 according to the output result and the loss function, and finally obtain the weight parameters that meet the requirements. The requirements that the weight parameters need to meet refer to that when the first model predicts the protection state of the electronic device based on the weight parameters, the error between the predicted protection state (output result) and the protection state (actual result) indicated by the input data label is less than the preset error, and the preset error can be determined according to the actual application situation. The weight parameters that meet the requirements are the weights of the trained first model, where the loss function is, for example, the cross-entropy loss function.

[0177] In the embodiment of the present application, the first model can be a network model based on deep learning. For example, ResNet18 (a residual network containing 18 convolutional layers) in Convolutional Neural Networks (CNN) is not specifically limited in this embodiment.

[0178] In the embodiment of the present application, through the above steps S51 - S54, a trained first model is obtained. The trained first model can determine the protection state of the electronic device according to the touch data generated by the electronic device for the touch operation.

[0179] It should be noted that the trained first model can be deployed in the second electronic device. The first model deployed in the second electronic device can determine the protection state of the second electronic device according to the touch data generated by the second electronic device for the touch operation. For the detailed description of deploying the trained first model to the second electronic device, refer to the following Figure 10 provided first model deployment method.

[0180] Figure 10 It is a flowchart of a first model deployment method provided by the embodiment of the present application. The first model deployment method is applied to a server or a device capable of terminal deployment. As Figure 10 shown, the first model deployment method includes: steps S1001 - S1002.

[0181] Step S1001: Perform a preset format conversion on the trained first model.

[0182] Among them, the preset format conversion can be the conversion from pyTorch (an open-source neural network framework) to NCNN (a deployment framework for deploying models to mobile devices).

[0183] In the embodiment of the present application, during the process of training the first model, the framework adopted by the first model can be pyTorch. PyTorch is a large machine learning framework that requires a large amount of storage space. Therefore, when deploying the model, a lightweight framework such as NCNN needs to be used. In this embodiment, after the first model is trained, the first model is converted from pyTorch to NCNN to facilitate the subsequent deployment of the first model.

[0184] Step S1002: Deploy the first model after the preset format conversion and other collaborative modules on the hardware abstraction layer of the second electronic device.

[0185] Among them, as Figure 4 shown, the other collaborative modules are, for example, a data processing module and a parameter configuration module. The data processing module has the same function as the preprocessing module of the server in the foregoing embodiment, and is used to process the received touch sample data into input data that can be input into the first model; the parameter configuration module is used to match corresponding sensitivity threshold parameters for the capacitance sensor according to the output data of the first model. The detailed description of the parameter configuration module can be found in the following embodiments and will not be elaborated here.

[0186] In the embodiment of the present application, the trained first model is deployed on the second electronic device, so that the second electronic device can automatically sense the film and case states of the electronic device through the first model.

[0187] In the embodiment of the present application, after the first model is deployed on the second electronic device, the second electronic device can automatically sense its own protection state through the first model, and thus can adaptively adjust the sensitivity threshold of the capacitance sensor according to its own protection state, so as to effectively sense the user's touch operation in each protection state and avoid phenomena such as touch interruption and false touch caused by the change of the protection state. Among them, after the second electronic device deploys the first model, the detailed description of automatically sensing its own protection state through the first model and then adaptively adjusting the sensitivity threshold of the capacitance sensor can be found in the Figure 11 provided method for determining the sensitivity threshold of the capacitance sensor.

[0188] Figure 11Schematic diagram of a method for determining the sensitivity threshold of a capacitance sensor provided by an embodiment of this application. The method for determining the sensitivity threshold of the capacitance sensor is applied to a second electronic device, which is an electronic device capable of automatically sensing its own protection state. The hardware abstraction layer of the second electronic device includes: a data processing module, a first model, and a parameter configuration module. The second electronic device may be an electronic device capable of screen touch such as a tablet, a mobile phone, or a laptop. As Figure 11 shown, the method for determining the sensor sensitivity threshold includes: steps S111 - S115.

[0189] S111. During the process of the sensing object contacting the screen of the second electronic device, the data processing module acquires target touch data.

[0190] Among them, the sensing object may be a user. The sensing object contacts the screen of the second electronic device. For example, the user touches the screen of the second electronic device with a finger. The target touch data is used to indicate data such as capacitance information collected by the second electronic device during the process of the user contacting the screen of the second electronic device. In some embodiments, the target touch data further includes one or more of charging state information, touch pressure information, and the attitude information of the second electronic device.

[0191] In one example, when the second electronic device is in an un-filmed state, the user's contact with the screen of the second electronic device means contacting the screen of the electronic device itself.

[0192] In another example, when the second electronic device is in a filmed state, the user's contact with the screen of the second electronic device means contacting the protective film attached to the outer screen of the second electronic device.

[0193] In one implementation manner, when the user contacts the screen of the second electronic device, the data processing module acquires target touch data, including the following step one.

[0194] Step one. During the process of the user contacting the screen of the second electronic device, the data processing module acquires target capacitance information.

[0195] Among them, the target capacitance information indicates the capacitance of each capacitance sensor in the second electronic device.

[0196] As Figure 1As shown, a plurality of capacitance sensors with an array distribution are configured on the second electronic device. The plurality of capacitance sensors are always on. When a sensing object performs a touch operation on the screen of the second electronic device, the second electronic device acquires the capacitance values of the plurality of capacitance sensors at a predetermined touch sampling rate. The touch module in the second electronic device is connected to the plurality of capacitance sensors. The capacitance sampling process can be executed by the touch module (or touch chip), and the data processing module acquires capacitance information from the touch module, or the touch module sends the capacitance information to the data processing module.

[0197] In the case where the sensing object is in contact with the screen of the second electronic device, target capacitance information is acquired based on a preset touch sampling rate. Multiple frames of target capacitance information sorted by the acquisition time form target capacitance value information. Among them, the target capacitance information includes the capacitance values of each capacitance sensor in the second electronic device at the corresponding acquisition time.

[0198] Among them, for the process of the second electronic device acquiring the target capacitance value information, reference can be made to the detailed description of the electronic device acquiring the capacitance value information provided above, which will not be elaborated here. The target capacitance value information includes multiple frames of target capacitance information acquired based on a preset touch sampling rate during a single touch operation. The multiple frames of target capacitance information are sorted by the acquisition time, and each frame of target capacitance information includes the capacitance values of each capacitance sensor in the second electronic device at the corresponding acquisition time. Figure 6

[0199] In another embodiment, when the user is in contact with the screen of the second electronic device, the data processing module acquires target touch data, which includes not only the above step one but also one or more of the following steps two to four.

[0200] Step two: The data processing module receives the target charging status information sent by the charging management module.

[0201] Among them, the target charging status information is used to indicate whether the second electronic device is charging when the user is in contact with the screen of the second electronic device. For example, when the user is in contact with the screen of the second electronic device and the second electronic device is charging, the target charging status information indicates the charging state; when the user is in contact with the screen of the second electronic device and the second electronic device is not charging, the target charging status information indicates the non-charging state.

[0202] Step three: The data processing module receives the target attitude information sent by the gyroscope sensor.

[0203] Figure 7 Among them, the target attitude information is used to indicate the attitude of the second electronic device when the user is in contact with the screen of the second electronic device. For the corresponding detailed description, reference can be made to the above, which will not be elaborated here.

[0204] ​​Step 4: The data processing module receives the touch pressure information sent by the pressure sensor.

[0205] The touch pressure information is used to indicate the pressure value of the pressure sensor in the second electronic device when the user touches the screen of the second electronic device.

[0206] In this embodiment, when the user touches the screen of the second electronic device, the data processing module respectively obtains corresponding data from the capacitance sensor, the pressure sensor, the gyroscope sensor, and the charging management module, and forms target touch data corresponding to this screen touch, providing a data basis for subsequent analysis.

[0207] S112: The data processing module performs second data processing on the target touch data to obtain touch feature data.

[0208] The second data processing includes, but is not limited to, data extraction processing, data fusion processing, etc.

[0209] In one embodiment, the data processing module performs second data processing on the target touch data to obtain touch feature data, including: step S1121.

[0210] Step S1121: Extract target capacitance feature data according to the target capacitance information.

[0211] The manner in which the data processing module extracts target capacitance feature data according to the target capacitance information is the same as that of the preprocessing module extracting capacitance feature data according to the first capacitance information in the foregoing step S521 and Figure 8 shown. Therefore, for the manner in which the data processing module extracts target capacitance feature data according to the target capacitance information, reference can be made to the detailed description of step S521 and Figure 8 in the foregoing embodiments, which will not be elaborated here.

[0212] In this embodiment, when the target touch data only includes the target capacitance information, the extracted target capacitance feature data is the touch feature data.

[0213] In another embodiment, the target touch data further includes target auxiliary data. Then, when the data processing module performs second data processing on the target touch data to obtain touch feature data, in addition to including the above step S1121, it further includes the following step S1122.

[0214] Step S1122: Perform data fusion processing on the target capacitance feature data and the target auxiliary data to obtain touch feature data.

[0215] Among them, the target auxiliary data includes one or more of target charging status information, target attitude information, and target touch pressure information. The target charging status information indicates whether the second electronic device is charging when the sensing object contacts the screen of the second electronic device, and the target charging status information includes a charging status or a non-charging status. The target attitude information indicates the attitude of the second electronic device when the sensing object contacts the screen of the second electronic device, and this attitude can be represented by the data collected by a gyroscope sensor. The target touch pressure information indicates the contact pressure when the sensing object contacts the screen of the second electronic device, and the target touch pressure information can be represented by the pressure value collected by a pressure sensor in the second electronic device.

[0216] The methods of data fusion processing include, but are not limited to, projection, splicing, addition, etc. In the actual application process, the specific method of data fusion processing can be selected according to the specific situation, and it is not limited in the embodiments of this application.

[0217] In one example, when the target touch data further includes target touch pressure information, the data processing module may perform data fusion processing on the target capacitance characteristic data, the target touch pressure information, and the target charging status information to obtain touch characteristic data. It should be noted that the target touch pressure information can represent the pressure value when the user touches the screen of the electronic device. When this pressure value is larger, the distance between the user's finger and the capacitance sensor in the electronic device is closer, and the capacitance of the capacitance sensor will change accordingly. Therefore, using the target touch pressure information as one of the target contact data can improve the accuracy of the protection state of the electronic device determined by the subsequent first model.

[0218] In another example, when the target touch data further includes target attitude information, the data processing module may perform data fusion processing on the target capacitance characteristic data, the target attitude information, and the target charging status information to obtain touch characteristic data. It should be noted that different attitude information of the electronic device will affect the capacitance value of the capacitance sensor when the user touches the electronic device. Therefore, using the target attitude information as one of the target touch data can improve the accuracy of the protection state of the electronic device determined by the subsequent first model.

[0219] In this embodiment, the data processing module performs second data processing on the target touch data to obtain input data (touch characteristic data) that can be input into the first model.

[0220] S113. The data processing module inputs the touch characteristic data into the first model.

[0221] Among them, the first model is pre-configured in the second electronic device and is used to sense the protection state of the second electronic device. This protection state is specifically the state of the screen protector and the case.

[0222] S114. The first model outputs protection status information to the parameter configuration module based on the input touch feature data.

[0223] Among them, the protection status information is used to indicate the current protection status of the second electronic device, and the protection status includes any one of the following: the state of having a film but no case, the state of having a case but no film, the state of having both a film and a case, and the state of having neither a film nor a case.

[0224] S115. The parameter configuration module determines the sensitivity threshold of the capacitance sensor according to the protection status information.

[0225] It should be noted that when the input of the sensor slowly increases from zero, its output remains zero until the input increases to a certain minimum value, and then the output changes. This minimum value is the sensitivity threshold. Taking the capacitance sensor as an example, during the process that the user's finger approaches and touches the screen of the electronic device, the capacitance of the capacitance sensor increases as the user's finger approaches. Only when the capacitance increases to be greater than the sensitivity threshold of the capacitance sensor, the capacitance sensor will output a capacitance signal, and then the electronic device will sense the user's touch operation based on this capacitance signal.

[0226] In the embodiment of the present application, a correspondence table between the protection status information and the sensitivity threshold of the capacitance sensor is pre-stored in the parameter configuration module. The parameter configuration module can determine the sensitivity threshold of the capacitance sensor corresponding to the protection status information from this correspondence table.

[0227] In one example, when the protection status information indicates the state of having neither a film nor a case, the sensitivity threshold of the capacitance sensor is determined to be the first threshold.

[0228] Among them, the first threshold is pre-stored in the parameter configuration module and is the sensitivity threshold of the capacitance sensor corresponding to the protection status of the electronic device having neither a film nor a case. This first threshold is, for example, the sensitivity threshold configured for the capacitance sensor when the electronic device leaves the factory.

[0229] In another example, when the protection status information indicates the state of having a film but no case, the sensitivity threshold of the capacitance sensor is determined to be the second threshold.

[0230] Among them, the second threshold is pre-stored in the parameter configuration module and is the sensitivity threshold of the capacitance sensor corresponding to the protection status of the electronic device having a film but no case. The second threshold is less than the above-mentioned first threshold.

[0231] It should be noted that as Figure 1As shown, for the same touch operation, the capacitance amplitude sensed by the capacitance value sensor in the case of a film attached is lower than that sensed in the case of no film attached. Therefore, when the electronic device has a film attached, it is necessary to correspondingly reduce the sensitivity threshold of the capacitance value sensor so that for the same touch operation, the capacitance amplitude sensed by the capacitance value sensor in the case of a film attached is also greater than the sensitivity threshold, thereby generating a sensing signal and enabling the electronic device to sense the user's touch operation based on this sensing signal.

[0232] In another example, when the protection status information indicates that the case is in the non-film-attached state, the sensitivity threshold of the capacitance value sensor is determined to be the third threshold.

[0233] Among them, the second threshold is the sensitivity threshold of the capacitance value sensor pre-stored in the parameter configuration module and corresponding to the protection state of the electronic device case without a film attached. The third threshold is less than the above-mentioned first threshold.

[0234] It should be noted that when the user holds the electronic device and the electronic device is in the non-cased state, the electronic device is equivalent to being grounded. At this time, when the user touches the screen of the electronic device, the capacitance signal of the capacitance value sensor does not include a floating ground signal. When the user holds the electronic device but the electronic device is in the cased state, the electronic device is equivalent to being semi-grounded. At this time, when the user touches the screen of the electronic device, the capacitance signal of the capacitance value sensor includes a partial floating ground signal, causing the capacitance amplitude sensed by the capacitance value sensor to decrease.

[0235] Figure 12 This is a distribution schematic diagram of the capacitance values of the capacitance value sensor collected when the electronic device provided in the embodiment of the present application is in the cased state and the non-cased state. In Figure 12 This is illustrated by taking the example that the user performs touch operations at the same position on the electronic device 100 in the non-cased state and the electronic device 100 in the cased state respectively.

[0236] Among them, the electronic device 100 is configured with a capacitance value sensor array 101. The capacitance value sensor array 101 includes a plurality of capacitance value sensors arranged in rows and columns. Among them, each small square is used to indicate a capacitance value sensor. It should be noted that Figure 12 the capacitance value sensor array in

[0237] is only for exemplary illustration. The number of rows and columns of the capacitance value sensor array and the number of capacitance value sensors included in each row or each column can be set according to the actual application situation, and no specific limitation is made in the embodiment of the present application. Figure 12As shown in (1), in a scenario where the electronic device 100 is in an unhoused state, the user touches the screen of the electronic device 100 with a finger. The capacitance values of some capacitance sensors near the user's finger are as shown in 1201-1. Here, each square represents a capacitance sensor, and the number in the square represents the magnitude of the capacitance value detected by the capacitance sensor. For example, the capacitance value detected by capacitance sensor 1201-01 is 880. It should be noted that capacitance values less than the sensitivity threshold of the capacitance sensor can be considered noise values, and the sensitivity threshold of this capacitance sensor is, for example, 120.

[0238] As Figure 12 As shown in (2), in a scenario where the electronic device 100 is in a housed state, a protective case 105 is put on the electronic device 100. The user touches the screen of the electronic device 100 with a finger. The capacitance values of some capacitance sensors near the user's finger are as shown in 1201-2. Here, each square represents a capacitance sensor, and the number in the square represents the magnitude of the capacitance value of the capacitance sensor. For example, the capacitance value of capacitance sensor 1201-01 is 100.

[0239] Comparing the capacitance value distribution shown in 1201-1 with the capacitance value distribution shown in 1201-2, the capacitance value of capacitance sensor 1201-01 is 880 when the electronic device 100 is in the unhoused state, and the capacitance value collected in the housed state is 100. That is, for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the housed state is lower than that sensed in the unhoused state. Therefore, when the electronic device is housed, it is necessary to appropriately reduce the sensitivity threshold of the capacitance sensor so that for the same touch operation, the capacitance amplitude sensed by the capacitance sensor in the housed state can also be greater than this sensitivity threshold, thereby outputting a sensing signal and enabling the electronic device to sense the user's touch operation based on this sensing signal.

[0240] In yet another example, in the case where the protection status information indicates that the device is both film-covered and housed, the sensitivity threshold of the capacitance sensor is determined to be the fourth threshold.

[0241] Among them, the fourth threshold is pre-stored in the parameter configuration module and is the sensitivity threshold of the capacitance sensor corresponding to the protection status of the electronic device being both film-covered and housed. The fourth threshold is less than the above-mentioned first threshold, second threshold, and third threshold.

[0242] In the embodiments of the present application, when a user touches the screen of an electronic device, a second electronic device obtains target touch data, and determines the protection state of the second electronic device according to the target touch data. That is, the second electronic device can automatically sense whether the electronic device is pasted with a film or covered with a case. Then, according to the protection state of the second electronic device, a sensitivity threshold of a capacitance sensor is determined, so that the capacitance sensor can work based on the determined sensitivity threshold, realizing automatic adjustment of the sensitivity threshold of the capacitance sensor, effectively solving problems such as touch interruption and false touch after the second electronic device is in a state of being pasted with a film or covered with a case, and improving the touch experience of the user when using the second electronic device.

[0243] The embodiments of the present application also provide an exemplary description of an application scenario. In the application scenario provided by the embodiments of the present application, it includes a server, a first electronic device, and a second electronic device. Among them, the server is connected to the first electronic device, and this connection can be a wired connection or a wireless connection. Data transmission is carried out between the server and the first electronic device through this connection.

[0244] In this application scenario, the server is used to train a first model, and this first model is a model for sensing the protection state of an electronic device. The first electronic device is an electronic device used to collect touch sample data for training this first model. The second electronic device is an electronic device used to deploy the trained first model.

[0245] Testers perform touch operations on the screens of the first electronic devices in various protection states respectively. The first electronic device obtains touch sample data of itself (the first electronic device) in various protection states according to this touch operation, and sends the touch sample data in various protection states to the server through a pre-established connection.

[0246] The server receives the touch sample data in various protection states sent by the first electronic device, and trains an initial first model based on the touch sample data in various protection states to obtain a trained first model. This trained first model can determine the protection state of the electronic device according to the touch data collected by the electronic device during the process of the user touching the screen of the electronic device.

[0247] The device manufacturer can deploy the trained first model into the second electronic device, so that the second electronic device has the ability to automatically sense its own protection state.

[0248] When the user uses the second electronic device and performs a touch operation on the screen of the second electronic device, the second electronic device can obtain target touch control data according to the user's touch operation, and input the target touch control data after data processing into a pre-deployed first model to obtain the protection status information output by the first model, so as to realize the automatic perception of the protection status where it is located. After automatically perceiving the protection status where it is located, the second electronic device can adjust the sensitivity threshold of the capacitance sensor in the second electronic device according to the perceived protection status, so as to realize the automatic adaptation of the sensitivity threshold of the capacitance sensor to the protection status where the second electronic device is located, and can effectively solve the problems of touch failure and false touch that occur after the electronic device is pasted with a film or covered with a case, and improve the touch experience of the user when using the second electronic device.

[0249] It can be understood that in order for the electronic device to implement the above functions, it includes the corresponding hardware and / or software modules for executing each function. Combining the algorithm steps of each example described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraint conditions of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0250] The steps executed by the electronic device in the above-mentioned first model training method, first model deployment method, and capacitance sensor sensitivity threshold determination method provided by the embodiments of the present application can also be executed by a chip system included in the electronic device. Among them, the chip system can include a processor and a Bluetooth chip. The chip system can be coupled to the memory so that when the chip system runs, it calls the computer program stored in the memory to implement the steps executed by the above-mentioned electronic device. Among them, the processor in the chip system can be an application processor or a processor other than an application processor.

[0251] This embodiment also provides a computer-readable medium, in which computer instructions are stored. When the computer instructions run on the electronic device, the electronic device is caused to execute the above-mentioned related method steps to implement the method in the above-mentioned embodiments.

[0252] This embodiment also provides a computer program product. When the computer program product runs on a computer, the computer is caused to execute the above-mentioned related steps to implement the method in the above-mentioned embodiments.

[0253] In addition, an embodiment of the present application further provides a device, which may specifically be a chip, a component or a module. The device may include a processor and a memory connected to each other. The memory is used to store computer-executable instructions. When the device runs, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the methods in the above method embodiments.

[0254] Among them, the electronic device, computer-readable medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be elaborated here.

[0255] Through the description of the above embodiments, those skilled in the art can understand that for the convenience and simplicity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0256] In several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0257] The units described as separate components may or may not be physically separated. The components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in each embodiment of the present application, each functional unit may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0258] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0259] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining the sensitivity threshold of a capacitance sensor, characterized in that Applied to a second electronic device, the method includes: When a sensing object contacts the screen of the second electronic device, obtaining target touch data; the target touch data at least includes target capacitance value information; the target capacitance value information indicates the capacitance value of a capacitance sensor in the second electronic device; According to the target touch data, determining the protection state of the second electronic device; the protection state includes one of a state of not having a film and not having a case, a state of having a film but not having a case, a state of having a case but not having a film, and a state of having a film and having a case; Based on the protection state, determining a sensitivity threshold corresponding to the capacitance sensor.

2. The method according to claim 1, wherein The step of, when a sensing object contacts the screen of the second electronic device, obtaining target touch data includes: When the sensing object contacts the screen of the second electronic device, obtaining target capacitance information based on a preset touch sampling rate, and multiple frames of the target capacitance information constitute the target capacitance value information; wherein, the target capacitance information includes the capacitance value of the capacitance sensor in the second electronic device at the corresponding acquisition moment.

3. The method according to claim 1 or 2, characterized in that, The step of, according to the target touch data, determining the protection state of the second electronic device includes: According to the target touch data, determining touch feature data; Inputting the touch feature data into a first model to obtain the protection state output by the first model.

4. The method according to claim 3, wherein According to the target touch data, determining touch feature data includes: According to the target capacitance value information, determining target capacitance value feature data; Using the target capacitance value feature data as the touch feature data; Or, The target touch data further includes target auxiliary data; performing data fusion processing on the target capacitance value feature data and the target auxiliary data to obtain touch feature data; wherein, the target auxiliary data includes one or more of target charging state information, target attitude information, and target touch pressure information.

5. The method according to claim 4, wherein The step of, according to the target capacitance value information, determining target capacitance value feature data includes: Selecting multiple frames of capacitance information from the target capacitance value information according to a preset selection rule; Extracting a capacitance value matrix with a preset size from each selected frame of capacitance information and combining them into the capacitance value feature data.

6. The method according to claim 5, characterized in that, The matrix center of the capacitance value matrix is the maximum capacitance value in the corresponding capacitance information.

7. The method according to claim 5, wherein The preset selection rule includes: In the target capacitance value information, taking the target capacitance information where the maximum capacitance value is located as the center, and selecting the center, a preset number of frames of target capacitance information before the center in order, and a preset number of frames of target capacitance information after the center in order.

8. The method according to claim 7, characterized in that, The preset selection rule further includes: Discarding the first N frames of target capacitance information from the preset number of frames of target capacitance information before the center in order; where N frames is greater than 1 frame and less than the preset number of frames; Discarding the last N frames of target capacitance information from the preset number of frames of target capacitance information after the center in order.

9. The method according to claim 3, characterized in that, Based on the protection state, determining a sensitivity threshold corresponding to the capacitance sensor includes: When the protection state indicates a state of not having a film and not having a case, determining that the sensitivity threshold corresponding to the capacitance sensor is a first threshold; When the protection state indicates the non - shell - covered state of the film, determine that the sensitivity threshold corresponding to the capacitance sensor is the second threshold; the second threshold is less than the first threshold; When the protection state indicates the non - film - covered state of the shell, determine that the sensitivity threshold corresponding to the capacitance sensor is the third threshold; the third threshold is less than the first threshold; When the protection state indicates the state of both film - covered and shell - covered, determine that the sensitivity threshold corresponding to the capacitance sensor is the fourth threshold; the fourth threshold is less than the second threshold and the third threshold.

10. The method according to claim 1, wherein After determining the sensitivity threshold corresponding to the capacitance sensor based on the protection state, the method further includes: Adjust the sensitivity parameter of the capacitance sensor to the determined sensitivity threshold, so that the touch chip connected to the capacitance sensor outputs a sensing signal based on the determined sensitivity threshold.

11. A model training method, characterized in that, Applied to a server, the server is connected to a first electronic device; the method includes: Obtain touch sample data of the first electronic device in each protection state; the protection state indicates the states of non - film - covered and non - shell - covered, film - covered and non - shell - covered, shell - covered and non - film - covered, and film - covered and shell - covered; the touch sample data includes capacitance information and charging state information; Perform first - stage data processing on the touch sample data to obtain training feature data; Train an initial first model according to the training feature data to obtain the trained first model; the trained first model is used to be configured in a second electronic device to determine the protection state of the second electronic device.

12. The method according to claim 11, wherein The obtaining of the touch sample data of the first electronic device in each protection state includes: Receive the first capacitance information sent by the first electronic device; the first capacitance information indicates the capacitance of the capacitance sensor in the first electronic device during the process that a tester touches the screen of the first electronic device in the first protection state; the first protection state indicates any one of the states of non - film - covered and non - shell - covered, film - covered and non - shell - covered, shell - covered and non - film - covered, and film - covered and shell - covered.

13. The method according to claim 11 or 12, characterized in that, The performing of first - stage data processing on the touch sample data to obtain training feature data includes: Extract first capacitance feature data according to the first capacitance information; Label the first capacitance feature data according to the first protection state of the first electronic device to obtain first training feature data; Or, The touch sample data further includes auxiliary parameters; perform data fusion processing on the first capacitance feature data and the auxiliary parameters to obtain first fusion feature data; wherein, the auxiliary parameters include one or more of first charging state information, first attitude information, and first touch pressure information; Label the first fusion feature data according to the first protection state of the first electronic device to obtain first training feature data.

14. The method according to claim 13, wherein The training of the initial first model according to the training feature data to obtain the trained first model includes: Input the training feature data into the initial first model to obtain an output result; Update the weights of the initial first model based on the output result and the loss function to obtain the trained first model.

15. The method according to claim 14, wherein The first model refers to a convolutional neural network model; the loss function refers to a cross-entropy loss function.

16. The method according to claim 11, wherein After obtaining the trained first model, the method further includes: Perform a preset format conversion on the trained first model; Deploy the first model after the preset format conversion on a second electronic device.

17. An electronic device, characterized in that, The electronic device includes: One or more processors; A memory; And a computer program, where the computer program is stored on the memory, and when the computer program is executed by the one or more processors, the electronic device is caused to execute the capacitance sensor sensitivity threshold determination method according to any one of claims 1-10.

18. A server, characterized in that, The server includes: One or more processors; A memory; And a computer program, where the computer program is stored on the memory, and when the computer program is executed by the one or more processors, the electronic device is caused to execute the model training method according to any one of claims 11-16.

19. A computer storage medium, characterized in that, Includes computer instructions, and when the computer instructions run on an electronic device, the electronic device is caused to execute the capacitance sensor sensitivity threshold determination method according to any one of claims 1-10.

20. A computer storage medium, characterized in that, Includes computer instructions, and when the computer instructions run on a server, the server is caused to execute the model training method according to any one of claims 11-16.

Images