Correction parameter determination method and electronic equipment

Abstract

The invention provides a correction parameter determination method and electronic equipment. The method comprises the steps that when the handwriting pen makes contact with a screen of the electronic equipment, target contact data are obtained, and the target contact data at least comprise target capacitance information and charging state information; wherein the target capacitance information comprises multiple frames of target capacitance value information, and each frame of target capacitance value information is used for indicating the capacitance value of each capacitance value sensor in a capacitance value sensor array of the electronic equipment; according to the target capacitance information and the charging state information, the correction parameter corresponding to the electronic equipment is determined, that is, the electronic equipment can realize automatic adjustment of the correction parameter after the handwriting pen is in contact with the screen of the electronic equipment, so that the accuracy of the correction parameter in the electronic equipment in different use states is effectively improved, and the user experience is improved. And the subsequent electronic equipment can accurately correct the report point position according to the correction parameter, so that the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a correction parameter determination method and electronic device. Background Art

[0002] In the process of using the terminal, users often put a protective film on the terminal screen to protect the terminal screen. However, the protective film will affect the user's experience of using the stylus. Usually, the stylus handwriting correction parameters on the terminal are set according to the terminal's unfilmed state, and the stylus handwriting correction parameters are not applicable to handwriting correction in the terminal's filmed state. Moreover, the terminal cannot automatically sense the screen film state and cannot adjust the corresponding stylus handwriting correction parameters in time. Therefore, when the terminal screen is affixed with a protective film, when the user uses a stylus to write on the terminal screen, the handwriting often becomes distorted and crooked. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a correction parameter determination method, so that the electronic device can adaptively adjust the correction parameters corresponding to the usage status of the electronic device after the stylus touches the screen, so that the electronic device can accurately correct the reporting point position according to the correction parameters, thereby improving the user experience.

[0004] In a first aspect, the present application provides a correction parameter determination method, in which when a stylus pen contacts a screen of an electronic device, target contact data is obtained; the target contact data includes at least target capacitance information and charging status information; the target capacitance information includes multiple frames of target capacitance information; each frame of target capacitance information is used to indicate the capacitance of each capacitance sensor in a capacitance sensor array of the electronic device; based on the target capacitance information and the charging status information, the correction parameters corresponding to the electronic device are determined, that is, the electronic device can automatically adjust the correction parameters after the stylus pen contacts the screen of the electronic device, effectively improving the accuracy of the correction parameters in the electronic device in different usage states, so that subsequent electronic devices can accurately correct the reporting point position according to the correction parameters, thereby improving the user experience.

[0005] According to the first aspect, when the stylus is in contact with the screen of the electronic device, target contact data is obtained, including: obtaining multiple frames of target capacitance information based on a preset capacitance sampling rate; obtaining charging status information, and the charging status information is used to indicate whether the electronic device is being charged. In an embodiment of the present application, the target capacitance information in the target contact data includes multiple frames of target capacitance information, which can effectively characterize the contact characteristics when the stylus is in contact with the screen of the electronic device, and the charging status information will affect the capacitance information. Acquiring multiple frames of target capacitance information and charging status information can provide effective data support for the subsequent correction parameters corresponding to the accurate determination of the reporting point position.

[0006] According to the first aspect, or any implementation method of the first aspect above, a correction parameter corresponding to the electronic device is determined according to the target capacitance information and the charging status information, including: extracting target capacitance characteristic data according to the target capacitance information; performing data fusion processing on the target capacitance characteristic data and the charging status information to obtain target fusion characteristic data; performing feature enhancement processing on the target fusion characteristic data to obtain target usage characteristic data; inputting the target usage characteristic data into the first model to obtain the correction parameter output by the first model.

[0007] In the embodiment of the present application, the electronic device only needs to input the target usage characteristic data into the first model to obtain the correction parameters output by the first model, which simplifies the way in which the electronic device determines the correction parameters corresponding to the reporting point position and can increase the speed at which the electronic device adjusts the correction parameters.

[0008] According to the first aspect, or any implementation method of the first aspect above, the target contact data also includes target auxiliary data; the target capacitance characteristic data and the charging status information are subjected to data fusion processing to obtain target fusion characteristic data, including: the target capacitance characteristic data, the charging status information and the target auxiliary data are subjected to data fusion processing to obtain target fusion characteristic data; wherein the target auxiliary data includes one or more of target pressure information, target posture information and sensor index information; the target pressure information is used to indicate the pressure value when the stylus is in contact with the screen of the electronic device; the target posture information is used to indicate the posture of the electronic device; the sensor index information is used to indicate the coordinate information of the capacitance sensor.

[0009] In an embodiment of the present application, the target auxiliary data is used to indicate data that will affect the capacitance of the capacitance sensor. Therefore, involving the target auxiliary data in the process of determining the correction parameters can improve the accuracy of the determined correction parameters, making it easier for subsequent electronic devices to accurately correct the reporting point position according to the correction parameters, thereby improving the user experience.

[0010] According to the first aspect, or any implementation method of the first aspect above, target capacitance characteristic data is extracted according to the target capacitance information, including: selecting first target capacitance information from the target capacitance information according to a preset selection rule; extracting a capacitance matrix of a preset size from each frame of target capacitance information in the first target capacitance information, and combining the capacitance matrix into target capacitance characteristic data.

[0011] In the embodiment of the present application, the target capacitance information may contain a large number of target capacitance information, and too much data may cause the subsequent determination of the correction parameters to take too long. Therefore, it is necessary to preliminarily select appropriate target capacitance information to participate in the subsequent determination of the correction parameters. In addition, when the stylus touches the screen of the electronic device, the capacitance of the capacitance sensor at the contact position and near the contact position changes significantly, which can be regarded as valid data, and can effectively characterize the contact characteristics of the stylus and the screen of the electronic device. A capacitance matrix of a preset size can be extracted from the target capacitance information to achieve redundancy of the target capacitance information and improve data validity.

[0012] According to the first aspect, or any implementation of the first aspect above, the preset selection rule includes: in the target capacitance information, determining the second capacitance information, the second capacitance information includes the maximum capacitance in the target capacitance information; the second capacitance information is the Kth frame target capacitance information in the target capacitance information; selecting the [KN, K+N] frame target capacitance information, and determining the first target capacitance information based on the [KN, K+N] frame target capacitance information, where N is a preset value. In the embodiment of the present application, the number of target capacitance information included in the target capacitance information may be large, and excessive data volume may cause the subsequent determination of the correction parameters to take too long, so it is necessary to preliminarily select appropriate target capacitance information to participate in the subsequent process of determining the correction parameters.

[0013] According to the first aspect, or any implementation of the first aspect above, the preset selection rule also includes: selecting the target capacitance information of the [KM, K+M]th frame as the first target capacitance information in the [KN, K+N]th frame, M is less than N. In the embodiment of the present application, when the stylus touches the screen of the electronic device, the capacitance of the capacitance sensor is unstable during the process of the stylus approaching the screen of the electronic device and leaving the screen, and the capacitance information obtained by the electronic device is of poor validity. Therefore, it is necessary to remove the capacitance information of the first and last frames that are arranged in the preliminary selected first sample capacitance information, and filter out the capacitance information with higher data stability.

[0014] According to the first aspect, or any implementation of the first aspect above, the matrix center of the capacitance matrix indicates the maximum capacitance in the corresponding target capacitance information. In the embodiment of the present application, when the stylus touches the screen of the electronic device, the capacitance of the capacitance sensor at and near the contact position changes significantly, which can be regarded as valid data and can effectively characterize the contact characteristics between the stylus and the screen of the electronic device. Therefore, the maximum capacitance in the target capacitance information is determined as the matrix center of the capacitance matrix corresponding to the target capacitance information, thereby improving the data validity of the capacitance matrix.

[0015] According to the first aspect, or any implementation of the first aspect above, after determining the correction parameters corresponding to the electronic device according to the target capacitance information and the charging state information, it also includes: correcting the reporting point position according to the determined correction parameters. In the embodiment of the present application, after the reporting point position is corrected, the error between the handwriting of the stylus displayed based on the corrected reporting point position and the contact trajectory of the user on the screen of the electronic device is small, which can achieve handwriting optimization and improve user experience.

[0016] According to the first aspect, or any implementation of the first aspect above, the electronic device is connected to the stylus pen via Bluetooth.

[0017] In a second aspect, the present application provides a model training method, which is applied to a server. The method includes: obtaining contact sample data and correction parameters of an electronic device in multiple usage states; the contact sample data includes at least capacitance information, charging state information and reporting point information; the capacitance information includes multiple frames of capacitance information; each frame of capacitance information is used to indicate the capacitance of each capacitance sensor in the capacitance sensor array of the electronic device; based on the contact sample data and the correction parameters, obtaining training feature data; according to the training feature data, training an initial first model to obtain a trained first model; the trained first model is used to determine the correction parameters.

[0018] In an embodiment of the present application, the first model is trained using contact sample data and correction parameters of the electronic device in multiple usage states, so that the trained first model can have the function of determining the correction parameters in the electronic device.

[0019] According to the second aspect, the above-mentioned usage states at least include: a state with a film attached and charged, a state without a film attached and charged, a state with a film attached and not charged, and a state without a film attached and not charged. In the embodiment of the present application, the more usage states there are, the richer the scenarios corresponding to the contact sample data, which facilitates the subsequent first model trained according to the contact sample data to adapt to more rich scenarios.

[0020] According to the second aspect, or any implementation method of the second aspect above, when the electronic device is in any usage state, contact sample data and correction parameters of the electronic device in multiple usage states are obtained, including: receiving first capacitance information sent by the electronic device; the first capacitance information includes multiple frames of first capacitance information; receiving first charging status information sent by the electronic device; receiving first reporting point information sent by the electronic device; and determining a first correction parameter based on the first reporting point information.

[0021] In the embodiment of the present application, the capacitance of the capacitance sensor can effectively characterize the contact characteristics between the stylus and the electronic device, and the charging status has a greater impact on the capacitance of the capacitance sensor. Therefore, using the capacitance of the capacitance sensor and the charging status information as the contact sample data can improve the validity of the contact sample data.

[0022] According to the second aspect, or any implementation of the second aspect above, based on the first reporting point information, determining the first correction parameter includes: establishing a coordinate relationship between theoretical handwriting and actual handwriting according to the first reporting point information; the first reporting point information includes a set of reporting point positions; determining a compensation value for each reporting point position according to the coordinate relationship between the theoretical handwriting and the actual handwriting, and determining the compensation value for each reporting point position as the first correction parameter. In the embodiment of the present application, the correction parameter corresponding to the use state can be determined according to the first reporting point information, so as to facilitate subsequent data annotation according to the correction parameter, so as to measure the accuracy of the first model output as a real result.

[0023] According to the second aspect, or any implementation of the second aspect above, training feature data is obtained based on contact sample data and correction parameters, including: extracting first capacitance feature data based on first capacitance information; performing data fusion processing on the first capacitance feature data and the first charging state information to obtain first fused feature data; performing feature enhancement processing on the first fused feature data to obtain first usage feature data; and labeling the first usage feature data based on the first correction parameter to obtain first training feature data.

[0024] In the embodiment of the present application, the capacitance of the capacitance sensor can effectively characterize the contact characteristics between the stylus and the electronic device, and the charging status has a greater impact on the capacitance of the capacitance sensor. Therefore, using the capacitance of the capacitance sensor and the charging status information as the contact sample data can improve the validity of the contact sample data.

[0025] According to the second aspect, or any implementation of the second aspect above, the initial first model is trained according to the training feature data to obtain the trained first model, including: inputting the training feature data into the initial first model to obtain an output result; based on the output result and the loss function, updating the weight of the initial first model, so that the error between the correction parameter (output result) predicted by the first model and the correction parameter (actual result) indicated by the input data label is less than a preset error, thereby obtaining the trained first model. Obtain the trained first model.

[0026] According to the second aspect, or any implementation of the second aspect above, the first model is indicated as a regression network model based on deep learning; and the loss function is indicated as focal loss.

[0027] According to the second aspect, or any implementation of the second aspect above, after training the initial first model according to the training feature data to obtain the trained first model, the method also includes: performing a preset format conversion on the trained first model to convert the trained first model into a lightweight model that can be adapted to the mobile terminal; deploying the first model converted to the preset format on an electronic device so that the electronic device has the function of automatically sensing and adjusting correction parameters. For example, when a user uses a stylus to perform a touch operation on the screen of the electronic device, the electronic device can obtain contact data based on the contact operation, and input the contact data after data processing into the pre-deployed first model to obtain the correction parameters output by the first model.

[0028] In a third aspect, the present application provides an electronic device, comprising: one or more processors; a memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by one or more processors, the electronic device executes the correction parameter determination method as in the first aspect, or any one of the implementations of the first aspect above.

[0029] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0030] In a fourth aspect, the present application provides a server comprising: one or more processors; a memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by one or more processors, the electronic device executes the model training method as in the second aspect.

[0031] The technical effects corresponding to the fourth aspect can refer to the technical effects corresponding to the above-mentioned second aspect, and will not be repeated here.

[0032] In a fifth aspect, the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes a correction parameter determination method as in the first aspect, or any one of the implementations of the first aspect above.

[0033] The fifth aspect corresponds to the first aspect and any implementation of the first aspect. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0034] In a sixth aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the model training method as in the second aspect.

[0035] The technical effects corresponding to the sixth aspect can refer to the technical effects corresponding to the above-mentioned second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0038] Figure 3 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of a communication connection between an electronic device and a stylus provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of data interaction between an electronic device and a stylus provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of the capacitance of a capacitance sensor for detecting when a screen of an electronic device has different film layer states provided by an embodiment of the present application;

[0042] Figure 7 A schematic diagram of obtaining capacitance information of an electronic device provided in an embodiment of the present application;

[0043] Figure 8 A schematic diagram of a flow chart of a first model training method provided in an embodiment of the present application;

[0044] Fig. 9 An example diagram of a contact sample data acquisition scenario provided in an embodiment of the present application;

[0045] Fig.10 A schematic diagram of determining handwriting correction parameters provided in an embodiment of the present application;

[0046] Fig.11 An example diagram of extracting capacitance characteristic data provided by an embodiment of the present application;

[0047] Fig.12 A schematic diagram of a model training provided in an embodiment of the present application;

[0048] Fig.13 A flowchart of a first model deployment method provided in an embodiment of the present application;

[0049] Fig.14 A flowchart of a method for determining a correction parameter provided in an embodiment of the present application

[0050] Fig.15 A schematic diagram of an application effect provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0053] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0054] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0056] When using electronic devices, users often put a protective film on the screen to protect the screen of the electronic device. The protective film can be a tempered film, a paper-like film, etc. However, the protective film will affect the user's use of the stylus. Among them, the stylus (also called a stylus or an active pen) is a device used to write on an electronic device.

[0057] Figure 1A schematic diagram of a scenario provided for an embodiment of the present application. Before introducing the embodiment of the present application, firstly based on Figure 1 An application scenario of an embodiment of the present application is described, and the application scenario includes an electronic device 100 and a stylus pen 101 .

[0058] Usually, the handwriting correction parameters of the stylus on the electronic device are set according to the state of the electronic device without film. Figure 1 As shown, in Figure 1 In the scenario shown in (1), the screen of the electronic device 100 is not covered with a protective film. The user uses the stylus 101 to perform a touch operation on the screen of the electronic device 100 that is not covered with a protective film, and the touch track of the touch operation is a straight line. The display interface 102 responds to the touch operation of the stylus 101, and the handwriting 103 displayed is a straight line corresponding to the touch track.

[0059] exist Figure 1 In the scenario shown in (2), after the screen of the electronic device 100 is pasted with a protective film 104, the user uses the stylus 101 to perform the same contact operation as shown in the above (1) figure on the screen pasted with the protective film 104, and the handwriting 106 displayed on the display interface 105 in response to the contact operation of the stylus 101 is a distorted and crooked line. This is because after the screen of the electronic device 100 is pasted with the protective film 104, the handwriting correction parameters of the electronic device's stylus are not adjusted as the state of the electronic device's film is changed, and the handwriting correction parameters of the electronic device before the electronic device is pasted with the film are not applicable to the handwriting correction in the state of the electronic device's film, resulting in the handwriting 106 displayed on the display interface 105 in response to the contact operation of the stylus 101, not a straight line corresponding to the contact trajectory, but a distorted and crooked line.

[0060] In order to solve the above problems, the embodiments of the present application provide a first model training method, a deployment method, and a correction parameter determination method, so that the electronic device can accurately determine the correction parameters corresponding to the usage status of the electronic device after the stylus touches the screen, so that the electronic device can accurately correct the stylus handwriting according to the correction parameters, thereby improving the user experience.

[0061] The data transmission method provided in the embodiments of the present application can be applied to electronic devices, and the electronic devices can be portable computers (such as mobile phones), tablet computers, laptop computers, personal computers (PCs), augmented reality (AR) and virtual reality (VR) devices, car computers and other devices. The following embodiments do not impose any special restrictions on the specific form of the electronic devices.

[0062] Before describing the technical solution of the embodiment of the present application, the electronic device of the embodiment of the present application is first described with reference to the accompanying drawings. Figure 2 This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. It should be understood that: Figure 2 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 2 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0063] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a capacitance sensor 180M, etc.

[0064] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0065] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0066] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0067] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0068] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0069] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0070] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0071] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0072] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0073] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0074] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0075] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0076] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0077] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0078] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0079] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0080] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0081] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0082] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0083] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0084] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0085] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0086] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0087] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0088] The capacitance sensor 180M is used to change the capacitance according to the contact operation of the stylus pen acting on or near it, and the contact operation is, for example, a touch operation, a writing operation, a drawing operation, etc. The capacitance sensor can transmit the detected contact operation of the stylus pen to the application processor, and provide a visual output related to the contact operation through the display screen 194.

[0089] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0090] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Indicator 192 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0091] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0092] Figure 3 A software structure block diagram of an electronic device 100 provided in an embodiment of the present application.

[0093] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the framework layer, the Android runtime (Android runtime), the hardware abstraction layer (HAL), the system library and the kernel layer.

[0094] The application layer may include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0095] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer, including various components and services to support developers' Android development. The application framework layer includes some predefined functions.

[0096] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a stylus management service, and the like.

[0097] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0098] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0099] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0100] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0101] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0102] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0103] The stylus management service may also be referred to as a stylus management service, which is used to manage a stylus connected to an electronic device, including controlling connection and disconnection with the stylus.

[0104] The Hardware Abstraction Layer (HAL) is an interface layer between the operating system kernel and the hardware circuit. The hardware abstraction layer provides a virtual hardware platform for the operating system. The operating system kernel or the hardware driver can call the hardware abstraction layer. Figure 3 As shown, in the embodiment of the present application, the hardware abstraction layer includes a data processing module, a first model, and a handwriting correction module.

[0105] The data processing module is used to perform corresponding data processing operations on the received data, such as data extraction processing, data fusion processing, etc.

[0106] The first model is used to predict calibration parameters of the electronic device.

[0107] The handwriting correction module is used to correct the reporting point position according to the correction parameters output by the first model.

[0108] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0109] Understandably, Figure 3 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0110] Figure 4 A schematic diagram of a communication connection between an electronic device and a stylus pen provided in an embodiment of the present application. Figure 4 As shown, the stylus pen 200 includes but is not limited to: a microcontroller unit (MCU) 201, a pressure sensor 202, a Bluetooth integrated circuit (IC) 203, a Bluetooth antenna 204, an electrode 205 and other components. It can be understood that Figure 4 The components included in the stylus pen 200 are shown, but do not constitute a specific limitation on the stylus pen 200. In other embodiments of the present application, the stylus pen 200 may include more or fewer components than shown, or combine or separate some components, or arrange the components differently.

[0111] Optionally, the pressure sensor 202 is used to obtain pressure parameters. For example, when the tip of the stylus touches an object (such as a touch screen), the pressure sensor 202 can obtain a corresponding pressure value based on the pressure received, and report the pressure value to the MCU 201. Optionally, the pressure value can also be called a pressure parameter, pressure data, pressure information, etc., which is not limited in this application.

[0112] Optionally, the MCU 201 is used to process the pressure parameters received from the pressure sensor 202. It should be noted that the MCU 201 is only an illustrative example, and other devices that can function as a processor or a microcontroller can be alternative forms of the above MCU.

[0113] Optionally, the MCU 201 is further used to control the Bluetooth antenna 204 to output a pressure-sensing signal, and the pressure-sensing signal is used to indicate the pressure parameter obtained by the MCU 201 from the pressure sensor 202. Exemplarily, the MCU 201 may generate a pressure-sensing signal based on the pressure parameter obtained from the pressure sensor 202, and output the pressure-sensing signal to the electronic device 210 through the Bluetooth IC 203 and the Bluetooth antenna 204. Accordingly, in some embodiments, the antenna (e.g. Figure 2 The antenna 2 in the electronic device 210 can obtain the corresponding pressure parameter based on the received pressure sensing signal, and transmit the obtained pressure parameter to the Bluetooth driver, which then reports the pressure parameter to the upper layer module (e.g., the data processing module of the HAL layer), and the upper layer module can process the pressure parameter accordingly. Figure 2 The antenna 2) in the apparatus can send the received pressure sensing signal to other devices, such as a server or other electronic devices.

[0114] Optionally, MCU201 is also used to receive and process data from electrode 205. Exemplarily, electrode 205 may be located at the tip side of the electronic device, and exemplarily, when the tip of stylus 200 is close to the screen of the electronic device, electrode 205 of the stylus may output a signal with a specified frequency (which may be called a radio frequency frequency).

[0115] Optionally, the MCU 201 , the pressure sensor 202 , the Bluetooth IC 203 , and the Bluetooth antenna 204 may be integrated into the same chip, or may be separate components and connected via a bus.

[0116] In some embodiments, the stylus of the present application may also use other short-range wireless technologies to exchange data with electronic devices such as electronic devices, such as Wi-Fi technology, Ultra Wide Band (UWB) and other short-range wireless communication technologies. It can be understood that when the stylus 200 and the electronic device transmit pressure information or square wave signals through Wi-Fi technology, both the stylus 200 and the electronic device 210 have Wi-Fi modules.

[0117] Figure 5 A schematic diagram of data interaction between an electronic device and a stylus provided in an embodiment of the present application.

[0118] Reference Figure 5 As shown in (1), after the stylus pen 101 establishes a Bluetooth connection with the electronic device 100, and the distance d between the stylus pen 101 and the screen of the electronic device 100 is less than the preset distance D, the stylus pen 101 will receive an uplink signal sent by the electronic device 100, and the uplink signal is used to instruct the electronic device to send a pulse signal, and the preset distance D can be determined based on the transmission power of the uplink signal. The stylus pen responds to the uplink signal and sends a pulse signal to the electronic device. The frequency of the pulse signal sent by the electronic device can be set according to the actual application. For example, the frequency of the pulse signal sent by the electronic device can be 200KHz or 300KHz, which is not specifically limited in the embodiments of the present application.

[0119] The pulse signal sent by the stylus pen will change the capacitance of the capacitance sensor in the electronic device that is closer to the stylus pen tip. The detection timing control signal of the electronic device is a signal used to control the electronic device to detect the capacitance of the capacitance sensor. Figure 5 As shown in (2), the detection timing control signal includes a high level and a low level, and the electronic device detects the capacitance of the capacitance sensor when the detection timing control signal is at a high level. The frequency of the detection timing control signal can be set according to actual application conditions, for example, to 240 Hz, 1.5 KHz, or other values, which is not limited in the embodiments of the present application.

[0120] In some examples, the electronic device can determine the reporting point position by detecting the capacitance of each capacitance sensor, and the reporting point position is used to indicate the contact position between the stylus and the electronic device screen determined by the electronic device. It should be noted that due to reasons such as a protective film attached to the screen of the electronic device, there may be a slight deviation between the actual contact position of the stylus on the screen of the electronic device and the contact position (reporting point position) between the stylus and the electronic device screen determined by the electronic device.

[0121] In some examples, a first coordinate system is established with the screen edge of the electronic device as the coordinate axis (for example, the longer side of the screen is used as the horizontal axis and the shorter side of the screen is used as the vertical axis), and the reported point position can be represented by the coordinates in the first coordinate system. Figure 5 As shown in (2), when the detection timing control signal is at a high level e, the electronic device detects the capacitance of each capacitance sensor, and then determines the horizontal coordinate x1 of the reporting point position 1 through the capacitance detected at the high level e; when the detection timing control signal is at a high level f, the electronic device detects the capacitance of each capacitance sensor, and then determines the vertical coordinate y1 of the reporting point position 1 through the capacitance detected at the high level f. Similarly, the electronic device can also determine the coordinates (x2, y2) of the reporting point position 2.

[0122] The pulse signal sent by the stylus will change the capacitance of the capacitance sensor in the electronic device that is closer to the stylus tip. The magnitude of the capacitance change will be affected by many factors, for example, whether the screen of the electronic device has a first film layer will affect the magnitude of the capacitance change, and the first film layer indicates a protective film of the electronic device screen.

[0123] Figure 6 A capacitance diagram of a capacitance sensor for detecting when a screen of an electronic device has different film layer states provided in an embodiment of the present application. The following description is made by taking the stylus 101 performing the same contact operation at the same position on the screen of an electronic device 100 without a first film layer and on the screen of an electronic device 100 with a first film layer as an example.

[0124] like Figure 6 As shown in (1) and (2), in a of the side view (1) and c of (2) of the electronic device 100, a capacitance sensor array 108 is provided in the screen assembly of the electronic device 100. As can be seen in b of (1) and d of (2) of the top view of the electronic device 100, the capacitance sensor array 108 includes a plurality of capacitance sensors 1081 arranged in rows and columns, for example, capacitance sensor 1081-1 and capacitance sensor 1081-2 shown in b or d. The specific values ​​of each capacitance in the following examples are all exemplary descriptions, and the unit of capacitance is pico-farad (pF).

[0125] like Figure 6 As shown in (1), in the scenario shown in a of (1) where the screen of the electronic device 100 does not have the first film layer, the tip of the stylus pen 101 contacts the screen of the electronic device 100. The electronic device 100 detects the capacitance of each capacitance sensor, and the capacitance of the capacitance sensor at the contact position between the tip of the stylus pen 101 and the screen and the contact position is as shown in FIG. Figure 6As shown in 108-1 in b of (1), each square is used to indicate a capacitance sensor, and the numbers in the squares are used to indicate the capacitance of the capacitance sensor. For example, the capacitance detected by capacitance sensor 1081-1 is 2560, and the capacitance detected by capacitance sensor 1081-2 is 112. It should be noted that in the embodiment of the present application, the interval between the capacitance sensors corresponding to the contact position does not exceed a preset number (for example, 5) of capacitance sensors, which can be regarded as "near the contact position".

[0126] like Figure 6 As shown in (2), in the scene shown in c of (2) where the screen of the electronic device 100 has the first film layer 104, the tip of the stylus pen 101 contacts the first film layer 104 on the screen of the electronic device 100. It should be noted that Figure 6 In (2), the position of the stylus pen 101 tip corresponding to the screen in c is the same as the position of the stylus pen 101 tip corresponding to the screen in a of (1), wherein the position of the stylus pen 101 tip corresponding to the screen refers to: the projection position of the stylus pen 101 tip in the first coordinate system. The first coordinate system is a coordinate system established with the screen edge of the electronic device as the coordinate axis (for example, the longer side of the screen is the horizontal axis and the shorter side of the screen is the vertical axis).

[0127] The electronic device detects the capacitance of each capacitance sensor. The capacitance of the capacitance sensor near the contact position between the tip of the stylus pen 101 and the first film layer 104 on the screen is as follows: Figure 6 As shown in 108-2 in d of (2), each square is used to indicate a capacitance sensor, and the numbers in the squares are used to represent the capacitance of the capacitance sensor detected by the electronic device. For example, the capacitance of the capacitance sensor 1081-1 is 2204, and the capacitance of the capacitance sensor 1081-2 is 103.

[0128] Each capacitance value varies, taking 108-1 and 108-2 as examples. Comparing the capacitance value distribution shown in 108-1 with the capacitance value distribution shown in 108-2, the capacitance value of capacitance sensor 1081-1 is 2560 when the screen of electronic device 100 does not have the first film layer, and the capacitance value is 2204 when the screen has the first film layer, while the capacitance value of capacitance sensor 1081-2 is 112 when the screen of electronic device 100 does not have the first film layer, and the capacitance value is 99 when the screen has the first film layer. That is, the first film layer on the screen of the electronic device will affect the capacitance value change range of the capacitance sensor.

[0129] For the same contact operation of the stylus on the screen of an electronic device with different film layer states, the distance between the tip of the stylus and each capacitance sensor will increase when the screen of the electronic device has a first film layer compared to when the screen does not have the first film layer, and the first film layer is added between the stylus and the capacitance sensor. The first film layer has a certain dielectric constant, so the dielectric constant between the stylus and the capacitance sensor will change, resulting in the capacitance distribution detected when the electronic device has the first film layer on the screen being completely different from the capacitance distribution detected when the electronic device does not have the first film layer on the screen.

[0130] The above embodiments only show the contact position of the stylus pen and the screen and the capacitance of the capacitance sensor near the contact position detected by the electronic device when the stylus pen contacts the screen of the electronic device. However, in actual application, during the contact between the stylus pen and the screen of the electronic device, the electronic device will obtain the capacitance of each capacitance sensor multiple times. Each time the capacitance of each capacitance sensor is obtained, a frame of capacitance information is formed.

[0131] Figure 7 A schematic diagram of an electronic device obtaining capacitance information provided by an embodiment of the present application. Figure 7 As shown in (1), as time t passes, the electronic device 100 can obtain capacitance information according to a preset capacitance sampling rate, where the capacitance sampling rate indicates the frequency at which the electronic device obtains capacitance information. The capacitance sampling rate can be set according to actual application conditions, for example, to 240 Hz, 1.5 kHz, or other values, which are not limited in the embodiments of the present application. Each frame of capacitance information includes the capacitance of each capacitance sensor in the capacitance sensor array 108 at the corresponding acquisition time. The capacitance information, for example Figure 7 (1) shows the capacitance information 700 acquired at time t0, the capacitance information 701 acquired at time t1, the capacitance information 702 acquired at time t2, etc.

[0132] by Figure 7Taking the capacitance information 701 obtained at time t1 shown in (2) as an example, the distribution of capacitance values ​​in a frame of capacitance information is exemplified. The capacitance information 701 includes the capacitance values ​​of each capacitance sensor at time t1. In the capacitance information 701, the contact position between the tip of the stylus pen 101 and the screen and the capacitance values ​​of the capacitance sensors near the contact position are shown in 701-1, wherein each square represents a capacitance sensor, and the number in each square represents the capacitance value of the capacitance sensor, for example, the capacitance value of capacitance sensor 701-10 is 2800, and the capacitance value of capacitance sensor 701-11 is 78. The capacitance value of the capacitance sensor relatively far from the contact position between the stylus pen tip and the screen is shown in 701-2, for example, the capacitance value of capacitance sensor 701-20 is 2, and the capacitance value of capacitance sensor 701-21 is 1. It can be seen that the closer the distance between the capacitance sensor and the tip of the stylus pen is, the larger the capacitance value of the capacitance sensor is.

[0133] It can be seen from the capacitance distribution shown in 701-1 and 701-2 that in the capacitance distribution shown in 701-2, the capacitance of each capacitance sensor is less than a preset value (preset value such as 50), which is a parameter used to measure whether the capacitance of the capacitance sensor is a noise value, and these capacitance values ​​less than the preset value will be regarded as noise values. In the capacitance distribution shown in 701-1, the capacitance of each capacitance sensor is greater than the preset capacitance value (preset capacitance value such as 50), which can effectively characterize the contact characteristics between the stylus and the screen of the electronic device and can be regarded as valid information.

[0134] Figure 8 A flowchart of a first model training method provided in an embodiment of the present application. The first model training method is applied to a server, and the server includes a preprocessing module and a training module. The first model is used to predict correction parameters. Figure 8 As shown, the first model training method includes: steps S81 to S85.

[0135] Step S81: The preprocessing module obtains contact sample data and handwriting correction parameters of the electronic device in multiple usage states.

[0136] The use status of the electronic device includes at least: a state with a film attached and being charged, a state without a film attached and being charged, a state with a film attached and not being charged, and a state without a film attached and not being charged.

[0137] In some embodiments, the usage status of the electronic device further includes one or more of the posture of the electronic device, the thickness of the protective film, the dielectric constant of the protective film, and the contact pressure when the stylus touches the screen of the electronic device.

[0138] The contact sample data is used to indicate the device status data acquired by the electronic device and the data generated by the contact with the stylus during the process of the stylus contacting the electronic device screen. The contact sample data at least includes charging status information, capacitance information and reporting point information, wherein the charging status information is used to indicate whether the electronic device is being charged; the capacitance information includes multi-frame capacitance information; the reporting point information indicates a coordinate set of the reporting point position. The reporting point position is the contact position between the stylus and the electronic device screen determined by the electronic device.

[0139] In some examples, the contact sample data also includes auxiliary parameters, which indicate parameters that affect the capacitance of the capacitance sensor when the stylus touches the screen of the electronic device. The auxiliary parameters include: one or more of posture information, pressure information, sensor index information, the thickness of the first film layer, and the dielectric constant of the first film layer. Among them, the posture information is used to characterize the posture of the electronic device, and the posture of the electronic device will affect the capacitance of the capacitance sensor when the stylus touches the screen of the electronic device; the pressure information is used to characterize the pressure value when the stylus touches the screen of the electronic device. When the pressure value is larger, the distance between the tip of the stylus and the capacitance sensor in the electronic device is closer, and the capacitance of the capacitance sensor will change accordingly; the thickness of the first film layer will affect the distance between the stylus and the capacitance sensor when the stylus touches the screen of the electronic device, thereby affecting the capacitance of the capacitance sensor. The dielectric constant of the first film layer will also affect the capacitance of the capacitance sensor when the stylus touches the screen of the electronic device.

[0140] Fig. 9 An example diagram of a contact sample data acquisition scenario provided in an embodiment of the present application. The acquisition scenario includes an electronic device 100, a stylus 101, a mechanical arm 109, and a test device (not shown in the figure). Among them, the test device is connected to the mechanical arm 109, and is used to control the movement of the mechanical arm 109 according to the control instructions input by the tester. The mechanical arm 109 is used to fix the stylus 101, and drive the stylus 101 to touch the screen of the electronic device 100 according to the control instructions of the test device. For example, the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, and the contact operation includes a uniform movement operation on the screen after the tip of the stylus 101 touches the screen, wherein the contact trajectory is a straight line segment. It should be noted that the mechanical arm 109 can make the stylus 101 touch a preset position on the screen of the electronic device 100 with a preset pressure value in different acquisition scenarios.

[0141] Example 1: Fig. 9As shown in (1), the electronic device is in the following usage states: no film applied, 45° with the horizontal plane, and not charged. The mechanical arm 109 drives the stylus 101 to write a straight line, such as the straight line segment 900, on the screen of the electronic device 100. The electronic device 100 obtains posture information through the gyroscope sensor, and obtains capacitance information and point information through the capacitance sensor. The stylus 101 obtains pressure information and sends the pressure information to the electronic device.

[0142] Example 2: Fig. 9 As shown in (2), the electronic device is in the following usage state: no film applied, an angle of 0° with the horizontal plane, and not charged. The process of obtaining the contact sample data is not described here.

[0143] Example 3: Fig. 9 As shown in (3), the electronic device is in the following usage state: film-applied state, angle of 45° with the horizontal plane, and uncharged state. The process of obtaining contact sample data is not described here.

[0144] Example 4: Fig. 9 As shown in (4), the electronic device is in the following usage state: no film attached, an angle of 0° with the horizontal plane, and not charged. The process of obtaining the contact sample data is not described here.

[0145] Example 5: Fig. 9 As shown in (5), the electronic device is in the following usage states: film-applied state, angle of 45° with the horizontal plane, and charged state. The process of acquiring contact sample data is not described here in detail.

[0146] Example 6: Fig. 9 As shown in (6), the electronic device is in the following usage state: film-applied state, angle with the horizontal plane is 0°, and the electronic device is in the charged state. The process of obtaining the contact sample data is not described in detail here.

[0147] It is understandable that the above Fig. 9 The use state of the electronic device is only an example. In actual application, the use state may include more types. For example, in the use state, the posture of the electronic device may be an angle of 20° with the horizontal plane, or may be a posture represented by a heading angle (Yaw), a pitch angle (Pitch) and a roll angle (Roll). In addition, the preset position where the stylus 101 contacts the screen of the electronic device 100 may be multiple positions, such as Fig. 9 (1) The position of straight line segment 900 and the position of straight line segment 901.

[0148] In one embodiment, taking the electronic device in the first state as an example, the preprocessing module obtains the contact sample data and handwriting correction parameters of the electronic device in multiple use states. The first state can be any of the aforementioned film-applied and charged state, film-unapplied and charged state, film-applied and uncharged state, and film-unapplied and uncharged state, or Fig. 9 Any one of the use states shown in (1)-(6) in the above step S81 includes: step S811-step S816.

[0149] Step S811: The preprocessing module receives first capacitance information sent by the electronic device.

[0150] The first capacitance information refers to multi-frame capacitance information acquired by the electronic device when the stylus performs a contact operation on the screen of the electronic device in the first state.

[0151] The first state is Fig. 9 Taking the usage state shown in (1) as an example, the first capacitance information indicates that the electronic device obtains multiple frames of capacitance value information during the process in which the robotic arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100 and the contact trajectory is a straight line segment 900.

[0152] Step S812: The preprocessing module receives first pressure information sent by the electronic device.

[0153] The first capacitance information refers to a pressure value of the stylus contacting the screen acquired by the electronic device when the stylus performs a contact operation on the screen of the electronic device in the first state.

[0154] The first state is Fig. 9 Taking the usage state shown in (1) as an example, the first pressure information indicates the pressure information sent by the stylus 101 to the electronic device 100 when the robotic arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100 and the contact trajectory is a straight line segment 900.

[0155] Step S813: The preprocessing module receives first posture information sent by the electronic device.

[0156] The first capacitance information refers to the posture of the electronic device when the stylus performs a contact operation on the screen of the electronic device in the first state. The first posture information includes the values ​​of the heading angle, pitch angle and roll angle collected by the gyroscope sensor.

[0157] The first state is Fig. 9Taking the usage state shown in (1) as an example, the first posture information indicates the posture information obtained by the electronic device through the gyroscope sensor when the robotic arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, that is, the first posture information in this scenario is an angle of 45° with the horizontal plane.

[0158] It should be noted that when the stylus touches the screen of the electronic device, the posture of the electronic device has an impact on the capacitance of the capacitance sensor. The capacitance of the capacitance sensor is the main data for predicting the handwriting correction parameters of the electronic device in the future. Therefore, in order to improve the accuracy of the first model in the subsequent training, the posture information of the electronic device can be used as one of the data for training the first model.

[0159] Step S814: The preprocessing module receives first charging status information sent by the electronic device.

[0160] The first capacitance information refers to whether the electronic device is being charged when the stylus is in contact with the screen of the electronic device in the first state. The first charging state information includes whether it is charged or not charged.

[0161] The first state is Fig. 9 Taking the usage status shown in (1) as an example, the first charging status information indicates whether the electronic device is being charged when the robotic arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, that is, the first charging status information in this scenario is not being charged.

[0162] It should be noted that when the stylus touches the electronic screen, whether the electronic device is being charged will affect the capacitance of the capacitance sensor. Therefore, the capacitance of the capacitance sensor is the main data for subsequent prediction of the handwriting correction parameters of the electronic device. Therefore, in order to improve the accuracy of the subsequent trained first model, the charging status information of the electronic device can be used as one of the data for training the first model.

[0163] Step S815: The preprocessing module receives the first reporting point information sent by the electronic device.

[0164] The first reporting point information indicates a coordinate set of the reporting point position determined by the electronic device when the stylus performs a contact operation on the screen of the electronic device in the first state.

[0165] The first state is Fig. 9 Taking the usage state shown in (1) as an example, the first reporting point information indicates a coordinate set of the reporting point position determined by the electronic device when the robotic arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100 and the contact trajectory is a straight line segment 900.

[0166] Step S816: The preprocessing module obtains the film thickness of the first film layer and / or the dielectric constant of the first film layer.

[0167] The film thickness of the first film layer and / or the dielectric constant of the first film layer are data input into the preprocessing module of the server by the tester. The film thickness of the first film layer and the dielectric constant of the first film layer can be obtained according to actual tests, or can be determined according to product parameters given by the manufacturer of the first model.

[0168] It should be noted that the order of description of the above steps S811-step S816 does not constitute a limitation on the order of execution of the steps. Also, any of the above steps S812, step S813 and step S816 can be selected to be executed or not executed according to the actual application situation. In some examples, the first contact sample data of the electronic device obtained by the preprocessing module in the server includes the above-mentioned first capacitance information, first charging status information and first reporting point information. In other examples, the first contact sample data of the electronic device obtained by the preprocessing module in the server includes the above-mentioned first capacitance information, first pressure information, first posture information, first charging status information, first reporting point information, film thickness and dielectric constant of the first film layer.

[0169] Step S817: The preprocessing module determines the first handwriting correction parameter based on the first reporting point information.

[0170] Fig.10 A schematic diagram of determining handwriting correction parameters provided in an embodiment of the present application. Fig.10 As shown in (1), the mechanical arm 109 fixes the stylus 101 and controls the stylus 101 to perform a contact operation on the screen of the electronic device 100 according to the control instruction of the test device, wherein the contact track is a straight line segment 900, and the electronic device 100 obtains the first point reporting information and sends the first point reporting information to the preprocessing module in the server. The step of determining the handwriting correction parameter based on the first point reporting information by the preprocessing module in the server includes the following steps 1 and 2.

[0171] Step 1: Establish the coordinate relationship between the theoretical handwriting and the actual handwriting according to the first reported point information.

[0172] Among them, the actual handwriting is as follows Fig.10 (2) is shown as the straight line segment 903, and the theoretical handwriting is shown as the straight line segment 902.

[0173] The theoretical handwriting is determined according to a plurality of preset coordinates for indicating the movement of the robot arm 109: the coordinate value corresponding to each preset coordinate in the first coordinate system is determined, and the coordinate values ​​corresponding to the preset coordinates are connected according to the movement sequence of the robot arm 109 to determine the theoretical handwriting. Fig.10As shown by the straight line segment 902 in (2).

[0174] The actual handwriting is determined by the first reporting information. The first reporting information includes the coordinate set of the reporting position determined by the electronic device during the process in which the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, wherein the contact track is a straight line segment 900. The reporting positions in the first reporting information are connected according to the time sequence of determining the reporting positions to determine the actual handwriting. The actual handwriting is as follows. Fig.10 As shown in line segment 903 in (2).

[0175] Step 2: Determine the compensation value of each reporting point position according to the coordinate relationship between the theoretical handwriting and the actual handwriting, and determine the compensation value of each reporting point position as the first handwriting correction parameter.

[0176] Among them, according to Fig.10 (2) The coordinate relationship between the theoretical handwriting and the actual handwriting is used to determine the compensation value of each reported point position. Fig.10 As shown in (3), the compensation value of each reported point position is equal to the difference between the actual handwriting position and the theoretical handwriting position. Fig.10 The reporting point position 9031 on the actual handwriting shown in (2) has actual coordinates (a, b). The contact position corresponding to this reporting point position on the theoretical handwriting is 9021, and its coordinates are (a+offsetx, b+offsety). Then the compensation value of this reporting point position 9031 is (offsetx, offsety).

[0177] In an embodiment of the present application, the preprocessing module receives contact sample data of the electronic device in the first state, and determines the handwriting correction parameters of the electronic device in the first state based on the contact sample data, so as to obtain a data basis for training the first model.

[0178] It can be understood that in the above embodiment, in steps S811 to S816, only the first state is Fig. 9 (1) shows an exemplary description of how the preprocessing module in the server obtains contact sample data and handwriting correction parameters in the usage state shown. However, it can be understood that in the scenario where the first state is other usage states, for example, the first state is Fig. 9 (2) The usage state shown, or the first state is Fig. 9 (3) When the usage status is shown, the way in which the preprocessing module in the server obtains the contact sample data and handwriting correction parameters is similar to that in the above embodiment, and will not be repeated in the embodiment of the present application.

[0179] Step S82: The preprocessing module performs a first data processing on the contact sample data to obtain usage feature data.

[0180] The first data processing includes but is not limited to data extraction processing, data fusion processing, etc.

[0181] In one implementation, the first data processing is described by taking the first contact sample data acquired by the aforementioned preprocessing module when the electronic device is in the first state as an example. The first contact sample data includes first capacitance information and first charging state information; in some embodiments, the first contact sample data also includes auxiliary parameters. The first data processing is performed on the contact sample data to obtain the usage characteristic data (the above step S82), including: steps S821-step S823.

[0182] Step S821: extract first capacitance characteristic data according to the first capacitance information.

[0183] The first capacitance information indicates multi-frame capacitance value information acquired by the electronic device during a process in which the stylus performs a contact operation on the screen of the electronic device in the first state.

[0184] In an embodiment of the present application, when extracting the first capacitance characteristic data according to the first capacitance information, it is necessary to extract a capacitance matrix from the capacitance information contained in the first capacitance information. For each frame of capacitance information in the first capacitance information, the position of the maximum capacitance is determined from the capacitance information, and the position of the maximum capacitance is the position of the capacitance sensor corresponding to the contact position. It should be noted that when the stylus touches the screen of the electronic device, the capacitance of the capacitance sensor at the contact position and near the contact position changes significantly, which can be regarded as valid data and can effectively characterize the contact characteristics between the stylus and the screen of the electronic device. In addition, the input data when training the first model is a matrix, and there are also restrictions on the length and width of the input data. Therefore, for each frame of capacitance information in the first capacitance information, according to the position of the maximum capacitance in the capacitance information and the length and width requirements of the input data, the corresponding matrix is ​​extracted from the capacitance information.

[0185] Fig.11 This is an example diagram of extracting capacitance feature data provided by an embodiment of the present application. Fig.11 In the embodiment, taking extracting the first capacitance characteristic data from the first capacitance information as an example, the method of extracting the capacitance characteristic data is exemplarily described.

[0186] like Fig.11 As shown, the first capacitance information 1100 includes multiple frames of first capacitance value information (such as Fig.11 The first capacitance information 601-613 in the first capacitance information is arranged in a default order of the acquisition time. In the embodiment of the present application, extracting the first capacitance characteristic data from the first capacitance information includes: the following steps 1 and 2.

[0187] Step 1: Select first sample capacitance value information from the first capacitance information according to a preset selection rule.

[0188] It should be noted that the first capacitance information may contain a large number of first capacitance information, and too much data may cause the subsequent training of the first model to take too long. Therefore, it is necessary to preliminarily select appropriate first capacitance information to participate in subsequent model training.

[0189] The preset selection rule includes: determining the third capacitance information in the first capacitance information, the third capacitance information includes the maximum capacitance in the first capacitance information; the third capacitance information is the target capacitance information of the Kth frame in the first capacitance information; selecting the first capacitance information of the [KN, K+N]th frame, and determining the first sample capacitance information based on the first capacitance information of the [KN, K+N]th frame. N is a preset value, which can be set according to actual usage, and is not specifically limited in the embodiment of the present application, for example, it can be set to 5, or it can be set to 6.

[0190] Reference Fig.11 , the third capacitance information determined is, for example, capacitance information 607, and the third capacitance information 607 includes the maximum capacitance in the first capacitance information, and the maximum capacitance in the first capacitance information is, for example, the capacitance 2800 of capacitance sensor 607-10 in capacitance information 607. The capacitance information 607 is the first capacitance information of the 7th frame in the first capacitance information 1100, and the first capacitance information of the [7-5, 7+5]th frame is selected (such as Fig.11 The first capacity value information 602-612 in the frame is used to determine the first sample capacity value information based on the first capacity value information of the [7-5, 7+5]th frame.

[0191] In one embodiment, in the above preset selection rule, when determining the first sample capacity value information based on the first capacity value information of the [KN, K+N]th frame, the first capacity value information of the [KN, K+N]th frame can be directly determined as the first sample capacity value information. Fig.11 For example, the first capacity value information (first capacity value information 602-612) of the [7-5, 7+5]th frame is determined as the first sample capacity value information.

[0192] In another embodiment, when the stylus touches the screen of the electronic device, the capacitance of the capacitance sensor is unstable during the process of the stylus approaching the screen of the electronic device and leaving the screen, and the capacitance information obtained by the electronic device is of poor validity. Therefore, it is necessary to remove the capacitance information of the first and last frames that are initially selected from the first sample capacitance information, and filter out the capacitance information with higher data stability. In this embodiment, the preset selection rule includes, and also includes: in the target capacitance information of the [KN, K+N]th frame, selecting the target capacitance information of the [KM, K+M]th frame as the first target capacitance information. M is less than N, and the value of M can be set according to the actual usage, and is not specifically limited in the embodiment of the present application. For example, it can be set to 1, or it can be set to 2.

[0193] by Fig.11 For example, from the first capacity information (first capacity information 602-612) of the [7-5, 7+5]th frame, the target capacity information (first capacity information 604-610) of the [7-3, 7+3]th frame is selected as the first target capacity information.

[0194] Step 2: extracting a capacitance matrix of a preset size from each frame of the first capacitance information in the first sample capacitance information, and combining the capacitance matrix into first capacitance feature data.

[0195] Taking the first capacitance information 607 obtained at time t1 as an example, the input data of the first model is a matrix, and when the length and width of the input data are required to be 7*7, the corresponding 7*7 matrix 607-1 is extracted from the first capacitance information 607. The matrix center of the matrix 607-1 is the maximum capacitance in the capacitance information 607, that is, the capacitance of the capacitance sensor 607-10 is 2800.

[0196] In the embodiment of the present application, for example, to improve the accuracy of training data, it is necessary to obtain corresponding 7*7 matrices from multiple frames of first capacitance information contained in the first sample capacitance information to form the first capacitance feature data. Fig.11 The first capacitance information 601-611 in the embodiment of the present invention are obtained from the first capacitance information 601-611, and the corresponding 7*7 matrix is ​​obtained. The capacitance matrix constitutes the first capacitance characteristic data 1101, and the size of the first capacitance characteristic data 1101 is 11×7×7.

[0197] It should be noted that when the contact time between the stylus and the screen of the electronic device is less than the preset duration, the electronic device cannot obtain enough frames of capacitance information, that is, the capacitance information contained in the first capacitance information will be less than the F frame. When the capacitance information obtained by the electronic device is less than the F frame, the stable capacitance information (the capacitance information of the [KM, K+M]th frame) contained in the capacitance information less than the F frame is not sufficient to support the first model training. The preset duration can be determined according to the capacitance sampling rate of the electronic device, such as 0.2s or 0.3s, which is not specifically limited in this embodiment. The value of F is 2N+1.

[0198] For example, it is taken that at least 7 frames of stable capacitance information are required to support the training of the first model. When the first capacitance information contains 11 frames of capacitance information, there are 7 frames of stable capacitance information (for example, the capacitance information remaining after removing the first 2 frames and the last 2 frames). When the first capacitance information only contains 7 frames of capacitance information, there are only 3 frames of stable capacitance information (for example, the capacitance information remaining after removing the first 2 frames and the last 2 frames), which cannot support the training of the first model. Therefore, in the above-mentioned second rule, it is necessary to preliminarily select (2N+1 frames) of capacitance information from the first capacitance information, and then discard the capacitance information sorted in the last (NM) frame after the first (NM) frame from the (2N+1 frame) capacitance information. If it is not possible to preliminarily select (2*N+1 frames) of capacitance information from the first capacitance information, the first capacitance information can be discarded and not used for model training.

[0199] Step S822: performing data fusion processing on the first capacitance characteristic data and the first charging state information to obtain first fused characteristic data.

[0200] Among them, data fusion processing is used to indicate a multi-level process of processing the association of data and information from single and multiple sources. The data fusion processing methods include but are not limited to projection, splicing, addition, etc. In the actual application process, the specific data fusion processing method can be selected according to the specific situation, and is not limited in the embodiments of this application.

[0201] In a possible implementation, when the contact sample data also includes auxiliary parameters, the preprocessing module may perform data fusion processing on the first capacitance characteristic data, the first pressure information and the auxiliary parameters to obtain first fused characteristic data. The auxiliary parameters include: posture information, pressure information, the film thickness of the first film layer, and one or more of the dielectric constant of the first film layer.

[0202] It should be noted that the auxiliary parameter indicates a parameter that affects the capacitance of the capacitance sensor when the stylus touches the screen of the electronic device. Therefore, the accuracy of the trained first model can be improved by incorporating the auxiliary parameter into the training process of the first model.

[0203] Step S823: perform feature enhancement processing on the first fused feature data to obtain first usage feature data.

[0204] Among them, feature enhancement processing is used to achieve dimensionality expansion of feature data.

[0205] In one example, the first fused feature data is subjected to feature enhancement processing, including: normalizing the first fused feature data; combining the first fused feature data and the normalized first fused feature data into two-dimensional feature data; taking the natural logarithm of the two-dimensional feature data; combining the two-dimensional feature data and the natural logarithm of the two-dimensional feature data into three-dimensional feature data, and the three-dimensional feature data is the obtained first usage feature data. Among them, the normalization method can be selected according to the actual application, for example, data type normalization or format normalization, etc., which is not specifically limited in the embodiments of the present application.

[0206] In this example, feature enhancement processing is performed on the first fused feature data, which can reduce invalid data in the first fused feature data, prevent underfitting problems in the trained first model, and make the trained first model have stronger generalization ability.

[0207] In this embodiment, through the above steps S821 to S823, usage feature data is obtained based on the contact sample data. Compared with the contact sample data, the usage feature data contains less invalid data, which is convenient for model training.

[0208] Step S83: The preprocessing module annotates the usage feature data based on the handwriting correction parameters in the first usage state to obtain training feature data.

[0209] The preprocessing module labels the usage feature data based on the handwriting correction parameters in the first usage state to obtain the training feature data, including: the preprocessing module adds the corresponding handwriting correction parameters as labels to each usage feature data to obtain the training feature data.

[0210] It should be noted that, since the handwriting correction parameters of the electronic device are different in different usage states, the handwriting correction parameter labels added to the usage characteristic data obtained in different usage states are also different. For example, when the usage state of the electronic device is the film-applied and charging state, the handwriting correction parameter of the electronic device is the first correction parameter, and the label added to the usage characteristic data obtained in the film-applied and charging state is the first correction parameter; when the usage state of the electronic device is the film-applied but not charged state, the handwriting correction parameter of the electronic device is the second correction parameter, and the label added to the usage characteristic data obtained in the film-applied but not charged state is the second correction parameter.

[0211] In the embodiment of the present application, the handwriting correction parameter label on the training feature data can be used as a true result to measure the accuracy of the output of the first model. In the subsequent training process of the first model, the training feature data is input into the first model to obtain the output result of the first model. According to the output result and the handwriting correction parameter label on the input training feature data, the error between the output result and the true result can be determined.

[0212] In one example, labeling the first usage feature data includes: labeling the first usage feature data based on the first handwriting correction parameter to obtain the first training feature data.

[0213] Step S84: The preprocessing module sends the training feature data to the training module.

[0214] In this embodiment, the training feature data is obtained through the above steps S81 to S84. It can be understood that, for the contact sample data when the electronic device is in different use states, the operations similar to the above steps S81 to S84 can be performed to obtain the training feature data in the corresponding use state, providing a data basis for the training process of the first model.

[0215] Step S85: The training module trains the initial first model according to the training feature data to obtain the trained first model.

[0216] Fig.12 A model training schematic diagram provided in an embodiment of the present application, such as Fig.12 As shown, the training module uses the training feature data as input data and inputs it into the initial first model 1200 to obtain the corresponding output result. The training feature data is multiple groups, and the multiple groups of training feature data are different. For example, one group of training feature data can be the electronic device in Fig. 9 (1) The feature data of the straight line segment 900 in the scene, and another set of training feature data can be the electronic device in Fig. 9 (1) The feature data of the straight line segment 901 written in the scene, and another set of training feature data can be the electronic device in Fig. 9(6) Feature data of the straight line segment 900 written in the scene, etc. The output result is used to indicate the predicted handwriting correction parameters of the electronic device. The weight of the first model 900 is updated according to the output result and the loss function, and finally a weight parameter that meets the requirements is obtained. The requirement that the weight parameter needs to meet is that when the first model predicts the handwriting correction parameter of the stylus pen based on the weight parameter, the error between the predicted handwriting correction parameter (output result) and the handwriting correction parameter (actual result) indicated by the input data label is less than the preset error. The weight parameter that meets the requirements is the weight of the first model after training. Among them, the loss function is, for example, focal loss (a loss function improved on the basis of cross entropy for dealing with sample classification imbalance).

[0217] In an embodiment of the present application, the first model can be a regression network model based on deep learning, such as any one of linear regression (Linear Regression), polynomial regression (Ploynomial Regression), ridge regression (RidgeRegression), Lasso regression and elastic regression network (ElasticNet Regression), which is not specifically limited in this embodiment.

[0218] In the embodiment of the present application, a trained first model is obtained through the above steps S81 to S85. The trained first model can determine the correction parameters of the electronic device according to the acquired touch data when the stylus touches the screen of the electronic device.

[0219] Fig.13 A flowchart of a first model deployment method provided in an embodiment of the present application. The first model deployment method is applied to a server, or to an electronic device capable of terminal deployment. Fig.13 As shown, the first model deployment method includes: step S131-step S132.

[0220] Step S131: convert the trained first model into a preset format.

[0221] Among them, the preset format conversion can be the conversion from pyTorch (an open source neural network framework) to NCNN (a deployment framework for deploying models to mobile devices).

[0222] In an embodiment of the present application, during the training of the first model, the framework used by the first model may be pyTorch. PyTorch is a large machine learning framework that requires a large amount of storage space, so a lightweight framework, such as NCNN, is required when deploying the model. In this embodiment, after the first model training is completed, the first model is converted from pyTorch to NCNN to facilitate the subsequent deployment of the first model.

[0223] Step S132: deploy the first model and other collaborative modules converted into a preset format in the hardware abstraction layer of the electronic device.

[0224] Among them, Figure 3 As shown, the other collaborative modules are, for example, a data processing module and a handwriting correction module. The data processing module has the same function as the preprocessing module of the server in the aforementioned embodiment, and is used to process the touch data acquired by the electronic device when the stylus touches into input data that can be input into the first model. The handwriting correction module is used to configure the correction parameters output by the first model. Similarly, the handwriting correction module is described in detail in the embodiments described later, and will not be repeated here.

[0225] In an embodiment of the present application, the trained first model is deployed on an electronic device, so that the electronic device can automatically determine the correction parameters of the electronic device in the current usage state through the first model.

[0226] Fig.14 A flowchart of a correction parameter determination method provided in an embodiment of the present application. The correction parameter determination method is applied to an electronic device, which may be an electronic device that can be written by a stylus, such as a tablet, a mobile phone, or a laptop. Fig.14 As shown, the correction parameter determination method includes: step S1401-step S1407.

[0227] S1401: When the stylus pen contacts the screen of the electronic device, the data processing module obtains target contact data.

[0228] The target contact data is used to indicate the device status data acquired by the electronic device and the data generated by the contact with the stylus during the process of the stylus contacting the electronic device screen. The contact sample data includes but is not limited to data such as capacitance information and charging status information. In some examples, the target contact data also includes data such as posture information and / or pressure information.

[0229] In one implementation, when the stylus pen contacts the screen of the electronic device, the data processing module obtains target contact data, including the following steps 1 and 2.

[0230] Step 1: When the stylus pen contacts the screen of the electronic device, the data processing module obtains target capacitance information.

[0231] The target capacitance information indicates a collection of multiple frames of capacitance information acquired by the electronic device during the process of the stylus pen performing a contact operation on the screen of the electronic device.

[0232] like Figure 6 As shown, the electronic device is equipped with multiple capacitance sensors distributed in an array, and the multiple capacitance sensors are normally open. During the process of the stylus pen performing a touch operation on the screen of the electronic device, the electronic device obtains the capacitance of the multiple capacitance sensors according to a preset capacitance sampling rate. The capacitance sampling rate can be set according to the actual application, for example, to 240Hz, 1.5KHz, or other values, which is not limited in the embodiments of the present application. The capacitance sampling process can be executed by a touch module (or touch chip) in the electronic device, and the data processing module obtains the capacitance information from the touch module, or the touch module sends the capacitance information to the data processing module.

[0233] Each time the electronic device obtains the capacitance values ​​of the multiple capacitance sensors, a frame of capacitance information can be formed. Figure 6 The target capacitance information includes capacitance information at multiple moments in a contact operation, and the capacitance information at each moment includes the capacitance of each capacitance sensor at that moment.

[0234] Step 2: The data processing module obtains charging status information.

[0235] The charging status information is used to indicate whether the electronic device is being charged when the stylus pen is in contact with the screen of the electronic device. For example, when the stylus pen is in contact with the screen of the electronic device, the electronic device is being charged, and the charging status information indicates that it is in a charged state; when the stylus pen is in contact with the screen of the electronic device, the electronic device is not being charged, and the charging status information indicates that it is not being charged. The data processing module obtains the charging status information from the charging status management module.

[0236] In another embodiment, when the stylus pen contacts the screen of the electronic device, the step of the data processing module acquiring the target contact data further includes the following step three and / or step four.

[0237] Step 3: The data processing module receives the target pressure information sent by the stylus.

[0238] The target pressure information is used to indicate the pressure value when the stylus pen contacts the screen of the electronic device. When the stylus pen contacts the screen of the electronic device, the target pressure information is collected and sent to the electronic device.

[0239] Step 4: The data processing module obtains the target posture information.

[0240] The target posture information is used to indicate the posture of the electronic device when the stylus pen contacts the screen of the electronic device. Fig. 9 The angle between the horizontal plane and the horizontal plane is 45° as shown in (1), or Fig. 9 The posture shown in (2) is 0° with the horizontal plane. The posture may also be a posture with other angles with the horizontal plane, which will not be described here. The data processing module can obtain the target posture information from the gyroscope sensor.

[0241] In this embodiment, when the stylus pen contacts the screen of the electronic device, the data processing module obtains corresponding data from the touch module, gyroscope sensor, charging management module and stylus respectively, forming the target contact data corresponding to the contact between the stylus pen and the screen, providing a data basis for subsequent analysis.

[0242] S1402: The data processing module performs second data processing on the target contact data to obtain target usage feature data.

[0243] Among them, the second data processing includes but is not limited to data extraction processing, data fusion processing, feature enhancement processing, etc.

[0244] In one embodiment, the data processing module performs second data processing on the target contact data to obtain feature data, including: the following steps 1 to 3.

[0245] Step 1: Extract target capacitance characteristic data based on target capacitance information.

[0246] Among them, according to the target capacitance information, the target capacitance characteristic data is extracted, including: from the target capacitance information, according to the preset selection rules, the first target capacitance information is selected; from each frame of the target capacitance information in the first target capacitance information, a capacitance matrix of a preset size is extracted, and the capacitance matrix is ​​combined into the target capacitance characteristic data.

[0247] The preset selection rule includes: determining second capacitance information in the target capacitance information, the second capacitance information including the maximum capacitance in the target capacitance information; the second capacitance information is the Kth frame target capacitance information in the target capacitance information; selecting [KN, K+N] frame target capacitance information, and determining the first target capacitance information based on the [KN, K+N] frame target capacitance information, where N is a preset value.

[0248] In one implementation, the [KN, K+N] frame target capacity information is used as the first target capacity information.

[0249] In another embodiment, from the target capacity information of the [KN, K+N] frames, the target capacity information of the [KM, K+M]th frame is selected as the first target capacity information, and M is less than N.

[0250] It should be noted that the method in which the data processing module extracts the target capacitance characteristic data according to the target capacitance information is the same as the method in which the preprocessing module extracts the first capacitance characteristic data according to the first capacitance information in the aforementioned step S821. Therefore, the specific method for extracting the target capacitance characteristic data according to the target capacitance information can be referred to the detailed description in step S821 in the aforementioned embodiment, which will not be repeated here.

[0251] Step 2: Perform data fusion processing on the target capacitance characteristic data and the charging status information to obtain target fusion characteristic data.

[0252] The data fusion processing methods include but are not limited to projection, splicing, addition and the like. In actual application, the specific data fusion processing method can be selected according to the specific situation and is not limited in the embodiments of the present application.

[0253] In a possible implementation, when the target contact data also includes target auxiliary data, in the aforementioned step 2, the target capacitance characteristic data, the charging state information and the target auxiliary data are subjected to data fusion processing to obtain target fusion characteristic data. The target auxiliary data includes one or more of target pressure information, target posture information and sensor index information.

[0254] In one example, when the target contact data also includes target pressure information, the data processing module can perform data fusion processing on the target capacitance characteristic data, the target pressure information and the charging status information to obtain the target fusion characteristic data. It should be noted that the target pressure information can represent the pressure value when the stylus touches the screen of the electronic device. When the pressure value is larger, the distance between the tip of the stylus and the capacitance sensor in the electronic device is closer, and the capacitance of the capacitance sensor will change accordingly. Therefore, using the target pressure information as one of the target contact data can improve the accuracy of the stylus handwriting correction parameters determined by the subsequent first model.

[0255] In another example, when the target contact data also includes target posture information, the data processing module can perform data fusion processing on the target capacitance feature data, the target posture information and the charging state information to obtain target fusion feature data. It should be noted that different posture information of the electronic device will affect the capacitance of the capacitance sensor when the stylus touches the electronic device. Therefore, using the target posture information as one of the target contact data can improve the accuracy of the stylus handwriting correction parameters determined by the subsequent first model.

[0256] In yet another example, when the target contact data also includes target posture information and target pressure information, the target capacitance characteristic data, target pressure information, target posture information and charging status information are subjected to data fusion processing to obtain target fusion characteristic data.

[0257] In another example, the sensor index information can also be fused with the target capacitance feature data, target pressure information, target posture information and charging status information to obtain touch feature data, so as to improve the validity of the target fusion feature data. The sensor index includes the coordinate information of each capacitance sensor in the electronic device.

[0258] Step 3: Perform feature enhancement processing on the target fusion feature data to obtain target usage feature data.

[0259] The feature enhancement process is used to achieve the dimension expansion of the feature data. The method of performing feature enhancement process on the target fused feature data is the same as the method of performing feature enhancement process on the first fused feature data in the aforementioned step S823. Please refer to the detailed description of the feature enhancement process on the first fused feature data in the aforementioned step S823, which will not be repeated here.

[0260] In this embodiment, the data processing module performs second data processing on the target processing data to obtain input data (target usage feature data) that can be input into the first model.

[0261] S1403: The data processing module inputs the target usage feature data into the first model.

[0262] The first model is pre-configured in the electronic device and is used to determine the correction parameters of the electronic device.

[0263] S1404: The first model outputs correction parameters to the handwriting correction module based on the input target usage feature data.

[0264] The correction parameter output by the first model is a parameter used to correct the reporting point position determined by the electronic device according to the contact operation when the stylus performs a contact operation on the electronic device. The handwriting correction parameter can indicate the compensation value of each reporting point position in the electronic device when the stylus contacts the screen of the electronic device. In some embodiments, the handwriting correction parameter output by the first model can be in the form of a function.

[0265] S1405: The handwriting correction module corrects the reporting point position based on the correction parameters output by the first model.

[0266] In one example, the step of correcting the reporting point position based on the correction parameters output by the first model includes: when the stylus is in contact with the screen of the electronic device, for each reporting point position, selecting a corresponding compensation value from the correction parameters output by the first model; adding the compensation value to the coordinates of the reporting point position to determine the theoretical position corresponding to the reporting point position. The theoretical positions corresponding to all the reporting point positions can constitute the corrected stylus handwriting.

[0267] In an embodiment of the present application, through the above-mentioned steps S1401 to S1405, when the stylus touches the screen of the electronic device, the correction parameters of the electronic device in the current use state can be accurately determined, and the reporting point position determined by the electronic device according to the contact operation of the stylus can be accurately corrected, thereby improving the user experience.

[0268] It should be noted that when the auxiliary parameters used in training the first model include the thickness of the first film layer, the trained first model has the function of implicitly fitting the mapping relationship between the stylus correction parameters and the film thickness. That is, when the stylus touches the screen of the electronic device when the screen has a first film layer of different thicknesses, the feature data input into the first model does not need to include the film thickness of the first film layer, and the correction parameters output by the first model will be different accordingly, which can effectively improve the accuracy of the correction parameters and enhance the user experience.

[0269] It should also be noted that when the auxiliary parameters used in training the first model include the dielectric constant of the first film layer, the trained first model also has the function of implicitly fitting the mapping relationship between the dielectric constant of the film layer and the correction parameters. When the screen of the electronic device has a first film layer with different dielectric constants, the stylus touches the screen of the electronic device, and the feature data input into the first model does not need to include the dielectric constant of the first film layer. The stylus correction parameters output by the first model will also be different accordingly, which can effectively improve the accuracy of the correction parameters and enhance the user experience.

[0270] Fig.15 A schematic diagram of an application effect provided by an embodiment of the present application. The electronic device 100 is configured with a first model, which can automatically determine the correction parameters of the electronic device in the current use state. For example, a user uses a stylus pen 101 to perform a touch operation on the electronic device 100 that has just been filmed and has not adjusted the stylus pen handwriting correction parameters. The touch track of the touch operation is a straight line segment. Fig.15As shown, the actual handwriting corresponding to the contact track (the actual handwriting is the track after connecting the reporting point position) displayed by the electronic device 100 may include part 1501 and part 1502. Among them, the handwriting of part 1501 is distorted and crooked. This is because when the user puts down the pen, the correction parameters have not been adjusted. The electronic device uses the unadjusted correction parameters to correct the reporting point position in the film-attached state, resulting in inaccurate calibration of the reporting point position. Therefore, the handwriting of part 1501 determined and displayed based on the reporting point position is distorted and crooked.

[0271] In the process of part 1501, electronic equipment can be Fig.14 The provided correction parameter determination method determines new correction parameters, and corrects the next reporting point position according to the determined new correction parameters, so that the next reporting point position can be accurately calibrated, so that the handwriting of part 1502 determined and displayed based on the reporting point position does not appear distorted or crooked.

[0272] It should be noted that Fig.15 This is an exemplary description of the application effect. In order to better illustrate the effect of the embodiment of the present application, part 1501 of the handwriting is enlarged for illustration. In actual application, the length of the distorted and crooked part of the handwriting is generally less than 2 mm. The correction parameter configuration method provided in the embodiment of the present application can complete the adaptive adjustment of the correction parameters of the electronic device without the user's obvious perception, realize the optimization of the handwriting of the stylus pen, and improve the user experience.

[0273] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0274] The steps performed by the electronic device in the method for determining the handwriting correction parameters of the stylus provided in the embodiment of the present application may also be performed by a chip system included in the electronic device, wherein the chip system may include a processor and a Bluetooth chip. The chip system may be coupled to a memory so that when the chip system is running, the computer program stored in the memory is called to implement the steps performed by the electronic device. The processor in the chip system may be an application processor or a processor other than an application processor.

[0275] This embodiment further provides a computer-readable medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0276] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0277] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0278] Among them, the electronic device, computer-readable medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0279] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0280] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0281] The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present application may be integrated in a processing unit, or each unit may exist physically separately, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0282] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above contents is within the scope of the present application. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read only memory, ROM), random access memory (random access memory, RAM), disk or optical disk and other media that can store program code.

[0283] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining a correction parameter, characterized in that: Applied to electronic equipment, the method comprises: When the stylus pen contacts the screen of the electronic device, target contact data is obtained; the target contact data at least includes target capacitance information and charging state information; the target capacitance information includes multiple frames of target capacitance information; each frame of the target capacitance information is used to indicate the capacitance of each capacitance sensor in the capacitance sensor array of the electronic device; A correction parameter corresponding to the electronic device is determined according to the target capacitance information and the charging state information.

2. The method according to claim 1, characterized in that When the stylus pen contacts the screen of the electronic device, acquiring target contact data includes: Acquiring the target capacity value information of the multiple frames based on a preset capacity value sampling rate; The charging status information is acquired, where the charging status information is used to indicate whether the electronic device is being charged.

3. The method according to claim 1 or 2, characterized in that: The step of determining a correction parameter corresponding to the electronic device according to the target capacitance information and the charging state information includes: Extracting target capacitance characteristic data according to the target capacitance information; Performing data fusion processing on the target capacitance characteristic data and the charging state information to obtain target fusion characteristic data; Performing feature enhancement processing on the target fusion feature data to obtain target usage feature data; The target usage characteristic data is input into a first model to obtain the correction parameters output by the first model.

4. The method according to claim 3, characterized in that The target contact data also includes target auxiliary data; the target capacitance characteristic data and the charging state information are subjected to data fusion processing to obtain target fusion characteristic data, including: The target capacitance characteristic data, the charging state information and the target auxiliary data are subjected to data fusion processing to obtain target fusion characteristic data; Wherein, the target auxiliary data includes one or more of target pressure information, target posture information and sensor index information; The target pressure information is used to indicate the pressure value when the stylus pen contacts the screen of the electronic device; the target posture information is used to indicate the posture of the electronic device; and the sensor index information is used to indicate the coordinate information of the capacitance sensor.

5. The method according to claim 3, characterized in that: The step of extracting target capacitance characteristic data according to the target capacitance information includes: Selecting first target capacitance value information from the target capacitance information according to a preset selection rule; A capacity matrix of a preset size is extracted from each frame of target capacity information in the first target capacity information, and the capacity matrix is ​​combined into the target capacity feature data.

6. The method according to claim 5, characterized in that The preset selection rules include: In the target capacitance information, second capacitance information is determined, where the second capacitance information includes a maximum capacitance in the target capacitance information; the second capacitance information is the Kth frame target capacitance information in the target capacitance information; Select [KN, K+N] frame target capacity information, and determine the first target capacity information based on the [KN, K+N] frame target capacity information, where N is a preset value.

7. The method according to claim 6, characterized in that The preset selection rules also include: The target capacity information of the [KM, K+M]th frame is selected from the target capacity information of the [KN, K+N]th frame as the first target capacity information, where M is less than N.

8. The method according to claim 5, characterized in that The matrix center of the capacity matrix indicates the maximum capacity in the corresponding target capacity information.

9. The method according to claim 3, characterized in that: After determining the correction parameter corresponding to the electronic device according to the target capacitance information and the charging state information, the method further includes: The reporting point position is corrected according to the determined correction parameters.

10. The method according to claim 1, characterized in that The electronic device is connected to the stylus pen via Bluetooth.

11. A model training method, characterized in that: Applied to a server, the method comprises: Acquire contact sample data and correction parameters of the electronic device in multiple usage states; the contact sample data at least includes capacitance information, charging state information and reporting point information; the capacitance information includes multiple frames of capacitance information; each frame of the capacitance information is used to indicate the capacitance of each capacitance sensor in the capacitance sensor array of the electronic device; Based on the contact sample data and the correction parameters, obtaining training feature data; According to the training feature data, an initial first model is trained to obtain a trained first model; the trained first model is used to determine correction parameters.

12. The method according to claim 11, characterized in that The usage status at least includes: The state of being charged with film attached, the state of being charged without film attached, the state of being not charged with film attached, and the state of being not charged with film attached.

13. The method according to claim 12, characterized in that When the electronic device is in any of the use states, the step of acquiring contact sample data and calibration parameters of the electronic device in multiple use states includes: Receiving first capacitance information sent by an electronic device; the first capacitance information includes multiple frames of first capacitance value information; Receiving first charging status information sent by the electronic device; Receiving first reporting point information sent by an electronic device; Based on the first reporting point information, a first correction parameter is determined.

14. The method according to claim 13, characterized in that The determining a first correction parameter based on the first reporting point information includes: According to the first reporting point information, a coordinate relationship between a theoretical handwriting and an actual handwriting is established; the first reporting point information includes a set of reporting point positions; According to the coordinate relationship between the theoretical handwriting and the actual handwriting, a compensation value of each of the reporting point positions is determined, and the compensation value of each of the reporting point positions is determined as the first correction parameter.

15. The method according to claim 13, characterized in that The step of obtaining training feature data based on the contact sample data and the correction parameters includes: Extracting first capacitance characteristic data according to the first capacitance information; Performing data fusion processing on the first capacitance characteristic data and the first charging state information to obtain first fused characteristic data; Performing feature enhancement processing on the first fused feature data to obtain first usage feature data; Based on the first correction parameter, the first usage feature data is labeled to obtain first training feature data.

16. The method according to claim 11, characterized in that The step of training an initial first model according to the training feature data to obtain a trained first model includes: Inputting the training feature data into an initial first model to obtain an output result; Based on the output result and the loss function, the weight of the initial first model is updated to obtain the trained first model.

17. The method according to claim 16, characterized in that The first model is indicated as a regression network model based on deep learning; and the loss function is indicated as focal loss.

18. The method according to claim 16, characterized in that After training the initial first model according to the training feature data to obtain the trained first model, the method further includes: Converting the trained first model into a preset format; The first model converted into a preset format is deployed on the electronic device.

19. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; And a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the one or more processors, the electronic device executes the correction parameter determination method according to any one of claims 1 to 10.

20. A server, characterized in that: The server comprises: one or more processors; Memory; And a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the one or more processors, the electronic device executes the model training method as described in any one of claims 11-17.

21. A computer storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the correction parameter determination method according to any one of claims 1 to 10.

22. A computer storage medium, characterized in that It includes computer instructions, which, when executed on a server, enable the server to execute the model training method as described in any one of claims 11 to 16.

Images