Method of correcting coordinates and electronic device

Abstract

The present application provides a method and electronic device for calibrating coordinates. The method includes: when a stylus pen contacts the screen of the electronic device, acquiring target contact data, the target contact data including at least target capacitance information and charging status information; wherein the target capacitance information includes multiple frames of target capacitance information, each frame of target capacitance information indicating the capacitance of each capacitance sensor in a capacitance sensor array of the electronic device; and determining corresponding calibration parameters for the electronic device based on the target capacitance information and charging status information. Specifically, the electronic device can automatically adjust the calibration parameters after the stylus pen contacts the screen of the electronic device, effectively improving the accuracy of the calibration parameters in the electronic device under different usage conditions, facilitating subsequent accurate calibration of the reported position of the electronic device based on the calibration parameters, thereby improving the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, in particular to a method for correcting coordinates and an electronic device. BACKGROUND

[0002] In the process of using the terminal, in order to protect the terminal screen, the user often pastes a protective film on the terminal screen. However, the protective film will affect the user's experience of using the stylus. Usually, the stylus handwriting correction parameter on the terminal is set according to the state of the terminal without the film. This stylus handwriting correction parameter is not applicable to the stylus handwriting correction in the state of the terminal with the film. Moreover, the terminal cannot automatically sense the state of the screen with the film and cannot timely adjust the corresponding stylus handwriting correction parameter. Therefore, when the screen of the terminal is pasted with the protective film, the user often experiences the distortion and skew of the handwriting when using the stylus to write on the terminal screen. SUMMARY

[0003] In order to solve the above technical problems, the present application provides a method for correcting coordinates, so that the electronic device can adaptively adjust the correction parameter corresponding to the use state of the electronic device after the stylus contacts the screen, so as to enable the electronic device to accurately correct the report point position according to the correction parameter and improve the user experience.

[0004] In the first aspect, the present application provides a method for correcting coordinates. In the method, when the stylus contacts the screen of the electronic device, target contact data is obtained. The target contact data at least includes target capacitance information and charging state information. The target capacitance information includes multiple frames of target capacitance value information. Each frame of target capacitance value information is used to indicate the capacitance value of each capacitance sensor in the capacitance sensor array of the electronic device. According to the target capacitance information and the charging state information, the corresponding correction parameter of the electronic device is determined. That is, the electronic device can automatically adjust the correction parameter after the stylus contacts the screen of the electronic device. This effectively improves the accuracy of the correction parameter in the electronic device in different use states, so as to facilitate the subsequent electronic device to accurately correct the report point position according to the correction parameter and improve the user experience.

[0005] According to the first aspect, when the stylus contacts the screen of the electronic device, the target contact data is obtained, including: obtaining multiple frames of target capacitance value information based on a preset capacitance sampling rate; and obtaining charging state information, which is used to indicate whether the electronic device is being charged. In the embodiment of the present application, the target capacitance information in the target contact data includes multiple frames of target capacitance value information. This multiple frames of target capacitance value information can effectively represent the contact characteristics when the stylus contacts the screen of the electronic device. The charging state information will affect the capacitance value information. Obtaining multiple frames of target capacitance value information and charging state information can provide effective data support for subsequently accurately determining the correction parameter corresponding to the report point position.

[0006] According to a first aspect, or any possible implementation mode of the above first aspect, the determining of the correction parameter corresponding to the electronic device according to the target capacitance information and the charging state information comprises: extracting target capacitance value feature data according to the target capacitance information; performing data fusion processing on the target capacitance value feature data and the charging state information to obtain target fusion feature data; performing feature enhancement processing on the target fusion feature data to obtain target use feature data; and inputting the target use feature data into the first model to obtain the correction parameter output by the first model.

[0007] In the embodiments of the present application, the electronic device only needs to input the target use feature data into the first model to obtain the correction parameter output by the first model, simplifying the mode of the electronic device to determine the correction parameter corresponding to the reporting position, and improving the speed of the electronic device to adjust the correction parameter.

[0008] According to the first aspect, or any possible implementation mode of the above first aspect, the target contact data further comprises target auxiliary data; the data fusion processing on the target capacitance value feature data and the charging state information to obtain the target fusion feature data comprises: performing data fusion processing on the target capacitance value feature data, the charging state information and the target auxiliary data to obtain the target fusion feature data; and the target auxiliary data comprises one or more of target pressure information, target posture information and sensor index information; the target pressure information is used to indicate a pressure value when the stylus contacts the screen of the electronic device; the target posture information is used to indicate a posture of the electronic device; and the sensor index information is used to indicate coordinate information of the capacitance value sensor.

[0009] In the embodiments of the present application, the target auxiliary data is used to indicate data that will affect the capacitance value of the capacitance value sensor, so that the target auxiliary data is involved in the process of determining the correction parameter, which can improve the accuracy of the determined correction parameter, facilitate the subsequent electronic device to accurately correct the reporting position according to the correction parameter, and improve the user experience.

[0010] According to the first aspect, or any possible implementation mode of the above first aspect, the extracting of the target capacitance value feature data according to the target capacitance information comprises: selecting first target capacitance value information according to a preset selection rule from the target capacitance information; and extracting a capacitance value matrix of a preset size from each frame of target capacitance value information in the first target capacitance value information, and combining the capacitance value matrix into the target capacitance value feature data.

[0011] In the embodiments of the present application, the target capacitance information contains a large number of target capacitance value information, and the large data volume may cause the time for determining the correction parameter to be too long. Therefore, it is necessary to preliminarily select appropriate target capacitance value information to participate in the process of determining the correction parameter. In addition, when the stylus contacts the screen of the electronic device, the capacitance value of the capacitance sensor at the contact position and the surrounding positions changes obviously, which can be regarded as effective data and can effectively represent the contact characteristics of the stylus and the screen of the electronic device. A preset size of the capacitance value matrix can be extracted from the target capacitance value information, the redundancy of the target capacitance information is removed, and the data effectiveness is improved.

[0012] According to the first aspect, or any one of the implementations of the first aspect, the preset selection rule comprises: determining second capacitance value information in the target capacitance information, the second capacitance value information comprising a maximum capacitance value in the target capacitance information; the second capacitance value information being K-th frame target capacitance value information in the target capacitance information; selecting [K-N, K+N] frame target capacitance value information, determining the first target capacitance value information based on the [K-N, K+N] frame target capacitance value information, and N being a preset value. In the embodiments of the present application, the target capacitance information contains a large number of target capacitance value information, and the large data volume may cause the time for determining the correction parameter to be too long. Therefore, it is necessary to preliminarily select appropriate target capacitance value information to participate in the process of determining the correction parameter.

[0013] According to the first aspect, or any one of the implementations of the first aspect, the preset selection rule further comprises: selecting [K-M, K+M] frame target capacitance value information in the [K-N, K+N] frame target capacitance value information as the first target capacitance value information, and M being less than N. In the embodiments of the present application, when the stylus contacts the screen of the electronic device, the capacitance value of the capacitance sensor is unstable during the process of the stylus approaching the screen of the electronic device and leaving the screen, and the capacitance value information obtained by the electronic device has poor effectiveness. Therefore, it is necessary to remove the several frames of capacitance value information arranged in the front and the rear from the preliminarily selected first sample capacitance value information, and screen out the capacitance value information with high data stability.

[0014] According to the first aspect, or any one of the implementations of the first aspect, the matrix center of the capacitance value matrix indicates the maximum capacitance value in the corresponding target capacitance value information. In the embodiments of the present application, when the stylus contacts the screen of the electronic device, the capacitance value of the capacitance sensor at the contact position and the surrounding positions changes obviously, which can be regarded as effective data and can effectively represent the contact characteristics of the stylus and the screen of the electronic device. Therefore, the maximum capacitance value in the target capacitance value information is determined as the matrix center of the capacitance value matrix corresponding to the target capacitance value information, and the data effectiveness of the capacitance value matrix is improved.

[0015] According to a first aspect, or any possible implementation mode of the first aspect, after the correction parameter corresponding to the electronic device is determined according to the target capacitance information and the charging state information, the method further includes correcting the report point position according to the determined correction parameter. In the embodiments of the present application, after the report point position is corrected, the error between the handwriting of the stylus displayed based on the corrected report point position and the contact track of the user on the screen of the electronic device is small, the handwriting of the stylus can be optimized, and the user experience is improved.

[0016] According to the first aspect, or any possible implementation mode of the first aspect, the electronic device and the stylus are connected through Bluetooth.

[0017] In a second aspect, the present application provides a model training method, which is applied to a server. The method includes: obtaining contact sample data and correction parameters of an electronic device in multiple use states; the contact sample data at least includes capacitance information, charging state information and report point information; the capacitance information includes multiple frames of capacitance value information; each frame of capacitance value information is used to indicate the capacitance value of each capacitance value sensor in the capacitance value sensor array of the electronic device; obtaining training feature data based on the contact sample data and the correction parameters; training an initial first model according to the training feature data to obtain a trained first model; and the trained first model is used to determine the correction parameters.

[0018] In the embodiments of the present application, the first model is trained by the contact sample data and the correction parameters of the electronic device in multiple use states, so that the trained first model can have the function of determining the correction parameters in the electronic device.

[0019] According to the second aspect, the use states at least include: a film attached and charged state, a film not attached and charged state, a film attached and not charged state and a film not attached and not charged state. In the embodiments of the present application, the more use states, the more rich the scenarios corresponding to the contact sample data, which is convenient for the first model trained according to the contact sample data to adapt to more rich scenarios.

[0020] According to the second aspect, or any possible implementation mode of the second aspect, in the case that the electronic device is in any use state, obtaining the contact sample data and the correction parameters of the electronic device in multiple use states includes: receiving first capacitance information sent by the electronic device; the first capacitance information includes multiple frames of first capacitance value information; receiving first charging state information sent by the electronic device; receiving first report point information sent by the electronic device; and determining a first correction parameter based on the first report point information.

[0021] In the embodiments of the present application, the capacitance value of the capacitance sensor can effectively represent the contact feature of the stylus and the electronic device, and the charging state has a greater impact on the capacitance value of the capacitance sensor. Therefore, taking the capacitance value of the capacitance sensor and the charging state information as the contact sample data can improve the effectiveness of the contact sample data.

[0022] According to the second aspect, or any one of the implementation forms of the second aspect, the first correction parameter is determined based on the first report point information, including: establishing a coordinate relationship between a theoretical handwriting and an actual handwriting according to the first report point information; the first report point information includes a set of report point positions; and a compensation value of each report point position is determined according to the coordinate relationship between the theoretical handwriting and the actual handwriting, and the compensation value of each report point position is determined as the first correction parameter. In the embodiments of the present application, the correction parameter in the corresponding use state can be determined according to the first report point information, which facilitates subsequent data labeling according to the correction parameter, so as to measure the accuracy of the first model output as a true result.

[0023] According to the second aspect, or any one of the implementation forms of the second aspect, the training feature data is obtained based on the contact sample data and the correction parameter, including: first capacitance value feature data is extracted according to the first capacitance value information; the first capacitance value feature data and the first charging state information are subjected to data fusion processing to obtain first fusion feature data; the first fusion feature data is subjected to feature enhancement processing to obtain first use feature data; and the first use feature data is labeled based on the first correction parameter to obtain the first training feature data.

[0024] In the embodiments of the present application, the capacitance value of the capacitance sensor can effectively represent the contact feature of the stylus and the electronic device, and the charging state has a greater impact on the capacitance value of the capacitance sensor. Therefore, taking the capacitance value of the capacitance sensor and the charging state information as the contact sample data can improve the effectiveness of the contact sample data.

[0025] According to the second aspect, or any one of the implementation forms of the second aspect, the initial first model is trained based on the training feature data to obtain a trained first model, including: inputting the training feature data into the initial first model to obtain an output result; and updating the weight of the initial first model based on the output result and a loss function, so that the error between the correction parameter (output result) predicted by the first model and the correction parameter (actual result) indicated by the input data label is less than a preset error, thereby obtaining the trained first model.

[0026] According to the second aspect, or any one of the implementation forms of the second aspect, the first model indicates a regression network model based on deep learning; and the loss function indicates a focal loss.

[0027] According to a second aspect, or any possible implementation mode of the second aspect, after the initial first model is trained according to the training feature data, and the trained first model is obtained, the method further comprises: performing preset format conversion on the trained first model, so that the trained first model is converted into a lightweight model, which can be adapted to a mobile terminal; and deploying the first model after the preset format conversion on the electronic device, so that the electronic device has the function of automatically sensing and adjusting the correction parameter. For example, during the process in which the user uses a stylus to perform a touch operation on the screen of the electronic device, the electronic device can acquire touch data according to the touch operation, and input the touch data after data processing into the first model deployed in advance, to obtain the correction parameter output by the first model.

[0028] According to a third aspect, the present application provides an electronic device, comprising: one or more processors; a memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the one or more processors, the electronic device performs the method for correcting coordinates according to the first aspect, or any possible implementation mode of the first aspect.

[0029] The third aspect and any possible implementation mode of the third aspect correspond to the first aspect and any possible implementation mode of the first aspect respectively. The technical effects corresponding to the third aspect and any possible implementation mode of the third aspect can be referred to the technical effects corresponding to the first aspect and any possible implementation mode of the first aspect, which will not be described here.

[0030] According to a fourth aspect, the present application provides a server, comprising: one or more processors; a memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the one or more processors, the electronic device performs the model training method according to the second aspect.

[0031] The technical effects corresponding to the fourth aspect can be referred to the technical effects corresponding to the second aspect, which will not be described here.

[0032] According to a fifth aspect, the present application provides a computer storage medium, comprising computer instructions, when the computer instructions are run on an electronic device, the electronic device performs the method for correcting coordinates according to the first aspect, or any possible implementation mode of the first aspect.

[0033] The fifth aspect corresponds to the first aspect and any possible implementation mode of the first aspect. The technical effects corresponding to the fifth aspect and any possible implementation mode of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any possible implementation mode of the first aspect, which will not be described here.

[0034] In a sixth aspect, the present application provides a computer storage medium, comprising computer instructions, when the computer instructions are executed on an electronic device, the electronic device executes the model training method of the second aspect.

[0035] The technical effects of the sixth aspect can refer to the technical effects of the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A scene schematic diagram provided by an embodiment of the present application;

[0037] Figure 2 A structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0038] Figure 3 A software structural block diagram of an electronic device provided by an embodiment of the present application;

[0039] Figure 4 A schematic diagram of an electronic device and a stylus in communication connection provided by an embodiment of the present application;

[0040] Figure 5 A schematic diagram of data interaction between an electronic device and a stylus provided by an embodiment of the present application;

[0041] Figure 6 A capacitance schematic diagram of a capacitance sensor detected by an electronic device in different film layer states of a screen provided by an embodiment of the present application;

[0042] Figure 7 A schematic diagram of an electronic device obtaining capacitance information provided by an embodiment of the present application;

[0043] Figure 8 A flowchart of a first model training method provided by an embodiment of the present application;

[0044] Figure 9 A contact sample data acquisition scene example diagram provided by an embodiment of the present application;

[0045] Figure 10 A schematic diagram of determining handwriting correction parameters provided by an embodiment of the present application;

[0046] Figure 11 An example diagram of extracting capacitance feature data provided by an embodiment of the present application;

[0047] Figure 12 A model training schematic diagram provided by an embodiment of the present application;

[0048] Figure 13 A flowchart of a first model deployment method provided by an embodiment of the present application;

[0049] Figure 14 A flowchart of a method for correcting coordinates provided by an embodiment of the present application

[0050] Figure 15 An application effect schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0052] The term “and / or” in the present application is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone.

[0053] The terms “first” and “second” and the like in the specification and claims of the embodiments of the present application are used to distinguish different objects, and are not used to describe the specific order of the objects. For example, the first target object and the second target object are used to distinguish different target objects, and are not used to describe the specific order of the target objects.

[0054] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of “exemplary” or “for example” is intended to present concepts in a concrete manner.

[0055] In the description of the embodiments of the present application, unless otherwise specified, “a plurality of” means two or more. For example, a plurality of processing units means two or more processing units; a plurality of systems means two or more systems.

[0056] In the process of using an electronic device, a user often pastes a protective film on the screen of the electronic device in order to protect the screen. The protective film is, for example, a tempered film, a paper-like film, etc. However, the protective film can affect the process of using a handwriting pen by the user. The handwriting pen (which can also be referred to as a stylus or a stylus pen) is a device used for writing on an electronic device.

[0057] Figure 1 A scene schematic diagram provided by an embodiment of the present application. Before introducing the embodiments of the present application, firstFigure 1 An application scenario of an embodiment of the present application is described, which includes an electronic device 100 and a stylus 101 .

[0058] Normally, the handwriting correction parameters of the stylus on the electronic device are set according to the state of the electronic device without film. Figure 1 As shown, in Figure 1 In the scenario shown in (1), the screen of electronic device 100 is not covered with a protective film. The user uses stylus 101 to perform a touch operation on the screen of electronic device 100 without a protective film, and the touch operation trace is a straight line. Display interface 102 responds to the touch operation of stylus 101, and the displayed handwriting 103 is a straight line corresponding to the touch operation trace.

[0059] exist Figure 1 In the scenario shown in (2), after the screen of the electronic device 100 is affixed with a protective film 104, the user uses the stylus 101 to perform the same contact operation as shown in the aforementioned (1) figure on the screen affixed with the protective film 104. The handwriting 106 displayed on the display interface 105 in response to the contact operation of the stylus 101 is a distorted and crooked line. This is because after the screen of the electronic device 100 is affixed with the protective film 104, the handwriting correction parameters of the electronic device are not adjusted as the state of the electronic device being affixed with the film, and the handwriting correction parameters of the electronic device before the electronic device is affixed with the film are not applicable to handwriting correction in the state of the electronic device being affixed with the film, resulting in the handwriting 106 displayed on the display interface 105 in response to the contact operation of the stylus 101 not being a straight line corresponding to the contact trajectory, but a distorted and crooked line.

[0060] In order to solve the above problems, the embodiments of the present application provide a first model training method, a deployment method, and a method for correcting coordinates, so that the electronic device can accurately determine the correction parameters corresponding to the usage status of the electronic device after the stylus touches the screen, so that the electronic device can accurately correct the stylus handwriting according to the correction parameters, thereby improving the user experience.

[0061] The data transmission method provided in the embodiments of the present application can be applied to electronic devices, and the electronic devices can be portable computers (such as mobile phones), tablet computers, laptop computers, personal computers (PCs), augmented reality (AR) and virtual reality (VR) devices, in-vehicle computers, and the like. The following embodiments do not impose any special restrictions on the specific form of the electronic devices.

[0062] Before describing the technical solutions of the embodiments of the present application, the electronic device of the embodiments of the present application will be described with reference to the accompanying drawings. Figure 2A structural schematic diagram of an electronic device 100 is provided for an embodiment of the present application. It should be understood that Figure 2 The electronic device 100 shown is only one example of an electronic device, and the electronic device 100 can have more or fewer components than shown, can combine two or more components, or can have a different configuration of components. Figure 2 The various components shown in the electronic device 100 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0063] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a capacitance sensor 180M, etc.

[0064] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated into one or more processors.

[0065] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of instruction fetching and instruction execution.

[0066] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0067] The charging management module 140 is configured to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 while also providing power to the electronic device through the power management module 141.

[0068] The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160, etc. The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0069] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with tuning switches.

[0070] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer the same to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor, and radiate the same as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the same device as at least part of the modules of the processor 110.

[0071] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the microphone 170B, etc.), or displays an image or a video through the display screen 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110, and disposed in the same device as the mobile communication module 150 or other functional modules.

[0072] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and radiate it as an electromagnetic wave via the antenna 2.

[0073] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0074] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0075] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, N being a positive integer greater than 1.

[0076] The electronic device 100 can implement a photographing function through an ISP, a camera 193, a video codec, a GPU, a display 194, and an application processor, etc.

[0077] The camera 193 is used to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to an ISP to convert into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB, YUV, etc. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0078] The NPU is a neural-network (NN) computing processor that quickly processes input information by drawing on the structure of a biological neural network, such as the transmission mode between neurons in the human brain, and can also constantly self-learn. Through the NPU, the electronic device 100 can implement intelligent cognitive applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0079] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, music, video, and other files can be saved in the external memory card.

[0080] The internal memory 121 can be used to store computer executable program code including instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during use of the electronic device 100 (such as audio data, a phonebook, etc.), etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0081] The electronic device 100 can implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0082] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and is also configured to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode an audio signal. In some embodiments, the audio module 170 can be disposed in the processor 110, or part of the functions of the audio module 170 can be disposed in the processor 110.

[0083] The speaker 170A, also known as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0084] The receiver 170B, also known as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 100 answers a call or a voice message, the receiver 170B can be held close to the ear of a person to listen to the voice.

[0085] The microphone 170C, also known as a "microphone", "sound transducer", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, a user can speak into the microphone 170C close to the mouth to input a sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, it can also implement a noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, it can also identify the source of the sound, implement directional recording function, etc.

[0086] The earphone interface 170D is configured to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0087] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shaking of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and lets the lens offset the shaking of the electronic device 100 by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0088] The capacitance sensor 180M is used to change the capacitance value according to the contact operation of the stylus acting on or near it, such as touch operation, writing operation, drawing operation, etc. The capacitance sensor can transmit the detected contact operation of the stylus to the application processor and provide visual output related to the contact operation through the display screen 194.

[0089] The keys 190 include a power-on key, a volume key, etc. The keys 190 can be mechanical keys. They can also be touch keys. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0090] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The indicator 192 can be an indicator light that can be used to indicate charging status, power changes, and also to indicate messages, missed calls, notifications, etc.

[0091] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100.

[0092] Figure 3 A software structure block diagram of the electronic device 100 provided by the embodiments of the present application.

[0093] The layered architecture of the electronic device 100 divides the software into several layers, each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, the application layer, the framework layer, the Android runtime, the hardware abstraction layer (Hardware Abstraction Layer, HAL), the system library, and the kernel layer.

[0094] The application layer can include a series of application packages. For example, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0095] The application framework layer provides application programming interface (API) and programming framework for the applications of the application layer, including various components and services to support Android development of developers. The application framework layer includes some pre-defined functions.

[0096] As shown in Figure 3 The application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, pen management service, etc.

[0097] The window manager is used to manage window programs. The window manager can acquire the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen, etc.

[0098] The content provider is used to store and acquire data, and make the data accessible by the applications. The data can include video, image, audio, dialed and received calls, browsing history and bookmarks, phonebook, etc.

[0099] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0100] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of call status (including call connection, call hang-up, etc.).

[0101] The resource manager provides various resources for the applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0102] The notification manager makes the applications able to display notification information in the status bar, which can be used to convey the type of messages that can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform the completion of download, message reminder, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as the notification of the application running in the background, and can also be a notification in the form of a dialogue window appearing on the screen. For example, the text information is prompted in the status bar, a prompt sound is emitted, the electronic device is vibrated, the indicator light flashes, etc.

[0103] The stylus management service can also be referred to as a stylus management service, and is configured to manage a stylus connected to an electronic device, including controlling connection and disconnection of the stylus.

[0104] A hardware abstraction layer (HAL) is an interface layer between an operating system kernel and hardware circuitry. The hardware abstraction layer provides a virtual hardware platform for the operating system. The operating system core or hardware driver can call the hardware abstraction layer. As shown in Figure 3 In an embodiment of the present application, the hardware abstraction layer includes a data processing module, a first model, and a handwriting correction module.

[0105] The data processing module is configured to perform a corresponding data processing operation on the received data, such as data extraction processing, data fusion processing, etc.

[0106] The first model is configured to predict a correction parameter of the electronic device.

[0107] The handwriting correction module is configured to correct the point position according to the correction parameter output by the first model.

[0108] The kernel layer is between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0109] It can be understood that, Figure 3 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer layers than shown, and each layer can include more or fewer components, which are not limited in the present application.

[0110] Figure 4 A schematic diagram of an electronic device and a stylus in communication connection according to an embodiment of the present application is shown in FIG. 2. Figure 4 As shown, the stylus 200 includes, but is not limited to, a microcontroller unit (MCU) 201, a pressure sensor 202, a Bluetooth integrated circuit (IC) 203, a Bluetooth antenna 204, an electrode 205, etc. It can be understood that, Figure 4 The components included in the stylus 200 shown do not constitute a specific limitation on the stylus 200. In other embodiments of the present application, the stylus 200 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements.

[0111] Optionally, the pressure sensor 202 is configured to acquire a pressure parameter. For example, when the stylus tip contacts an object (e.g., a touch screen), the pressure sensor 202 can acquire a corresponding pressure value based on the pressure received and report the pressure value to the MCU 201. Optionally, the pressure value can also be referred to as a pressure parameter, pressure data, pressure information, etc., which are not limited in the present application.

[0112] Optionally, the MCU 201 is configured to process the pressure parameter received from the pressure sensor 202. It should be noted that the MCU 201 is only an illustrative example, and other devices that can function as a processor or microcontroller can also be used as an alternative to the MCU.

[0113] Optionally, the MCU 201 is further configured to control the Bluetooth antenna 204 to output a pressure signal, which is used to indicate the pressure parameter acquired by the MCU 201 from the pressure sensor 202. For example, the MCU 201 can generate a pressure signal based on the pressure parameter acquired from the pressure sensor 202, and output the pressure signal to the electronic device 210 through the Bluetooth IC 203 and the Bluetooth antenna 204. Correspondingly, in some embodiments, the antenna (e.g., antenna 2 in the electronic device 210) can acquire a corresponding pressure parameter based on the received pressure signal, and transmit the acquired pressure parameter to the Bluetooth driver, which continues to report the pressure parameter to the upper module (e.g., the data processing module of the HAL layer), which can process the pressure parameter accordingly. In other embodiments, the antenna (e.g., antenna 2 in the electronic device 210) can resend the received pressure signal to other devices, such as a server or other electronic devices. Figure 2 Figure 2

[0114] Optionally, the MCU 201 is further configured to receive and process data from the electrode 205. For example, the electrode 205 can be located on the tip side of the electronic device. For example, when the tip of the stylus 200 is close to the screen of the electronic device, the electrode 205 of the stylus can output a signal with a specified frequency (which can be referred to as a radio frequency).

[0115] Optionally, the MCU 201, the pressure sensor 202, the Bluetooth IC 203, and the Bluetooth antenna 204 can be integrated into the same chip, or can be separate elements connected through a bus.

[0116] ​​In some embodiments, the handwriting pen of the present application can also use other short-range wireless technologies to interact with electronic devices, such as electronic devices, for example, Wi-Fi technology, Ultra Wide Band (UWB) and other short-range wireless communication technologies. It can be understood that when the handwriting pen 200 and the electronic device transmit pressure information or square wave signals through Wi-Fi technology, the handwriting pen 200 and the electronic device 210 both have Wi-Fi modules.

[0117] Figure 5 An electronic device and a handwriting pen data interaction schematic diagram provided by an embodiment of the present application.

[0118] Referring to Figure 5 As shown in (1) of the, after the handwriting pen 101 establishes a Bluetooth connection with the electronic device 100, and the distance d between the handwriting pen 101 and the screen of the electronic device 100 is less than a preset distance D, the handwriting pen 101 will receive an uplink signal sent by the electronic device 100, which is used to instruct the electronic device to send a pulse signal. The preset distance D can be determined based on the transmission power of the uplink signal. The handwriting pen sends a pulse signal to the electronic device in response to the uplink signal. The frequency of the pulse signal sent by the electronic device can be set according to actual application conditions, for example, the frequency of the pulse signal sent by the electronic device can be 200KHz, or 300KHz, which is not limited in the embodiment of the present application.

[0119] The pulse signal sent by the handwriting pen will change the capacitance of the capacitance sensor in the electronic device that is close to the tip of the handwriting pen. The detection timing control signal of the electronic device is a signal used to control the detection of the capacitance of the capacitance sensor. As shown in (2) of the, Figure 5 The detection timing control signal includes a high level and a low level, and the electronic device detects the capacitance of the capacitance sensor when the detection timing control signal is at a high level. The frequency of the detection timing control signal can be set according to actual application conditions, for example, set to 240Hz, 1.5KHz, or other values, which are not limited in the embodiment of the present application.

[0120] In some examples, the electronic device can determine the point reporting position by detecting the capacitance of each capacitance sensor, which is used to indicate the contact position of the handwriting pen and the screen of the electronic device determined by the electronic device. It should be noted that due to the protective film on the screen of the electronic device, there may be a small deviation between the actual contact position of the handwriting pen on the screen of the electronic device and the contact position of the handwriting pen and the screen of the electronic device determined by the electronic device (point reporting position).

[0121] In some examples, a first coordinate system is established with the screen edge of the electronic device as the coordinate axis (for example, the longer side of the screen is used as the horizontal axis and the shorter side of the screen is used as the vertical axis), and the reported point position can be represented by the coordinates in the first coordinate system. Figure 5 As shown in (2), when the detection timing control signal is at a high level e, the electronic device detects the capacitance of each capacitance sensor and then determines the horizontal coordinate x1 of the reporting point position 1 based on the capacitance values ​​detected at the high level e. When the detection timing control signal is at a high level f, the electronic device detects the capacitance of each capacitance sensor and then determines the vertical coordinate y1 of the reporting point position 1 based on the capacitance values ​​detected at the high level f. Similarly, the electronic device can also determine the coordinates (x2, y2) of the reporting point position 2.

[0122] The pulse signal sent by the stylus causes the capacitance of a capacitance sensor located closest to the stylus tip in the electronic device to change. The magnitude of this capacitance change is affected by many factors, such as whether the electronic device's screen has a first film layer, which represents a protective film on the electronic device's screen.

[0123] Figure 6 A schematic diagram of a capacitance sensor for detecting capacitance values ​​when the screen of an electronic device has different film layer states, provided in an embodiment of the present application. The following description uses the example of a stylus 101 performing the same contact operation at the same location on the screen of an electronic device 100 without a first film layer and on the screen of an electronic device 100 with a first film layer, respectively.

[0124] like Figure 6 As shown in (1) and (2), in the side view (1) a and (2) c of the electronic device 100, a capacitance sensor array 108 is provided in the screen assembly of the electronic device 100. As can be seen in (1) b and (2) d of the top view of the electronic device 100, the capacitance sensor array 108 includes a plurality of capacitance sensors 1081 arranged in rows and columns, for example, capacitance sensor 1081-1 and capacitance sensor 1081-2 shown in b or d. The specific values ​​of each capacitance in the following examples are all illustrative, and the unit of capacitance is picofarad (pF).

[0125] like Figure 6 As shown in (1), in the scenario where the screen of the electronic device 100 shown in a of (1) does not have the first film layer, the tip of the stylus pen 101 contacts the screen of the electronic device 100. The electronic device 100 detects the capacitance of each capacitance sensor, and the capacitance of the capacitance sensor near the contact position of the stylus pen 101 and the screen is as follows: Figure 6As shown in 108-1 in b of (1), each square is used to indicate a capacitance sensor, and the number in the square is used to represent the capacitance value of the capacitance sensor. For example, the capacitance value detected by capacitance sensor 1081-1 is 2560, and the capacitance value detected by capacitance sensor 1081-2 is 112. It should be noted that in the embodiment of the present application, the interval between the capacitance sensors corresponding to the contact position does not exceed a preset number (for example, 5) of capacitance sensors, and can be regarded as "near the contact position."

[0126] like Figure 6 As shown in (2), in the scene where the screen of the electronic device 100 shown in c of (2) has the first film layer 104, the tip of the stylus 101 contacts the first film layer 104 on the screen of the electronic device 100. It should be noted that Figure 6 In (2), the position of the stylus tip 101 on the screen corresponding to c is the same as the position of the stylus tip 101 on the screen corresponding to a in (1), wherein the position of the stylus tip 101 on the screen refers to the projection position of the stylus tip 101 in the first coordinate system. The first coordinate system is a coordinate system established with the screen edge of the electronic device as the coordinate axis (for example, the longer side of the screen is the horizontal axis and the shorter side of the screen is the vertical axis).

[0127] The electronic device detects the capacitance of each capacitance sensor. The capacitance of the capacitance sensor near the contact position between the tip of the stylus 101 and the first film layer 104 on the screen is as follows: Figure 6 As shown in 108-2 in d of (2), each square is used to indicate a capacitance sensor, and the numbers in the squares are used to represent the capacitance of the capacitance sensor detected by the electronic device. For example, the capacitance of the capacitance sensor 1081-1 is 2204, and the capacitance of the capacitance sensor 1081-2 is 103.

[0128] Each capacitance value varies. For example, take 108-1 and 108-2. Comparing the capacitance distribution shown in 108-1 with that shown in 108-2, the capacitance value of capacitance sensor 1081-1 is 2560 when the screen of electronic device 100 does not have the first film layer, and 2204 when the screen has the first film layer. In contrast, the capacitance value of capacitance sensor 1081-2 is 112 when the screen of electronic device 100 does not have the first film layer, and 99 when the screen has the first film layer. This means that the first film layer on the screen of an electronic device affects the amplitude of the capacitance value variation of the capacitance sensor.

[0129] For the same contact operation of the handwriting pen on the screen of the electronic device with different film layer states, the distance between the pen tip of the handwriting pen and each capacitance sensor will increase when the screen of the electronic device has the first film layer compared with the case that the screen of the electronic device does not have the first film layer, and the first film layer with a certain dielectric constant is added between the handwriting pen and the capacitance sensor, so that the dielectric constant between the handwriting pen and the capacitance sensor changes, resulting in that the capacitance distribution detected by the electronic device when the screen of the electronic device has the first film layer is completely different from the capacitance distribution detected by the electronic device when the screen of the electronic device does not have the first film layer.

[0130] In the foregoing embodiments, only the capacitance of the capacitance sensor near the contact position of the handwriting pen and the screen detected by the electronic device when the handwriting pen contacts the screen of the electronic device is shown. However, in actual application, the electronic device can obtain the capacitance of each capacitance sensor multiple times during the process that the handwriting pen contacts the screen of the electronic device. Each time the capacitance of each capacitance sensor is obtained, a frame of capacitance information is formed.

[0131] Figure 7 A schematic diagram of obtaining capacitance information by an electronic device provided in an embodiment of the present application is shown in FIG. 7. Figure 7 As time t elapses, the electronic device 100 can obtain capacitance information according to a pre-set capacitance sampling rate, which indicates the frequency of obtaining capacitance information by the electronic device. The capacitance sampling rate can be set according to actual application, for example, 240 Hz, 1.5 KHz, or other values, which are not limited in the embodiments of the present application. Each frame of capacitance information includes the capacitance of each capacitance sensor in the capacitance sensor array 108 at the corresponding acquisition time. The capacitance information, for example, the capacitance information 700 obtained at t0 in FIG. 7, the capacitance information 701 obtained at t1, the capacitance information 702 obtained at t2, and the like. Figure 7 As time t elapses, the electronic device 100 can obtain capacitance information according to a pre-set capacitance sampling rate, which indicates the frequency of obtaining capacitance information by the electronic device. The capacitance sampling rate can be set according to actual application, for example, 240 Hz, 1.5 KHz, or other values, which are not limited in the embodiments of the present application. Each frame of capacitance information includes the capacitance of each capacitance sensor in the capacitance sensor array 108 at the corresponding acquisition time. The capacitance information, for example, the capacitance information 700 obtained at t0 in FIG. 7, the capacitance information 701 obtained at t1, the capacitance information 702 obtained at t2, and the like.

[0132] In the foregoing embodiments, only the capacitance of the capacitance sensor near the contact position of the handwriting pen and the screen detected by the electronic device when the handwriting pen contacts the screen of the electronic device is shown. However, in actual application, the electronic device can obtain the capacitance of each capacitance sensor multiple times during the process that the handwriting pen contacts the screen of the electronic device. Each time the capacitance of each capacitance sensor is obtained, a frame of capacitance information is formed. Figure 7The capacitance information 701 acquired at the time t1 shown in (2) is taken as an example to illustrate the distribution of the capacitance in a frame of capacitance information. The capacitance information 701 includes the capacitance of each capacitance sensor at the time t1. In the capacitance information 701, the contact position of the stylus 101 and the capacitance of the capacitance sensor near the contact position are shown in 701-1, where each square represents a capacitance sensor, and the number in each square represents the capacitance of the capacitance sensor, for example, the capacitance of the capacitance sensor 701-10 is 2800, and the capacitance of the capacitance sensor 701-11 is 78. The capacitance of the capacitance sensor far from the contact position of the stylus and the screen is shown in 701-2, for example, the capacitance of the capacitance sensor 701-20 is 2, and the capacitance of the capacitance sensor 701-21 is 1. As can be seen, the closer the distance between the capacitance sensor and the stylus tip, the greater the capacitance of the capacitance sensor.

[0133] As can be seen from the capacitance distribution shown in 701-1 and 701-2, in the capacitance distribution shown in 701-2, the capacitance of each capacitance sensor is less than a preset value (for example, 50), which is a parameter for measuring whether the capacitance of the capacitance sensor is a noise value. These capacitances less than the preset value are considered as noise values. In the capacitance distribution shown in 701-1, the capacitance of each capacitance sensor is greater than the preset capacitance (for example, 50), which can effectively represent the characteristics of the contact between the stylus and the screen of the electronic device, and can be considered as valid information.

[0134] Figure 8 A flowchart of a first model training method provided by an embodiment of the present application. The first model training method is applied to a server, and the server includes a preprocessing module and a training module. The first model is used to predict correction parameters. As shown in Figure 8 The first model training method includes steps S81-S85.

[0135] In step S81, the preprocessing module acquires contact sample data and handwriting correction parameters of the electronic device in multiple use states.

[0136] The use state of the electronic device includes at least the following states: with a protective film and being charged, without a protective film and being charged, with a protective film and not being charged, and without a protective film and not being charged.

[0137] In some embodiments, the use state of the electronic device further includes one or more of the following: the posture of the electronic device, the thickness of the protective film, the dielectric constant of the protective film, and the contact pressure of the stylus when contacting the screen of the electronic device.

[0138] The contact sample data is used to indicate the process of the stylus contacting the screen of the electronic device, the device state data obtained by the electronic device and the data generated by the stylus contacting. The contact sample data at least includes charging state information, capacitance information and point reporting information, wherein the charging state information is used to indicate whether the electronic device is being charged; the capacitance information includes multiple frames of capacitance value information; and the point reporting information indicates a coordinate set of a point reporting position. The point reporting position is the contact position of the stylus and the screen of the electronic device determined by the electronic device.

[0139] In some examples, the contact sample data further includes an auxiliary parameter indicating a parameter that has an impact on the capacitance value of the capacitance value sensor when the stylus contacts the screen of the electronic device. The auxiliary parameter includes one or more of posture information, pressure information, sensor index information, a film layer thickness of a first film layer and a dielectric constant of the first film layer. Wherein the posture information is used to represent the posture of the electronic device, and the posture of the electronic device will affect the capacitance value of the capacitance value sensor when the stylus contacts the screen of the electronic device; the pressure information is used to represent the pressure value when the stylus contacts the screen of the electronic device, and the greater the pressure value, the closer the distance between the stylus tip and the capacitance value sensor in the electronic device, and the capacitance value of the capacitance value sensor will change accordingly; the film layer thickness of the first film layer will affect the distance between the stylus and the capacitance value sensor when the stylus contacts the screen of the electronic device, and further affect the capacitance value of the capacitance value sensor. The dielectric constant of the first film layer will also affect the capacitance value of the capacitance value sensor when the stylus contacts the screen of the electronic device.

[0140] Figure 9 The contact sample data acquisition scene example provided by the embodiment of the application is shown in the figure. The acquisition scene includes an electronic device 100, a stylus 101, a mechanical arm 109 and a test device (not shown in the figure). The test device is connected with the mechanical arm 109 and is used to control the movement of the mechanical arm 109 according to the control instruction input by the test personnel. The mechanical arm 109 is used to fix the stylus 101 and drive the stylus 101 to touch the screen of the electronic device 100 according to the control instruction of the test device, for example, the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, and the contact operation includes a uniform speed moving operation on the screen after the stylus 101 tip contacts the screen, wherein the contact trajectory is a straight line segment. It should be noted that the mechanical arm 109 can make the stylus 101 touch the preset position in the screen of the electronic device 100 with a preset pressure value in different acquisition scenes.

[0141] Example one, as Figure 9(1) as shown, the use state of the electronic device is: no film state, the angle with the horizontal plane is 45°, and the electronic device is not charged. The mechanical arm 109 drives the stylus 101 to write a straight line, such as straight line segment 900, on the screen of the electronic device 100. The electronic device 100 obtains attitude information through the gyroscope sensor, obtains capacity information through the capacity sensor, and reports point information. The stylus 101 obtains pressure information and sends the pressure information to the electronic device.

[0142] Example two, as shown in Figure 9 (2) as shown, the use state of the electronic device is: no film state, the angle with the horizontal plane is 0°, and the electronic device is not charged. The process of obtaining the contact sample data is not repeated here.

[0143] Example three, as shown in Figure 9 (3) as shown, the use state of the electronic device is: film state, the angle with the horizontal plane is 45°, and the electronic device is not charged. The process of obtaining the contact sample data is not repeated here.

[0144] Example four, as shown in Figure 9 (4) as shown, the use state of the electronic device is: no film state, the angle with the horizontal plane is 0°, and the electronic device is not charged. The process of obtaining the contact sample data is not repeated here.

[0145] Example five, as shown in Figure 9 (5) as shown, the use state of the electronic device is: film state, the angle with the horizontal plane is 45°, and the electronic device is charged. The process of obtaining the contact sample data is not repeated here.

[0146] Example six, as shown in Figure 9 (6) as shown, the use state of the electronic device is: film state, the angle with the horizontal plane is 0°, and the electronic device is charged. The process of obtaining the contact sample data is not repeated here.

[0147] It can be understood that the use state of the electronic device in the above Figure 9 is only an example, and in actual application, the use state can include more types, for example, in the use state, the attitude of the electronic device can be an angle of 20° with the horizontal plane, and can also be an attitude represented by a yaw angle (Yaw), a pitch angle (Pitch) and a roll angle (Roll). In addition, the preset position of the stylus 101 contacting the screen of the electronic device 100 can be multiple positions, for example Figure 9 the position of the straight line segment 900 and the position of the straight line segment 901 in (1).

[0148] In an embodiment, the pre-processing module obtains the contact sample data and the handwriting correction parameters of the electronic device in multiple use states, taking the electronic device in the first state as an example. The first state can be any one of the aforementioned states of being pasted and being charged, not pasted and being charged, pasted and not being charged, and not pasted and not being charged, or Figure 9 Any one of the use states (1)-(6) shown in the above embodiment. This step (step S81) includes steps S811-S816.

[0149] In step S811, the pre-processing module receives the first capacitance information sent by the electronic device.

[0150] The first capacitance information refers to the multi-frame capacitance value information obtained by the electronic device when the stylus performs a contact operation on the screen of the electronic device in the first state.

[0151] Taking the use state (1) shown in the above embodiment as an example, the first capacitance information indicates the multi-frame capacitance value information obtained by the electronic device when the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, and the contact trajectory is a straight line segment 900. Figure 9

[0152] In step S812, the pre-processing module receives the first pressure information sent by the electronic device.

[0153] The first pressure information refers to the pressure value of the stylus when the stylus performs a contact operation on the screen of the electronic device in the first state.

[0154] Taking the use state (1) shown in the above embodiment as an example, the first pressure information indicates the pressure information sent by the stylus 101 to the electronic device 100 when the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, and the contact trajectory is a straight line segment 900. Figure 9

[0155] In step S813, the pre-processing module receives the first attitude information sent by the electronic device.

[0156] The first attitude information refers to the attitude of the electronic device when the stylus performs a contact operation on the screen of the electronic device in the first state. The first attitude information includes the values of the heading angle, the pitch angle, and the roll angle collected by the gyroscope sensor.

[0157] Taking the use state (1) shown in the above embodiment as an example, the first attitude information indicates the attitude of the electronic device 100 when the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, and the contact trajectory is a straight line segment 900. Figure 9 ​​(1) The use state shown is taken as an example, and the first attitude information indicates that when the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, the electronic device obtains attitude information through a gyroscope sensor, that is, the first attitude information in this scenario is an angle of 45° with the horizontal plane.

[0158] It should be noted that when the stylus contacts the screen of the electronic device, the posture of the electronic device has an influence on the capacitance value of the capacitance sensor. The capacitance value of the capacitance sensor is the main data for subsequent prediction of the handwriting correction parameter of the electronic device. Therefore, in order to improve the accuracy of the first model trained subsequently, the attitude information of the electronic device can be used as one of the data for training the first model.

[0159] In step S814, the preprocessing module receives the first charging state information sent by the electronic device.

[0160] The first capacitance information indicates whether the electronic device is being charged when the stylus performs a contact operation on the screen of the electronic device in the first state. The first charging state information includes being charged or not being charged.

[0161] The first state is taken as an example. Figure 9 (1) The use state shown is taken as an example, and the first charging state information indicates whether the electronic device is being charged when the mechanical arm 109 controls the stylus 101 to perform a contact operation on the screen of the electronic device 100, that is, the first charging state information in this scenario is not being charged.

[0162] It should be noted that when the stylus contacts the screen of the electronic device, whether the electronic device is being charged has an influence on the capacitance value of the capacitance sensor. The capacitance value of the capacitance sensor is the main data for subsequent prediction of the handwriting correction parameter of the electronic device. Therefore, in order to improve the accuracy of the first model trained subsequently, the charging state information of the electronic device can be used as one of the data for training the first model.

[0163] In step S815, the preprocessing module receives the first report point information sent by the electronic device.

[0164] The first report point information indicates a coordinate set of a report point position determined by the electronic device in the process of the stylus performing a contact operation on the screen of the electronic device in the first state.

[0165] The first state is taken as an example. Figure 9 (1) The use state shown is taken as an example, and the first report point information indicates a coordinate set of a report point position determined by the electronic device in the process of the mechanical arm 109 controlling the stylus 101 to perform a contact operation on the screen of the electronic device 100, and the contact trajectory is a straight line segment 900.

[0166] In step S816, the pre-processing module obtains the film layer thickness of the first film layer and / or the dielectric constant of the first film layer.

[0167] The film layer thickness of the first film layer and / or the dielectric constant of the first film layer are inputted by the tester into the pre-processing module of the server. The film layer thickness of the first film layer and / or the dielectric constant of the first film layer can be obtained according to actual test or determined according to product parameters provided by the manufacturer of the first model.

[0168] It should be noted that the above-mentioned steps S811-S816 are not limited in the order of execution. In addition, any one of the above-mentioned steps S812, S813 and S816 can be selected to be executed or not executed according to actual application. In some examples, the first contact sample data of the electronic device obtained by the pre-processing module in the server includes the above-mentioned first capacitance information, first charge state information and first point reporting information. In other examples, the first contact sample data of the electronic device obtained by the pre-processing module in the server includes the above-mentioned first capacitance information, first pressure information, first posture information, first charge state information, first point reporting information, film layer thickness of the first film layer and dielectric constant.

[0169] In step S817, the pre-processing module determines the first handwriting correction parameter based on the first point reporting information.

[0170] Figure 10 A schematic diagram for determining a handwriting correction parameter is provided in an embodiment of the present application. As shown in (1) of Figure 10 The mechanical arm 109 fixes the handwriting pen 101 and controls the handwriting pen 101 to perform a contact operation on the screen of the electronic device 100 according to the control instruction of the test equipment, wherein the contact trajectory is a straight line segment 900, and the electronic device 100 obtains the first point reporting information and sends the first point reporting information to the pre-processing module in the server. The step of determining the handwriting correction parameter based on the first point reporting information by the pre-processing module in the server includes the following step one and step two.

[0171] In step one, a coordinate relationship between a theoretical handwriting and an actual handwriting is established according to the first point reporting information.

[0172] The actual handwriting is shown as a straight line segment 903 in (2), and the theoretical handwriting is shown as a straight line segment 902. Figure 10 (2).

[0173] The theoretical handwriting is determined according to a plurality of preset coordinates used to indicate the movement of the mechanical arm 109: determining a coordinate value corresponding to each preset coordinate in the first coordinate system, connecting the coordinate value corresponding to the preset coordinate according to the movement order of the mechanical arm 109, and determining the theoretical handwriting. The theoretical handwriting is shown as a straight line segment 902 in (2). Figure 10As shown by the straight line segment 902 in (2).

[0174] The actual handwriting is determined by the first reporting point information. The first reporting point information includes the coordinate set of the reporting point position determined by the electronic device during the process of the robot arm 109 controlling the stylus 101 to perform a contact operation on the screen of the electronic device 100, wherein the contact trajectory is a straight line segment 900. The reporting point positions in the first reporting point information are connected according to the time sequence of determining the reporting point positions to determine the actual handwriting. The actual handwriting is as follows. Figure 10 As shown in line segment 903 in (2).

[0175] Step 2: Determine the compensation value of each reporting point position according to the coordinate relationship between the theoretical handwriting and the actual handwriting, and determine the compensation value of each reporting point position as the first handwriting correction parameter.

[0176] Among them, according to Figure 10 (2) The coordinate relationship between the theoretical handwriting and the actual handwriting is used to determine the compensation value of each reported point position. Figure 10 As shown in (3), the compensation value of each reported point position is equal to the difference between the actual handwriting position and the theoretical handwriting position. Figure 10 The reporting point position 9031 on the actual handwriting shown in (2) has actual coordinates (a, b). The contact position corresponding to this reporting point position on the theoretical handwriting is 9021, and the coordinates are (a+offsetx, b+offsety). Then the compensation value of this reporting point position 9031 is (offsetx, offsety).

[0177] In an embodiment of the present application, the preprocessing module receives contact sample data of the electronic device in the first state, and determines the handwriting correction parameters of the electronic device in the first state based on the contact sample data, thereby obtaining a data basis for training the first model.

[0178] It is understandable that in the above embodiment, in steps S811 to S816, only the first state is Figure 9 (1) shows an example of how the pre-processing module in the server obtains the contact sample data and handwriting correction parameters. However, it is understandable that in the scenario where the first state is other use states, for example, the first state is Figure 9 (2) The usage state shown, or the first state is Figure 9 (3) When in the usage state shown, the way in which the pre-processing module in the server obtains the contact sample data and handwriting correction parameters is similar to that in the above embodiment, and will not be repeated in the embodiment of this application.

[0179] Step S82: The pre-processing module performs a first data processing on the contact sample data to obtain usage feature data.

[0180] The first data processing includes but is not limited to data extraction processing, data fusion processing, etc.

[0181] In one embodiment, the first data processing is described using the first contact sample data acquired by the aforementioned preprocessing module when the electronic device is in the first state as an example. The first contact sample data includes first capacitance information and first charging state information; in some embodiments, the first contact sample data also includes auxiliary parameters. The first data processing of the contact sample data to obtain usage characteristic data (step S82 described above) includes steps S821 through S823.

[0182] Step S821: extract first capacitance characteristic data according to the first capacitance information.

[0183] The first capacitance information indicates multi-frame capacitance value information acquired by the electronic device during a process in which the stylus performs a contact operation on the screen of the electronic device in the first state.

[0184] In an embodiment of the present application, when extracting the first capacitance characteristic data based on the first capacitance information, it is necessary to extract a capacitance matrix from the capacitance information contained in the first capacitance information. For each frame of capacitance information in the first capacitance information, the position of the maximum capacitance is determined from the capacitance information, and the position of the maximum capacitance is the position of the capacitance sensor corresponding to the contact position. It should be noted that when the stylus touches the screen of the electronic device, the capacitance of the capacitance sensor at and near the contact position changes significantly, which can be regarded as valid data and can effectively characterize the contact characteristics between the stylus and the screen of the electronic device. In addition, the input data when training the first model is a matrix, and there are also restrictions on the length and width of the input data. Therefore, for each frame of capacitance information in the first capacitance information, according to the position of the maximum capacitance in the capacitance information and the length and width requirements of the input data, the corresponding matrix is ​​extracted from the capacitance information.

[0185] Figure 11 This is an example diagram of extracting capacitance feature data provided by the embodiment of the present application. Figure 11 In the embodiment, the method of extracting the capacitance characteristic data is exemplarily described by taking the extraction of the first capacitance characteristic data from the first capacitance information as an example.

[0186] like Figure 11 As shown, the first capacitance information 1100 includes multiple frames of first capacitance value information (such as Figure 11 The first capacitance information 601-613 in the first capacitance value information is arranged in the default order of the acquisition time. In the embodiment of the present application, extracting the first capacitance characteristic data from the first capacitance information includes the following steps 1 and 2.

[0187] Step one, from the first capacitance information, according to a preset selection rule, selecting first sample capacitance value information.

[0188] It should be noted that the number of first capacitance value information contained in the first capacitance information can be large, and the large amount of data can cause the training time of the first model to be too long. Therefore, it is necessary to preliminarily select appropriate first capacitance value information to participate in subsequent model training.

[0189] The preset selection rule includes: determining third capacitance value information in the first capacitance information, the third capacitance value information including the maximum capacitance value in the first capacitance information; the third capacitance value information is the Kth frame target capacitance value information in the first capacitance information; selecting [K-N, K+N] frames of first capacitance value information, and determining the first sample capacitance value information based on the [K-N, K+N] frames of first capacitance value information. The N is a preset value, which can be set according to actual use, and is not specifically limited in the embodiment of the application, for example, it can be set to 5, or it can be set to 6.

[0190] Referring to Figure 11 , the third capacitance value information determined, for example, is the capacitance value information 607, which includes the maximum capacitance value in the first capacitance information, for example, the capacitance value 2800 of the capacitance sensor 607-10 in the capacitance value information 607. The capacitance value information 607 is the 7th frame first capacitance value information in the first capacitance information 1100, and the [7-5, 7+5] frames of first capacitance value information (such as the first capacitance value information 602-612 in the first capacitance information 600 in Figure 11 , the [7-5, 7+5] frames of first capacitance value information are selected, and the first sample capacitance value information is determined based on the [7-5, 7+5] frames of first capacitance value information.

[0191] In one embodiment, in the above-mentioned preset selection rule, when the first sample capacitance value information is determined based on the [K-N, K+N] frames of first capacitance value information, the [K-N, K+N] frames of first capacitance value information can be directly determined as the first sample capacitance value information. For example, Figure 11 , the [7-5, 7+5] frames of first capacitance value information (first capacitance value information 602-612) are determined as the first sample capacitance value information.

[0192] In another embodiment, when the stylus is in contact with the screen of the electronic device, the capacitance of the capacitance sensor is unstable during the process of the stylus approaching the screen of the electronic device and leaving the screen, and the capacitance information obtained by the electronic device is of poor validity. Therefore, it is necessary to remove the capacitance information of the frames that are arranged earlier and later from the first sample capacitance information preliminarily selected, and filter out capacitance information with higher data stability. In this embodiment, the preset selection rule includes, and also includes: among the target capacitance information of the [KN, K+N]th frame, selecting the target capacitance information of the [KM, K+M]th frame as the first target capacitance information. M is less than N, and the value of M can be set according to actual usage. It is not specifically limited in the embodiment of the present application. For example, it can be set to 1, or it can be set to 2.

[0193] by Figure 11 For example, from the first capacity value information (first capacity value information 602-612) of the [7-5, 7+5]th frame, the target capacity value information (first capacity value information 604-610) of the [7-3, 7+3]th frame is selected as the first target capacity value information.

[0194] Step 2: extracting a capacity matrix of a preset size from each frame of first capacity information in the first sample capacity information, and combining the capacity matrix into first capacity feature data.

[0195] Taking the first capacitance information 607 obtained at time t1 as an example, the input data of the first model is a matrix. If the length and width of the input data are required to be 7*7, a corresponding 7*7 matrix 607-1 is extracted from the first capacitance information 607. The center of matrix 607-1 is the maximum capacitance value in capacitance information 607, that is, the capacitance value of capacitance sensor 607-10, which is 2800.

[0196] In the embodiment of the present application, for example, to improve the accuracy of the training data, it is necessary to obtain the corresponding 7*7 matrix from the multiple frames of first capacitance information contained in the first sample capacitance information, and together form the first capacitance feature data. Figure 11 The first capacitance information 601-611 in the first capacitance information 601-611 is obtained from each corresponding 7*7 matrix, and the capacitance matrix constitutes the first capacitance feature data 1101. The size of the first capacitance feature data 1101 is 11×7×7.

[0197] It should be noted that when the contact time of the stylus with the screen of the electronic device is less than the preset time length, the electronic device cannot obtain enough frames of capacitance information, that is, the first capacitance information contains less than F frames of capacitance information. When the capacitance information obtained by the electronic device is less than F frames, the stable capacitance information (the [K-M, K+M] frame capacitance information) contained in the less than F frames of capacitance information is insufficient to support the first model training. The preset time length can be determined according to the capacitance sampling rate of the electronic device, for example, 0.2s or 0.3s, which is not limited in the embodiment. The value of F is 2N+1.

[0198] For example, at least 7 frames of stable capacitance information are required to support the first model training. When the first capacitance information contains 11 frames of capacitance information, the stable capacitance information (for example, the remaining capacitance information after removing the first 2 frames and the last 2 frames) is 7 frames. When the first capacitance information contains only 7 frames of capacitance information, the stable capacitance information (for example, the remaining capacitance information after removing the first 2 frames and the last 2 frames) is only 3 frames, which cannot support the first model training. Therefore, in the above-mentioned second rule, (2N+1) frames of capacitance information are preliminarily selected from the first capacitance information, and the capacitance information of the last (N-M) frames after the first (N-M) frames in the order is discarded from the (2N+1) frames of capacitance information. If (2N+1) frames of capacitance information cannot be preliminarily selected from the first capacitance information, the first capacitance information can be discarded and not used for model training.

[0199] In step S822, the first capacitance feature data and the first charging state information are subjected to data fusion processing to obtain first fusion feature data.

[0200] The data fusion processing is used to indicate a multi-level process of processing the association of data and information from a single and multiple sources. The data fusion processing mode includes but is not limited to projection, splicing, addition and the like. In actual application process, the specific selection of the data fusion processing mode can be selected according to the specific situation, which is not limited in the embodiment.

[0201] In a possible implementation, in the case that the contact sample data further includes an auxiliary parameter, the preprocessing module can perform data fusion processing on the first capacitance feature data, the first pressure information and the auxiliary parameter to obtain the first fusion feature data. The auxiliary parameter includes one or more of attitude information, pressure information, film layer thickness of the first film layer and dielectric constant of the first film layer.

[0202] It should be noted that the auxiliary parameter indicates a parameter that has an effect on the capacitance of the capacitance sensor when the stylus contacts the screen of the electronic device. Therefore, participating the auxiliary parameter in the training process of the first model can improve the accuracy of the trained first model.

[0203] Step S823, performing feature enhancement processing on the first fused feature data to obtain first use feature data.

[0204] The feature enhancement processing is used to realize dimension expansion of the feature data.

[0205] In one example, the feature enhancement processing on the first fused feature data includes: normalizing the first fused feature data; grouping the first fused feature data and the normalized first fused feature data to form two-dimensional feature data; taking natural logarithm of the two-dimensional feature data; and grouping the two-dimensional feature data and the natural logarithm of the two-dimensional feature data to form three-dimensional feature data, which is the obtained first use feature data. The normalization method can be selected according to actual application, for example, data type normalization or format normalization, which is not specifically limited in the embodiments of the present application.

[0206] In this example, the feature enhancement processing on the first fused feature data can reduce invalid data in the first fused feature data, prevent underfitting of the trained first model, and make the trained first model have stronger generalization ability.

[0207] In this embodiment, through the above steps S821-S823, the use feature data is obtained based on the contact sample data. Compared with the contact sample data, the use feature data has reduced invalid data, which is convenient for model training.

[0208] Step S83, the pre-processing module labels the use feature data based on the handwriting correction parameter in the first use state to obtain training feature data.

[0209] The labeling is a process of adding labels to data. The pre-processing module labels the use feature data based on the handwriting correction parameter in the first use state to obtain the training feature data, including: the pre-processing module adds the corresponding handwriting correction parameter as a label to each use feature data to obtain the training feature data.

[0210] It should be noted that the handwriting correction parameter of the electronic device is different in different use states, and therefore the handwriting correction parameter label added to the use feature data obtained in different use states is also different. For example, when the use state of the electronic device is the state of being attached with a film and being charged, the handwriting correction parameter of the electronic device is the first correction parameter, and therefore the use feature data obtained in the state of being attached with a film and being charged is added with the label of the first correction parameter; when the use state of the electronic device is the state of being attached with a film but not being charged, the handwriting correction parameter of the electronic device is the second correction parameter, and therefore the use feature data obtained in the state of being attached with a film but not being charged is added with the label of the second correction parameter.

[0211] In this embodiment of the present application, the handwriting correction parameter labels on the training feature data can be used as true results to measure the accuracy of the first model output. During subsequent training of the first model, the training feature data is input into the first model to obtain the first model output. Based on this output and the handwriting correction parameter labels on the input training feature data, the error between the output and the true result can be determined.

[0212] In one example, labeling the first usage feature data includes labeling the first usage feature data based on the first handwriting correction parameter to obtain first training feature data.

[0213] Step S84: The preprocessing module sends the training feature data to the training module.

[0214] In this embodiment, training feature data is obtained through the above steps S81-S84. It is understood that, by performing similar operations as the above steps S81-S84 on contact sample data when the electronic device is in different usage states, training feature data under the corresponding usage states can be obtained, providing a data basis for the training process of the first model.

[0215] Step S85: The training module trains the initial first model according to the training feature data to obtain the trained first model.

[0216] Figure 12 A model training diagram provided in an embodiment of the present application is shown in FIG. Figure 12 As shown, the training module takes the training feature data as input data and inputs it into the initial first model 1200 to obtain the corresponding output result. The training feature data is multiple groups, and the multiple groups of training feature data are different. For example, one group of training feature data can be the electronic device in Figure 9 (1) Characteristic data of writing straight line segment 900 in the scene, another set of training characteristic data can be the electronic device in Figure 9 (1) Characteristic data of the straight line segment 901 in the scene, and another set of training characteristic data can be the electronic device in the Figure 9(6) The feature data of the straight line segment 900 written under the scene, and the like. The output result is used to indicate the predicted handwriting correction parameter of the electronic device. The weight of the first model 900 is updated according to the output result and the loss function, and finally a weight parameter satisfying a requirement is obtained. The requirement to be satisfied is that when the first model predicts the handwriting correction parameter of the stylus based on the weight parameter, the error between the predicted handwriting correction parameter of the stylus (output result) and the handwriting correction parameter of the stylus (actual result) indicated by the input data label is less than a preset error. The weight parameter satisfying the requirement is the weight of the first model after training. The loss function is, for example, a focal loss (a loss function improved on the basis of cross entropy for processing sample classification imbalance).

[0217] In the embodiments of the present application, the first model can be a regression network model based on deep learning, for example, any one of linear regression, ploynomial regression, ridge regression, lasso regression and elastic net regression, which is not specifically limited in the embodiments.

[0218] In the embodiments of the present application, through the above steps S81-S85, the first model after training is obtained, which can determine the correction parameter of the electronic device according to the obtained touch data when the stylus touches the screen of the electronic device.

[0219] Figure 13 A flowchart of a first model deployment method provided by the embodiments of the present application. The first model deployment method is applied to a server or an electronic device capable of terminal deployment. As shown in Figure 13 The first model deployment method includes steps S131-S132.

[0220] Step S131, performing preset format conversion on the first model after training.

[0221] The preset format conversion can be conversion from pyTorch (an open source neural network framework) to NCNN (a deployment framework for deploying models to mobile devices).

[0222] In an embodiment of the present application, during the training of the first model, the framework used by the first model may be pyTorch. PyTorch is a large machine learning framework that requires a large amount of storage space. Therefore, a lightweight framework, such as NCNN, is required when deploying the model. In this embodiment, after the first model training is completed, the first model is converted from pyTorch to NCNN to facilitate subsequent deployment of the first model.

[0223] Step S132: deploy the first model and other collaborative modules converted into a preset format in the hardware abstraction layer of the electronic device.

[0224] Among them, such as Figure 3 As shown, the other collaborative modules include a data processing module and a handwriting correction module. The data processing module has the same function as the pre-processing module of the server in the aforementioned embodiment, and is used to process the touch data acquired by the electronic device when the stylus is touched into input data that can be input into the first model. The handwriting correction module is used to configure the correction parameters output by the first model. Similarly, this handwriting correction module is described in detail in the embodiments described below and is not repeated here.

[0225] In an embodiment of the present application, the trained first model is deployed on an electronic device so that the electronic device can automatically determine the correction parameters of the electronic device in the current usage state through the first model.

[0226] Figure 14 A flowchart of a method for correcting coordinates provided in an embodiment of the present application. The method for correcting coordinates is applied to an electronic device, which can be an electronic device that can be written by a stylus, such as a tablet, a mobile phone, or a laptop. Figure 14 As shown, the coordinate correction method includes: steps S1401 to S1407.

[0227] S1401: When the stylus pen contacts the screen of the electronic device, the data processing module obtains target contact data.

[0228] Target contact data indicates the device status data acquired by the electronic device during the stylus contact with the electronic device screen, as well as data generated by the stylus contact. This contact sample data includes, but is not limited to, data such as capacitance and charging status. In some examples, this target contact data also includes data such as posture information and / or pressure information.

[0229] In one embodiment, when the stylus pen contacts the screen of the electronic device, the data processing module obtains target contact data, including the following steps 1 and 2.

[0230] Step one, when the stylus contacts the screen of the electronic device, the data processing module acquires target capacitance information.

[0231] The target capacitance information indicates a set of multiple frames of capacitance value information acquired by the electronic device during the contact operation of the stylus on the screen of the electronic device.

[0232] As shown in Figure 6 , the electronic device is configured with a plurality of array-distributed capacitance sensors, and the plurality of capacitance sensors are always on. During the contact operation of the stylus on the screen of the electronic device, the electronic device acquires the capacitance values of the plurality of capacitance sensors according to a preset capacitance sampling rate. The capacitance sampling rate can be set according to actual application conditions, for example, set to 240Hz, 1.5KHz, or other values, which are not limited in the embodiments of the present application. The capacitance sampling process can be performed by a touch module (or a touch chip) in the electronic device, and the data processing module acquires the capacitance information from the touch module, or the touch module sends the capacitance information to the data processing module.

[0233] The electronic device can form a frame of capacitance value information every time it acquires the capacitance values of the plurality of capacitance sensors. The process of detecting the capacitance values of the capacitance sensors by the electronic device can be referred to the detailed description of the foregoing Figure 6 The target capacitance information includes capacitance value information at multiple time points in one contact operation, and the capacitance value information at each time point includes the capacitance values of the capacitance sensors at the time point.

[0234] Step two, the data processing module acquires charging state information.

[0235] The charging state information is used to indicate whether the electronic device is being charged when the stylus contacts the screen of the electronic device. For example, when the stylus contacts the screen of the electronic device, the electronic device is being charged, and the charging state information indicates a charged state; when the stylus contacts the screen of the electronic device, the electronic device is not being charged, and the charging state information indicates an uncharged state. The data processing module acquires the charging state information from a charging state management module.

[0236] In another embodiment, when the stylus contacts the screen of the electronic device, the data processing module acquires target contact data, and the step further includes the following steps three and / or step four.

[0237] Step three, the data processing module receives target pressure information sent by the stylus.

[0238] The target pressure information is used to indicate the pressure value when the stylus contacts the screen of the electronic device. The stylus acquires the target pressure information during contact with the screen of the electronic device, and sends the target pressure information to the electronic device.

[0239] Step four, the data processing module obtains target posture information.

[0240] The target posture information is used to indicate the posture of the electronic device when the stylus contacts the screen of the electronic device. The posture may be, for example, Figure 9 the posture with an angle of 45° with the horizontal plane shown in (1) of FIG. 1, or Figure 9 the posture with an angle of 0° with the horizontal plane shown in (2) of FIG. 1. The posture may also be a posture with other angles with the horizontal plane, which is not described herein again. The data processing module can obtain the target posture information from the gyroscope sensor.

[0241] In this embodiment, when the stylus contacts the screen of the electronic device, the data processing module obtains corresponding data from the touch module, the gyroscope sensor, the charge management module, and the stylus, respectively, to form target contact data corresponding to the stylus contacting the screen this time, thereby providing a data basis for subsequent analysis.

[0242] S1402, the data processing module performs second data processing on the target contact data to obtain target use feature data.

[0243] The second data processing includes but is not limited to data extraction processing, data fusion processing, feature enhancement processing, etc.

[0244] In one embodiment, the data processing module performs second data processing on the target contact data to obtain feature data, including the following steps one to three.

[0245] Step one, according to the target capacitance information, extracting target capacitance value feature data.

[0246] According to the target capacitance information, extracting target capacitance value feature data includes: selecting first target capacitance value information from the target capacitance information according to a preset selection rule; extracting a capacitance value matrix of a preset size from each frame of target capacitance value information in the first target capacitance value information, and combining the capacitance value matrix into target capacitance value feature data.

[0247] The preset selection rule includes: determining second capacitance value information in the target capacitance information, the second capacitance value information including the maximum capacitance value in the target capacitance information; the second capacitance value information is the Kth frame of target capacitance value information in the target capacitance information; selecting [K-N, K+N] frames of target capacitance value information, determining the first target capacitance value information based on the [K-N, K+N] frames of target capacitance value information, and N is a preset value.

[0248] In one embodiment, [K-N, K+N] frames of target capacitance value information are used as the first target capacitance value information.

[0249] In another embodiment, the target capacitance value information of the [K-M, K+M] frames is selected from the target capacitance value information of the [K-N, K+N] frames as the first target capacitance value information, and M is less than N.

[0250] It should be noted that the manner in which the data processing module extracts the target capacitance value feature data according to the target capacitance information is the same as the manner in which the preprocessing module extracts the first capacitance value feature data according to the first capacitance information in the foregoing step S821. Therefore, the specific manner in which the target capacitance value feature data is extracted according to the target capacitance information can be referred to the detailed description in the foregoing step S821 in the embodiments, which will not be described here again.

[0251] Step two, performing data fusion processing on the target capacitance value feature data and the charging state information to obtain target fusion feature data.

[0252] The manner of data fusion processing includes but is not limited to projection, splicing, addition, etc. In actual application, the specific manner of data fusion processing can be selected according to specific conditions, which is not limited in the embodiments of the present application.

[0253] In a possible implementation, in the case that the target contact data further includes target auxiliary data, the target capacitance value feature data, the charging state information, and the target auxiliary data are subjected to data fusion processing in the foregoing step two to obtain the target fusion feature data. The target auxiliary data includes one or more of target pressure information, target posture information, and sensor index information.

[0254] In an example, in the case that the target contact data further includes target pressure information, the data processing module can perform data fusion processing on the target capacitance value feature data, the target pressure information, and the charging state information to obtain the target fusion feature data. It should be noted that the target pressure information can represent a pressure value when the stylus contacts the screen of the electronic device. The greater the pressure value, the closer the stylus nib to the capacitance sensor in the electronic device, and the capacitance value of the capacitance sensor will change accordingly. Therefore, taking the target pressure information as one of the target contact data can improve the accuracy of the stylus handwriting correction parameter determined by the first model subsequently.

[0255] In another example, in the case that the target contact data further includes target posture information, the data processing module can perform data fusion processing on the target capacitance value feature data, the target posture information, and the charging state information to obtain the target fusion feature data. It should be noted that different posture information of the electronic device will affect the capacitance value of the capacitance sensor when the stylus contacts the electronic device. Therefore, taking the target posture information as one of the target contact data can improve the accuracy of the stylus handwriting correction parameter determined by the first model subsequently.

[0256] In yet another example, in a case where the target contact data further comprises target posture information and target pressure information, the target capacitance feature data, the target pressure information, the target posture information and the charging state information are subjected to data fusion processing to obtain target fusion feature data.

[0257] In still another example, the sensor index information can also be subjected to data fusion processing with the target capacitance feature data, the target pressure information, the target posture information and the charging state information to obtain touch feature data, so as to improve the effectiveness of the target fusion feature data. The sensor index contains coordinate information of each capacitance sensor in the electronic device.

[0258] Step three, performing feature enhancement processing on the target fusion feature data to obtain target use feature data.

[0259] The feature enhancement processing is used to realize dimension expansion of the feature data. The manner of performing feature enhancement processing on the target fusion feature data is the same as that of performing feature enhancement processing on the first fusion feature data in the aforementioned step S823. For details, refer to the detailed description of performing feature enhancement processing on the first fusion feature data in the aforementioned step S823, which will not be described here.

[0260] In this embodiment, the data processing module performs second data processing on the target processing data to obtain input data (target use feature data) that can be input into the first model.

[0261] S1403, the data processing module inputs the target use feature data into the first model.

[0262] The first model is pre-configured in the electronic device and is used to determine the correction parameter of the electronic device.

[0263] S1404, the first model outputs the correction parameter to the handwriting correction module based on the input target use feature data.

[0264] The correction parameter output by the first model is a parameter used to correct the point reporting position determined by the electronic device according to the contact operation when the handwriting pen performs the contact operation on the electronic device. The handwriting correction parameter can indicate the compensation value of each point reporting position in the electronic device when the handwriting pen contacts the screen of the electronic device. In some embodiments, the handwriting correction parameter output by the first model can be in the form of a function.

[0265] S1405, the handwriting correction module corrects the point reporting position based on the correction parameter output by the first model.

[0266] In one example, based on the correction parameters output by the first model, the step of correcting the report point positions comprises: when the stylus is in contact with the screen of the electronic device, for each report point position, selecting a corresponding compensation value from the correction parameters output by the first model; adding the compensation value to the coordinates of the report point position to determine the theoretical position corresponding to the report point position. The theoretical positions corresponding to all the report point positions can constitute the corrected stylus handwriting.

[0267] In the embodiments of the present application, through the above steps S1401-S1405, the correction parameters of the electronic device in the current use state can be accurately determined when the stylus touches the screen of the electronic device, the report point positions determined by the electronic device according to the contact operation of the stylus are accurately corrected, and the user experience is improved.

[0268] It should be noted that, in the process of training the first model, the auxiliary parameters used include the film layer thickness of the first film layer, and the trained first model has the function of implicitly fitting the mapping relationship between the stylus correction parameters and the film layer thickness. That is, when the stylus contacts the screen of the electronic device in the case that the screen has first film layers with different thicknesses, the film layer thickness of the first film layer does not need to be included in the feature data input to the first model, and the correction parameters output by the first model will also be different accordingly, which can effectively improve the accuracy of the correction parameters and improve the user experience.

[0269] It should be further noted that, in the process of training the first model, the auxiliary parameters used include the dielectric constant of the first film layer, and the trained first model also has the function of implicitly fitting the mapping relationship between the dielectric constant of the film layer and the correction parameters. That is, when the stylus contacts the screen of the electronic device in the case that the screen has first film layers with different dielectric constants, the dielectric constant of the first film layer does not need to be included in the feature data input to the first model, and the stylus correction parameters output by the first model will also be different accordingly, which can effectively improve the accuracy of the correction parameters and improve the user experience.

[0270] Figure 15 An application effect schematic diagram is provided for the embodiments of the present application. The electronic device 100 is configured with a first model, which can automatically determine the correction parameters of the electronic device in the current use state. Taking the case where a user uses the stylus 101 to perform a contact operation on the electronic device 100 with a newly attached film and without adjusting the stylus handwriting correction parameters as an example, the contact trajectory of the contact operation is a straight line segment. As shown in FIG. 8, the stylus 101 contacts the screen of the electronic device 100, and the contact trajectory of the stylus 101 is a straight line segment. The electronic device 100 determines the report point positions of the contact trajectory of the stylus 101 according to the contact operation of the stylus 101, and the report point positions are shown as points in FIG. 8. Figure 15As shown, the actual handwriting (the actual handwriting is the trajectory after connecting the report point positions) displayed by the electronic device 100 corresponding to the contact trajectory can include a 1501 part and a 1502 part. Among them, the handwriting of the 1501 part has a problem of distortion and skew, because when the user starts writing, the correction parameter has not been adjusted, and the electronic device uses the unadjusted correction parameter to correct the report point position in the film state, resulting in inaccurate calibration of the report point position, so that the handwriting of the 1501 part determined and displayed based on the report point position has a problem of distortion and skew.

[0271] During the process of the 1501 part, the electronic device can determine a new correction parameter according to the foregoing method of providing correction coordinates, and correct the subsequent report point position according to the determined new correction parameter, so that the subsequent report point position can be accurately calibrated, so that the handwriting of the 1502 part determined and displayed based on the report point position does not have a problem of distortion and skew. Figure 14 The method of providing correction coordinates provided by the present application can determine a new correction parameter, and correct the subsequent report point position according to the determined new correction parameter, so that the subsequent report point position can be accurately calibrated, so that the handwriting of the 1502 part determined and displayed based on the report point position does not have a problem of distortion and skew.

[0272] It should be noted that, Figure 15 The above is an exemplary description of the application effect, and the 1501 part of the handwriting is enlarged for better illustrating the effect of the embodiments of the present application. In actual application, the length of the part of the handwriting with distortion and skew is generally less than 2mm. The correction parameter configuration method provided by the embodiments of the present application can complete the adaptive adjustment of the correction parameter of the electronic device without obvious perception of the user, realize the handwriting optimization of the stylus, and improve the user experience.

[0273] It can be understood that in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to each function. The algorithm steps of each example described in conjunction with the embodiments disclosed herein can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of the present application.

[0274] The steps performed by the electronic device in the handwriting correction coordinate method of the stylus provided by the embodiments of the present application can also be performed by a chip system included in the electronic device, wherein the chip system can include a processor and a Bluetooth chip. The chip system can be coupled with a memory, so that the chip system calls a computer program stored in the memory when running, and realizes the steps performed by the above-mentioned electronic device. Among them, the processor in the chip system can be an application processor or a processor other than an application processor.

[0275] The embodiment further provides a computer readable medium, which stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the related method steps to implement the method in the above embodiment.

[0276] The embodiment further provides a computer program product, which, when executed on a computer, causes the computer to execute the related steps to implement the method in the above embodiment.

[0277] In addition, the embodiment of the present application further provides an apparatus, which can be a chip, a component or a module, and the apparatus can include a processor and a memory connected to each other, wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor can execute the computer execution instructions stored in the memory to enable the chip to execute the method in each method embodiment.

[0278] The electronic device, the computer readable medium, the computer program product or the chip provided by the embodiment are used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method provided above, which will not be described herein again.

[0279] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above.

[0280] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiment described above is only illustrative, for example, the division of the module or unit is only a logical function division, and in actual implementation, another division manner can be used, for example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, which can be electrical, mechanical or other forms.

[0281] The units described as separate components may or may not be physically separate, and the components displayed as units may be one physical unit or multiple physical units, i.e. may be located in one place, or also distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0282] Any content of each embodiment of the present application, and any content of the same embodiment, can be freely combined. Any combination of the above is within the scope of the present application. If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to make a device (which can be a single chip, chip, etc.) or processor execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various program code storage media.

[0283] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for correcting coordinates, characterized in that: Applied to electronic equipment, the method includes: Receive the pulse signal sent by the stylus; Acquiring target capacitance information based on the received pulse signal; the target capacitance information includes multiple frames of target capacitance information; each frame of the target capacitance information is used to indicate the capacitance of each capacitance sensor in the capacitance sensor array of the electronic device; Extracting target capacitance characteristic data according to the target capacitance information; The target capacitance characteristic data, charging state information, pressure information and posture information are subjected to data fusion processing to obtain target fusion characteristic data; the charging state information is used to indicate whether the electronic device is being charged; the pressure information is used to indicate the pressure value when the stylus is in contact with the screen of the electronic device; and the posture information is used to indicate the posture of the electronic device; The target usage feature data is input into the first model to obtain correction parameters; the first model is used to determine the compensation value of each reporting point position based on the coordinate relationship between the theoretical handwriting and the actual handwriting, and the compensation value of each reporting point position is determined as the correction parameter; the actual coordinates corresponding to the handwriting of the handwriting are obtained, and the actual coordinates are corrected according to the determined correction parameters.

2. The method according to claim 1, characterized in that The step of extracting target capacitance characteristic data according to the target capacitance information includes: Selecting first target capacitance value information from the target capacitance information according to a preset selection rule; A capacity matrix of a preset size is extracted from each frame of target capacity information in the first target capacity information, and the capacity matrix is ​​combined into the target capacity feature data.

3. The method according to claim 2, characterized in that The preset selection rules include: Determine second capacitance information in the target capacitance information, where the second capacitance information includes a maximum capacitance in the target capacitance information; and the second capacitance information is the Kth frame target capacitance information in the target capacitance information. [KN, K+N] frame target capacity information is selected, and the first target capacity information is determined based on the [KN, K+N] frame target capacity information, where N is a preset value.

4. The method according to claim 3, characterized in that The preset selection rules also include: The target capacity information of the [KM, K+M]th frame is selected from the [KN, K+N]th frame target capacity information as the first target capacity information, where M is less than N.

5. The method according to claim 2, characterized in that The matrix center of the capacity matrix indicates the maximum capacity in the corresponding target capacity information.

6. The method according to claim 1, characterized in that The electronic device is connected to the stylus via Bluetooth.

7. A model training method, characterized in that: Applied to a server, the method includes: Acquiring contact sample data and correction parameters of an electronic device in multiple usage states; the contact sample data includes at least capacitance information, charging state information, pressure information, and posture information; the capacitance information includes multiple frames of capacitance information; each frame of capacitance information is used to indicate the capacitance value of each capacitance sensor in a capacitance sensor array of the electronic device; the pressure information is used to indicate the pressure value when a stylus contacts the screen of the electronic device; the posture information is used to indicate the posture of the electronic device; the usage states include at least: a film-applied and charged state, a film-unapplied and charged state, a film-applied and uncharged state, and a film-unapplied and uncharged state; When the electronic device is in any of the usage states, obtaining contact sample data and correction parameters of the electronic device in multiple usage states includes: Receiving first capacitance information sent by an electronic device, wherein the first capacitance information includes multiple frames of first capacitance value information; receiving first charging status information sent by the electronic device; receiving first time reporting information sent by the electronic device; According to the first reporting point information, a coordinate relationship between theoretical handwriting and actual handwriting is established; the first reporting point information includes a set of reporting point positions; Determining a compensation value for each of the reporting point positions according to a coordinate relationship between the theoretical handwriting and the actual handwriting, and determining the compensation value for each of the reporting point positions as a first correction parameter; obtaining training feature data based on the contact sample data and the correction parameters; An initial first model is trained according to the training feature data to obtain a trained first model; the trained first model is used to determine correction parameters.

8. The method according to claim 7, characterized in that The step of obtaining training feature data based on the contact sample data and the correction parameters includes: extracting first capacitance characteristic data according to the first capacitance information; performing data fusion processing on the first capacitance characteristic data and the first charging state information to obtain first fused characteristic data; performing feature enhancement processing on the first fused feature data to obtain first usage feature data; The first usage feature data is labeled based on the first correction parameter to obtain first training feature data.

9. The method according to claim 7, characterized in that The step of training an initial first model according to the training feature data to obtain a trained first model includes: Inputting the training feature data into an initial first model to obtain an output result; Based on the output result and the loss function, the weights of the initial first model are updated to obtain the trained first model.

10. The method according to claim 7, characterized in that The first model is indicated as a regression network model based on deep learning; the loss function is indicated as focal loss.

11. The method according to claim 7, characterized in that After training the initial first model according to the training feature data to obtain the trained first model, the method further includes: Converting the trained first model into a preset format; The first model converted into a preset format is deployed on the electronic device.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the one or more processors, the electronic device executes the method for correcting coordinates according to any one of claims 1 to 6.

13. A server, characterized in that: The server includes: one or more processors; Memory; And a computer program, wherein the computer program is stored on the memory, and when the computer program is executed by the one or more processors, the electronic device executes the model training method as described in any one of claims 7-11.

14. A computer storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method for correcting coordinates according to any one of claims 1 to 6.

15. A computer storage medium, characterized in that It includes computer instructions, which, when executed on a server, cause the server to execute the model training method according to any one of claims 7 to 11.

Images