Data processing method, display screen and handwriting pen

Abstract

The invention discloses a data processing method, a display screen and a handwriting pen. The method comprises the steps of obtaining spatial displacement data of the handwriting pen in a space; according to a writing mode of the handwriting pen relative to a display screen, converting the spatial displacement data to obtain screen displacement data of the handwriting pen relative to the display screen; wherein the writing mode comprises a mode that the handwriting pen is in contact with the display screen or a mode that the handwriting pen is not in contact with the display screen; and displaying a corresponding input track in the display screen according to the screen displacement data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data processing method, a display screen and a stylus. BACKGROUND

[0002] At present, many electronic devices do not support touch functions due to cost problems, resulting in the inability to realize many functions of stylus input. Electronic devices supporting touch functions require the stylus tip to contact the screen, which limits the scenarios in which electronic devices realize stylus input. SUMMARY

[0003] Therefore, the present application provides a data processing method, a display screen and a stylus, as follows.

[0004] A data processing method comprises the following steps:

[0005] obtaining spatial displacement data of the stylus in space;

[0006] converting the spatial displacement data according to a writing mode of the stylus relative to the display screen to obtain screen displacement data of the stylus relative to the display screen;

[0007] wherein the writing mode comprises a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen;

[0008] displaying a corresponding input track in the display screen according to the screen displacement data.

[0009] In the above method, preferably, the writing mode is determined by the following method:

[0010] obtaining a first included angle of the display screen relative to a horizontal plane;

[0011] obtaining a second included angle of the stylus relative to the horizontal plane;

[0012] determining the writing mode of the stylus relative to the display screen according to the first included angle, the second included angle and acceleration data of the stylus.

[0013] In the above method, preferably, wherein:

[0014] when the difference between the first included angle and the second included angle is within a first angle range and the change in the acceleration component of each coordinate axis in the acceleration data within a unit time is less than or equal to a first threshold value, the writing mode is a mode in which the stylus contacts the display screen for input;

[0015] the difference between the first included angle and the second included angle is in the first angle range, and the change of the acceleration component of each coordinate axis in the acceleration data in a unit of time is greater than or equal to a corresponding second threshold value, the writing mode is a mode in which the stylus does not contact the display screen but inputs towards the display screen;

[0016] the second included angle is in a second angle range, and the change of the acceleration component of only one coordinate axis in the acceleration data in a unit of time is less than or equal to a third threshold value, the writing mode is a mode in which the stylus contacts a physical desktop to input.

[0017] The above method, preferably, obtaining the first included angle of the display screen relative to the horizontal plane comprises:

[0018] collecting first acceleration data of the display screen through a sensor arranged in the middle of the display screen;

[0019] processing the first acceleration data to obtain the first included angle of the display screen relative to the horizontal plane;

[0020] The above method, preferably, obtaining the second included angle of the stylus relative to the horizontal plane comprises:

[0021] collecting second acceleration data of the stylus through a sensor arranged in the stylus;

[0022] processing the second acceleration data to obtain the second included angle of the stylus relative to the horizontal plane.

[0023] The above method, preferably, the writing mode corresponds to a displacement rotation matrix from a writing plane of the stylus to a screen plane of the display screen; different displacement rotation matrices correspond to different writing modes;

[0024] The above method, preferably, the writing mode corresponds to a displacement rotation matrix from a writing plane of the stylus to a screen plane of the display screen; different displacement rotation matrices correspond to different writing modes;

[0025] The above method, preferably, the writing mode corresponds to a displacement rotation matrix from a writing plane of the stylus to a screen plane of the display screen; different displacement rotation matrices correspond to different writing modes;

[0026] The above method, preferably, the displacement rotation matrix corresponding to the writing mode is determined by the following method:

[0027] outputting first prompt information, the first prompt information being used to prompt a clicking operation of at least three target points by the stylus, the range formed by the target points being a writing range of the stylus;

[0028] obtaining point coordinates of at least three target points clicked by the stylus;

[0029] establishing a displacement rotation matrix from a writing plane represented by the point coordinates to a screen plane of the display screen according to the writing plane and the screen plane of the display screen.

[0030] The method further comprises:

[0031] in response to a display switching operation, converting the spatial displacement data to a target screen using the displacement rotation matrix corresponding to the writing mode and a screen rotation matrix to obtain screen displacement data of the stylus relative to the target screen;

[0032] displaying a corresponding input track in the target screen according to the screen displacement data of the stylus relative to the target screen.

[0033] The method further comprises:

[0034] obtaining corresponding screen position data of the stylus on the display screen according to the screen displacement data;

[0035] obtaining target pixel position data of an input cursor in the display screen according to the screen position data;

[0036] displaying an input track of the input cursor in a display area of the display screen according to initial pixel position data and the target pixel position data of the input cursor in the display screen.

[0037] A display screen comprises:

[0038] a transceiver configured to receive sensing data collected by a deployed sensor of a stylus;

[0039] a processor configured to obtain spatial displacement data of the stylus in space according to the sensing data, convert the spatial displacement data to obtain screen displacement data of the stylus relative to the display screen according to a writing mode of the stylus relative to the display screen, and display a corresponding input track in the display screen according to the screen displacement data.

[0040] The writing mode comprises a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen.

[0041] A stylus comprises:

[0042] a sensor configured to collect sensing data of the stylus;

[0043] a processor configured to obtain spatial displacement data of the stylus in space according to the sensing data, and convert the spatial displacement data to screen displacement data of the stylus relative to the display screen according to a writing mode of the stylus relative to the display screen;

[0044] wherein the writing mode includes a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen, and the second sensing data is sensing data collected by a sensor disposed in the display screen;

[0045] a transceiver configured to send the screen displacement data to the display screen, and the display screen displays a corresponding input track in the display screen according to the screen displacement data.

[0046] As can be seen from the above technical solution, in the data processing method, the display screen and the stylus disclosed in the present application, by obtaining the spatial position data of the stylus in space, the spatial displacement data can be converted to the screen position data relative to the display screen according to the writing mode of the stylus whether contacting the display screen or not, and then the corresponding input track is displayed in the display screen according to the screen position data. It can be seen that, in the present application, whether the stylus contacts the display screen or not, that is, whether the display screen supports the touch function or not, the spatial displacement data of the stylus can be converted to the screen position data on the display screen according to the corresponding writing mode, so that the track input can be performed on the display screen, thereby improving the handwriting experience of the user. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 a flowchart of a data processing method provided by the embodiments of the present application;

[0049] Figure 2 an example diagram for the stylus to input relative to the display screen;

[0050] Figure 3 an example diagram for establishing a conversion relationship between the screen plane and the writing plane in the embodiments of the present application;

[0051] Figure 4 an example diagram for the proportion between the screen plane and the handwriting range in the embodiments of the present application;

[0052] Figure 5Part of flow chart of data processing method provided by the embodiment of the application;

[0053] Figure 6 Another part of flow chart of data processing method provided by the embodiment of the application;

[0054] Figure 7 Example diagram of the first mode in the embodiment of the application;

[0055] Figure 8 Example diagram of the two second modes in the embodiment of the application;

[0056] Figure 9 Example diagram of the third mode in the embodiment of the application;

[0057] Figure 10 Part of flow chart of data processing method provided by the embodiment of the application;

[0058] Figure 11 Example diagram of entering the writing mode to establish the displacement rotation matrix in the embodiment of the application;

[0059] Figure 12 Another example diagram of establishing the displacement rotation matrix in the embodiment of the application;

[0060] Figure 13 Example diagram of the display screen switching to establish the writing relationship with the stylus in the embodiment of the application;

[0061] Figure 14 Structural schematic diagram of the display screen provided by the embodiment of the application;

[0062] Figure 15 Structural schematic diagram of the stylus provided by the embodiment of the application;

[0063] Figure 16 Hardware architecture diagram of the stylus suitable for the scenario of the stylus writing on the notebook screen in the embodiment of the application;

[0064] Figure 17 Interaction schematic diagram of the stylus and the notebook screen for realizing the handwriting input between the stylus and the notebook screen in the embodiment of the application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the application.

[0066] Reference Figure 1 Fig. 1 is a flow chart of an implementation of a data processing method provided by an embodiment of the present application, which can be applied to a handwriting pen or a display screen. As shown in Fig. 1, a user can hold a handwriting pen to perform handwriting input relative to a display screen. The handwriting pen can be in contact with the display screen or not in contact with the display screen. The display screen can support a touch function (capable of detecting a touch input operation) or not support the touch function. The technical solution in the embodiment is mainly used to improve the handwriting experience of the user. Figure 2

[0067] Specifically, the method in the embodiment can include the following steps:

[0068] Step 101: Obtain spatial displacement data of the handwriting pen in space.

[0069] The spatial displacement data is displacement data based on a handwriting pen coordinate system. The handwriting pen coordinate system and a display screen coordinate system have a conversion relationship. The conversion relationship between the handwriting pen coordinate system and the display screen coordinate system can be established when the writing function of the display screen is turned on.

[0070] For example, as shown in Fig. 2, the conversion relationship between the handwriting pen coordinate system and the display screen coordinate system can be a conversion relationship between a writing plane of the handwriting pen and a screen plane of the display screen. Specifically, the conversion relationship can be represented by a displacement rotation matrix between the writing plane of the handwriting pen and the screen plane of the display screen. Figure 3

[0071] In an implementation, when the spatial displacement data is obtained in step 101, the acceleration data of the handwriting pen can be first obtained by a sensor such as a gravity sensor Gsensor or an inertial measurement unit IMU (Inertial Measurement Unit) deployed in the handwriting pen, and then the acceleration data of the handwriting pen is integrated twice to obtain the spatial displacement data of the handwriting pen in space.

[0072] Specifically, after the acceleration data of the handwriting pen is collected by the sensor deployed in the handwriting pen, the acceleration data can be integrated twice by the processor deployed in the handwriting pen to obtain the spatial displacement data of the handwriting pen in space. Alternatively, after the acceleration data of the handwriting pen is collected by the sensor deployed in the handwriting pen, the acceleration data can be transmitted by the handwriting pen to the display screen, and the acceleration data is integrated twice by the processor deployed in the display screen to obtain the spatial displacement data of the handwriting pen in space.

[0073] For example, the processor deployed in the handwriting pen or the processor deployed in the display screen can obtain the spatial displacement data of the handwriting pen in space by the following formula (1) and formula (2): ​​

[0074] (1)

[0075] (2)

[0076] wherein, is an acceleration component of the acceleration data of the stylus on the horizontal axis, is an acceleration component of the acceleration data of the stylus on the vertical axis; is a time variation; is a displacement of the stylus on the horizontal axis in the spatial displacement data; is a displacement of the stylus on the vertical axis in the spatial displacement data.

[0077] Step 102: converting the spatial displacement data to obtain screen displacement data of the stylus relative to the display screen according to a writing mode of the stylus relative to the display screen.

[0078] The writing mode includes a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen. In different writing modes, the displacement rotation matrix between the stylus coordinate system and the display screen coordinate system is different. Based on this, in the embodiment, after the spatial displacement data is obtained, the conversion relationship between the writing plane of the stylus and the screen plane of the display screen, such as the displacement rotation matrix between the writing plane of the stylus and the screen plane of the display screen, can be determined according to the recognized writing mode of the stylus relative to the display screen, and the spatial displacement data is converted to the display screen according to the conversion relationship, so as to obtain the screen displacement data on the display screen.

[0079] It should be noted that the screen displacement data is coordinate data based on the display screen coordinate system, specifically the coordinate data of the screen plane of the display screen. The displacement rotation matrix between the writing plane of the stylus and the screen plane of the display screen at least represents the conversion ratio between the writing plane and the screen plane, for example, as shown in FIG. 2, the plane ratio between the writing plane (writing range) and the screen plane is 2:1, and correspondingly, the conversion ratio between the spatial displacement data and the screen displacement data is 2:1. Figure 4

[0080] Step 103: displaying a corresponding input track in the display screen according to the screen displacement data.

[0081] Specifically, in the embodiment, the input cursor of the display screen can be controlled to move according to the screen displacement data, so as to display a corresponding input track in the display screen.

[0082] ​As can be seen from the above technical solutions, in the data processing method provided by the embodiments of this application, by obtaining the spatial position data of the stylus in space, the spatial displacement data can be converted into screen position data relative to the display screen according to the writing mode of whether the stylus is in contact with the display screen. Then, based on the screen position data, the corresponding input trajectory is displayed on the display screen. Therefore, in this application, regardless of whether the stylus is in contact with the display screen, that is, regardless of whether the display screen supports touch functionality, the spatial displacement data of the stylus can be converted into screen position data on the display screen according to the corresponding writing mode. This allows for trajectory input on the display screen, thereby improving the user's handwriting experience.

[0083] In one implementation, step 103 can be achieved in the following way, such as... Figure 5 As shown:

[0084] Step 501: Based on the screen displacement data, obtain the screen position data corresponding to the stylus on the display screen.

[0085] Specifically, in this embodiment, the stylus's screen displacement data relative to the display screen can be superimposed on the initial position data of the stylus on the display screen according to the screen displacement data to obtain the new position data of the stylus on the display screen, i.e., the screen position data.

[0086] For example, in this embodiment, screen position data can be obtained from screen displacement data using the following formulas (3) and (4):

[0087] (3)

[0088] (4)

[0089] in, and These are the screen position coordinates on the physical width (horizontal axis) and the screen position coordinates on the physical height (vertical axis) of the screen, respectively, in the initial position data of the stylus on the display screen. and These are the screen displacements in the physical width of the screen and the screen displacements in the physical height of the screen, respectively. and These are the screen position coordinates of the stylus on the screen's physical width and screen position coordinates on the screen's physical height, respectively, from the screen position data of the stylus on the display screen.

[0090] Step 502: Obtain the target pixel position data of the input cursor on the display screen based on the screen position data.

[0091] Specifically, in the embodiment, the screen position data is converted into new pixel position data of the input cursor, i.e., target pixel position data, according to the correspondence between the screen resolution and the physical size of the display screen.

[0092] For example, in the embodiment, the screen position data is converted into the target pixel position data of the input cursor by the following formula (5) and formula (6):

[0093] (5)

[0094] (6)

[0095] wherein, and are the screen physical width and the screen physical height of the display screen, respectively; and are the screen resolution width and the screen resolution height (pixel number) of the display screen, respectively; and are the screen position coordinates of the screen position data on the screen physical width (horizontal axis) and the screen position coordinates on the screen physical height (vertical axis), respectively; and are the new pixel coordinates of the input cursor (track point) on the screen resolution width (horizontal axis) and the new pixel coordinates on the screen resolution height (vertical axis), i.e., the target pixel position data.

[0096] Step 503: displaying the input track of the input cursor in the display area of the display screen according to the initial pixel position data and the target pixel position data of the input cursor in the display screen.

[0097] For example, as shown in the figure, the input track is formed by connecting the initial pixel coordinates of the input cursor ( , ) and the new pixel coordinates of the input cursor ( , ), so as to display the input track of the input cursor in the display area.

[0098] It should be noted that the display area in the display screen refers to the mask layer constructed in the display screen, and based on this, the input track of the input cursor is displayed in the mask layer in the embodiment.

[0099] In one implementation mode, the embodiment converts the screen position data into the target pixel position data of the input cursor by the following flow, as shown in Figure 6 .

[0100] Step 601: obtaining a first included angle of the display screen relative to the horizontal plane.

[0101] In the embodiment, the first acceleration data of the display screen can be collected by the sensor arranged in the middle of the display screen, and then the first acceleration data is processed to obtain the first included angle of the display screen relative to the horizontal plane.

[0102] For example, the first acceleration data can be processed by formula (7) to obtain the first included angle in the embodiment:

[0103] (7)

[0104] wherein, is the acceleration component of the first acceleration data of the display screen on the horizontal axis X, is the acceleration component of the first acceleration data of the display screen on the vertical axis Y; is the acceleration component of the first acceleration data of the display screen on the vertical axis Z; is the first included angle.

[0105] Specifically, when the process of determining the writing mode in the embodiment is executed on the stylus, the first included angle sent by the display screen can be received; when the process of determining the writing mode in the embodiment is executed on the display screen, the first included angle can be obtained according to the first acceleration data on the processor of the display screen.

[0106] Step 602: obtaining the second included angle of the stylus relative to the horizontal plane.

[0107] In the embodiment, the second acceleration data of the stylus can be collected by the sensor arranged in the middle of the stylus, and then the second acceleration data is processed to obtain the first included angle of the stylus relative to the horizontal plane.

[0108] For example, the first acceleration data can be processed by formula (8) to obtain the first included angle in the embodiment:

[0109] (8)

[0110] wherein, is the acceleration component of the second acceleration data of the stylus on the horizontal axis X, is the acceleration component of the second acceleration data of the stylus on the vertical axis Y, is the acceleration component of the second acceleration data of the stylus on the vertical axis Z; is the second included angle.

[0111] Specifically, when the process of determining the writing mode in the embodiment is executed on the stylus, the second included angle can be obtained according to the second acceleration on the processor in the stylus; when the process of determining the writing mode in the embodiment is executed on the display screen, the second included angle sent by the stylus can be received.

[0112] It should be noted that the execution sequence between step 601 and step 602 is not limited to the sequence shown in Figure 6 Step 602 is executed first and then step 601 is executed, or steps 601 and 602 are executed synchronously. The different technical solutions achieved by the execution sequence between step 601 and step 602 are within the protection scope of the present application.

[0113] Step 603: determining the writing mode of the stylus relative to the display screen according to the first included angle, the second included angle and the acceleration data of the stylus.

[0114] The writing mode of the stylus relative to the display screen is at least divided into: a first mode, a second mode and a third mode.

[0115] For example, as shown in Figure 7 The first mode is a mode in which the stylus contacts the display screen for input; as shown in Figure 8 The second mode is a mode in which the stylus does not contact the display screen but faces the display screen for input, at this time, the display screen is in a state of being hung on a wall or suspended in the air vertically, and correspondingly, the user holds the stylus suspended in the air and faces the display screen for writing, or the display screen is in a state of being placed on a table horizontally or at an angle, and correspondingly, the user holds the stylus suspended in the air and faces the display screen for writing; as shown in Figure 9 The third mode is a mode in which the stylus does not contact the display screen but contacts a physical table for input, at this time, the display screen can be placed in any position in any state, such as suspended in the air or placed on a table.

[0116] Based on the above implementation, the writing mode can be determined in step 603 by the following way:

[0117] In a case where the included angle difference between the first included angle and the second included angle is in a first angle range, and the change amount of the acceleration component of the stylus on each coordinate axis in a unit time is less than or equal to a first threshold value, the writing mode is a mode in which the stylus contacts the display screen for input, that is, the first mode.

[0118] The first angle range can be an angle range of 80 degrees to 100 degrees, the first included angle and the included angle difference between the second included angle can be represented by . The first threshold value can be represented by represents. Based on this, when the first angle range is between 80 degrees and 100 degrees, and the change amount of the acceleration component on each coordinate axis of the stylus is not large (i.e., less than or equal to the first threshold value), it can be determined that the user holds the stylus to write on the screen on the display screen, i.e., the first mode, as shown in FIG. 1C. Figure 7

[0119] It should be noted that in the first mode, the display screen can have slight jitter due to the input of the stylus, at which time the acceleration component of the display screen can change but the change amount is within a small range.

[0120] When the angle difference between the first included angle and the second included angle is within the first angle range, and the change amount of the acceleration component on each coordinate axis in the acceleration data within a unit time is greater than or equal to the corresponding second threshold value, the writing mode is a mode in which the stylus does not contact the display screen but inputs toward the display screen.

[0121] wherein the second threshold value can be represented by , and is a larger threshold value. Based on this, when the first angle range is between 80 degrees and 100 degrees, and the change amount of the acceleration component on each coordinate axis of the stylus is large (i.e., greater than or equal to the second threshold value), it can be determined that the user holds the stylus to write toward the display screen without contacting the display screen, i.e., the second mode, as shown in FIG. 1D. Figure 8

[0122] It should be noted that in the second mode, the display screen is placed in a fixed position, at which time the acceleration component of the display screen is constant.

[0123] When the second included angle is within the second angle range, and the change amount of the acceleration component on only one coordinate axis in the acceleration data within a unit time is less than or equal to a third threshold value, the writing mode is a mode in which the stylus contacts the physical desktop to input.

[0124] wherein the second angle range can be an acute angle range such as an angle range of 65 degrees to 80 degrees. The third threshold value can be represented by , and is a smaller threshold value. Based on this, when the second included angle is between 65 degrees and 80 degrees, and the change amount of only one acceleration component on each coordinate axis of the stylus is not large (i.e., less than or equal to the third threshold value), it can be determined that the user holds the stylus to write on the physical desktop, i.e., the third mode, as shown in FIG. 1E. Figure 9

[0125] It should be noted that in the third mode, the display screen is placed in a fixed position, at which time the acceleration component of the display screen is constant. ​​​

[0126] In an implementation manner, the writing mode of the stylus relative to the display screen corresponds to a displacement rotation matrix from a writing plane of the stylus to a screen plane of the display screen. Different writing modes correspond to different displacement rotation matrices.

[0127] Based on this, when the spatial displacement data is converted in step 102, the displacement rotation matrix corresponding to the writing mode can be used to multiply the spatial displacement data to obtain the screen displacement data of the stylus relative to the display screen.

[0128] Specifically, when it is determined that the writing mode is the first mode, the displacement rotation matrix corresponding to the first mode is used to multiply the spatial displacement data to obtain the screen displacement data of the stylus relative to the display screen in the first mode. For example, when it is determined that the writing mode is the second mode, the displacement rotation matrix corresponding to the second mode is used to multiply the spatial displacement data to obtain the screen displacement data of the stylus relative to the display screen in the second mode. For example, when it is determined that the writing mode is the third mode, the displacement rotation matrix corresponding to the third mode is used to multiply the spatial displacement data to obtain the screen displacement data of the stylus relative to the display screen in the third mode.

[0129] For example, the spatial displacement data can be converted to the screen displacement data of the stylus relative to the display screen by the following formula (9) in the embodiment:

[0130] (9)

[0131] wherein R is a displacement rotation matrix, is a displacement of the stylus in the spatial displacement data in the space in the horizontal axis X; is a displacement of the stylus in the spatial displacement data in the space in the vertical axis Y; is a displacement of the stylus in the spatial displacement data in the space in the vertical axis Z; is a screen displacement in the screen physical width (horizontal axis X) in the screen displacement data; is a screen displacement in the screen physical height (vertical axis Y) in the screen displacement data; is a screen displacement in the direction perpendicular to the screen (vertical axis Z) in the screen displacement data.

[0132] Specifically, the displacement rotation matrix corresponding to the writing mode can be determined by the following manner, as shown in the following formula (10): Figure 10

[0133] Step 1001: output first prompt information, the first prompt information is used to prompt that at least three target points are clicked by the stylus.

[0134] ​The range formed by the target points is the writing range of the stylus, and the writing range corresponds to the screen range of the display screen to form a writing relationship between the display screen and the stylus. The first prompt information can be output on the display screen or other screen. The display screen can be the screen of a tablet device or a television set, and the other screen can be the screen of a portable device such as a mobile phone.

[0135] For example, as shown in FIG. 1, the first prompt information "Please select four points to determine the writing range and enter the writing mode" can be output on the display screen (such as a tablet device or a television set) or other screen (such as a mobile phone), and the user can click the four vertices of the writing range in the air, on the display screen, or on a physical desktop. Figure 11

[0136] Step 1002: Obtain the point coordinates of the at least three target points clicked by the stylus.

[0137] For example, as shown in FIG. 1, after the user clicks the four vertices (i.e., target points) in the air, on the desktop, or on the display screen, the sensor deployed on the stylus can collect corresponding sensing data, and the stylus or the display screen can obtain the point coordinates of the four vertices according to the sensing data. The point coordinates are coordinate data based on the coordinate system of the stylus. The four vertices can construct a writing plane where the writing range is located. Figure 11 Step 1003: Establish a displacement rotation matrix from the writing plane represented by the point coordinates to the screen plane of the display screen.

[0138] The screen plane of the display screen can be determined according to the sensing data collected by the sensor deployed on the display screen. The screen plane is a plane based on the coordinate system of the display screen. Based on this, in this embodiment, a displacement rotation matrix between the writing plane represented by the point coordinates and the screen plane of the display screen can be established.

[0139] For example, as shown in FIG. 1, in this embodiment, the displacement rotation matrix from the writing plane represented by the four vertices to the screen plane of the display screen can be constructed.

[0140] Figure 11 In another embodiment, the range formed by the target points clicked by the stylus is not limited to the writing range of the stylus, and the target points clicked by the stylus are used to determine a writing plane, and the stylus can write at any position on the writing plane, as shown in FIG. 2.

[0141] Accordingly, the spatial displacement data formed thereby is mapped to screen displacement data through fixed-size mapping or fixed-mapping proportion (displacement rotation matrix) to realize trajectory input. Figure 12

[0142] ​​​Furthermore, in this embodiment, a configuration interface can be provided on the display screen, which provides configuration items for modifying the font size.

[0143] Furthermore, in this embodiment, after establishing the writing relationship between the display screen and the stylus (i.e., the displacement and rotation matrix from the writing plane of the stylus to the screen plane of the display screen), the display screen can be switched to another screen to establish the writing relationship between the other screen and the stylus.

[0144] Specifically, in this embodiment, after obtaining the spatial displacement data of the stylus, in response to the display switching operation, the spatial displacement data is switched to the target screen using the displacement rotation matrix corresponding to the writing mode and the screen rotation matrix to obtain the screen displacement data of the stylus relative to the target screen; then, based on the screen displacement data of the stylus relative to the target screen, the corresponding input trajectory is displayed on the target screen.

[0145] For example, first, the displacement rotation matrix is ​​multiplied by the screen rotation matrix to obtain a superimposed rotation matrix. Then, the superimposed rotation matrix is ​​multiplied by the spatial displacement data to obtain the screen displacement data of the stylus relative to the target screen. Next, based on the screen displacement data, the screen position data of the stylus on the target screen is obtained. Then, based on the screen position data, the target pixel position data of the input cursor on the target screen is obtained. Finally, based on the initial pixel position data and the target pixel position data of the input cursor on the target screen, the input trajectory of the input cursor is displayed in the display area of ​​the target screen, such as in a mask layer.

[0146] The target screen is the screen that needs to establish a writing relationship with the stylus. For example, such as Figure 13 As shown, display screen 1 (such as a laptop) is the screen that initially establishes a writing relationship with the stylus, and display screen 2 (such as a television) is the target screen that needs to establish a new writing relationship with the stylus. Display screen 1 and display screen 2 can collect sensing data through their respective deployed sensors. In this embodiment, a screen rotation matrix between display screen 1 and display screen 2 can be established on display screen 1 or display screen 2. Based on this, the screen rotation matrix is ​​multiplied by the displacement rotation matrix, and the resulting superimposed rotation matrix is ​​the rotation matrix between the writing plane and the screen plane of the target screen. Thus, after obtaining the spatial displacement data of the stylus in space, the superimposed rotation matrix is ​​multiplied by the spatial displacement data to obtain the screen displacement data of the stylus relative to display screen 2. Finally, based on the screen displacement data, the corresponding input trajectory is displayed on display screen 2.

[0147] refer to Figure 14 The present application provides a structural view of a display screen, which may include the following structures:

[0148] The transceiver 1401 is configured to receive the sensing data collected by the deployed sensor, which is referred to as first sensing data.

[0149] The processor 1402 is configured to obtain spatial displacement data of the stylus in space according to the sensing data, convert the spatial displacement data according to a writing mode of the stylus relative to the display screen to obtain screen displacement data of the stylus relative to the display screen, and display a corresponding input track in the display screen according to the screen displacement data.

[0150] The writing mode includes a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen.

[0151] As can be seen from the above technical solution, in the display screen disclosed in the present application, by obtaining the spatial position data of the stylus in space, the spatial displacement data can be converted to the screen position data relative to the display screen according to the writing mode in which the stylus contacts or does not contact the display screen, and then the corresponding input track can be displayed in the display screen according to the screen position data. It can be seen that in the present application, whether the stylus contacts the display screen or not, that is, whether the display screen supports the touch function or not, the spatial displacement data of the stylus can be converted to the screen position data on the display screen according to the corresponding writing mode, so that the track input can be performed on the display screen, thereby improving the handwriting experience of the user.

[0152] Reference Figure 15 The present application provides a structure schematic diagram of a stylus, which can include the following structures.

[0153] The sensor 1501 is configured to collect the sensing data of the stylus, which is referred to as second sensing data.

[0154] The processor 1502 is configured to obtain spatial displacement data of the stylus in space according to the sensing data, convert the spatial displacement data according to a writing mode of the stylus relative to the display screen to obtain screen displacement data of the stylus relative to the display screen.

[0155] The writing mode includes a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen; and the second sensing data is the sensing data collected by the sensor deployed in the display screen.

[0156] The transceiver 1503 is configured to send the screen displacement data to the display screen, and the display screen displays a corresponding input track in the display screen according to the screen displacement data.

[0157] From the above technical solutions, in the handwriting pen disclosed by the application, by obtaining the spatial position data of the handwriting pen in space, the spatial displacement data can be converted to the screen position data relative to the display screen according to whether the handwriting pen contacts the display screen in the writing mode, and then the corresponding input track is displayed in the display screen according to the screen position data. It can be seen that in the application, whether the handwriting pen contacts the display screen, that is, whether the display screen supports the touch function, the spatial displacement data of the handwriting pen can be converted to the screen position data on the display screen according to the corresponding writing mode, so that the track input can be performed on the display screen, thereby improving the handwriting experience of the user.

[0158] In another embodiment, the processor 1502 in the handwriting pen obtains the spatial displacement data of the handwriting pen in space according to the second sensing data collected by the sensor 1501, the transceiver 1401 in the display screen receives the spatial displacement data of the handwriting pen in space sent by the handwriting pen, the processor 1402 in the display screen converts the spatial displacement data according to the writing mode of the handwriting pen relative to the display screen to obtain the screen displacement data of the handwriting pen relative to the display screen, and displays the corresponding input track in the display screen according to the screen displacement data.

[0159] In another embodiment, the processor 1502 in the handwriting pen obtains the spatial displacement data of the handwriting pen in space according to the second sensing data, converts the spatial displacement data according to the writing mode of the handwriting pen relative to the display screen to obtain the screen displacement data of the handwriting pen relative to the display screen, and sends the screen displacement data to the display screen, the transceiver 1401 in the display screen receives the screen displacement data of the handwriting pen relative to the display screen sent by the transceiver 1503 in the handwriting pen, and finally, the processor 1402 in the display screen displays the corresponding input track in the display screen according to the screen displacement data.

[0160] In an implementation manner, the display screen can further include a sensor 1403, the sensor 1403 is used to collect first sensing data such as first acceleration data, the first sensing data can be provided to the processor 1402 in the display screen to determine the first included angle, and then the processor 1402 in the display screen determines the writing mode in combination with the second included angle (obtained by the handwriting pen according to the second sensing data such as the second acceleration data) and the second acceleration data of the handwriting pen, and obtains the displacement rotation matrix corresponding to the writing mode, the displacement rotation matrix is used to convert the spatial displacement data to obtain the screen displacement data of the handwriting pen relative to the display screen.

[0161] In another implementation, after the display screen collects the first sensing data through the sensor 1403, the first included angle determined according to the first sensing data is sent to the stylus by the processor 1402, the processor 1502 in the stylus determines the writing mode according to the first included angle, the second included angle obtained by the stylus (the processor 1502 in the stylus obtains the second included angle according to the second sensing data such as second acceleration data), and the second acceleration data of the stylus, and obtains the displacement rotation matrix corresponding to the writing mode, which is used to convert the spatial displacement data to obtain the screen displacement data of the stylus relative to the display screen.

[0162] Taking a notebook computer without supporting touch function as an example, the technical solutions of the present application are illustrated as follows:

[0163] In the present application, the intelligent stylus (AI PEN) is improved. The stylus can not only be virtually formed as a touch pen on the system side of the display screen, realize the touch pen function, and recognize the ink trace through the screen interaction with the display screen, but also can perform touch pen operation in other space without being limited by the screen to complete the function.

[0164] As shown in Figure 16 , it is a schematic diagram of the hardware architecture of the AI PEN, which includes a power supply module, a sensor, a data processing module (processor), and a data sending module (transceiver), etc. Among them:

[0165] The sensor is used to obtain the 3-axis (horizontal axis X, vertical axis Y, and vertical axis Z) motion state, force state, and other sensing data of the AI PEN;

[0166] The data processing module is used to process the sensing data to calculate the second included angle of the AI PEN relative to the horizontal plane;

[0167] The data sending module is used to send data (such as sensing data, included angle, etc.) to the system side of the notebook computer;

[0168] The power supply module is used to supply power to the above-mentioned modules.

[0169] It should be noted that the hardware architecture of the notebook computer is similar to that of the AI PEN, which will not be described in detail here.

[0170] The way of determining the writing mode of the AI PEN relative to the notebook computer in the present application is described as follows:

[0171] In the present application, the writing scene, i.e. the writing mode, can be determined according to different forms of the notebook computer and different relative positions of the AI PEN and the notebook computer screen, as follows:

[0172] I. Determination condition of the first mode:

[0173] The absolute value of the difference between the first angle between the notebook screen and the horizontal plane and the second angle between the AI PEN and the horizontal plane is between 80 degrees and 100 degrees, and the data of each coordinate axis (horizontal axis X, vertical axis Y, and vertical axis Z) of the AI PEN does not change much, the angle between the screen and the horizontal plane can change slightly, and the AI PEN is determined to be writing on the notebook screen at this time.

[0174] The algorithm logic is as follows:

[0175] 1. Obtain the first angle between the notebook screen and the horizontal plane through formula (7);

[0176] 2. Obtain the second angle between the AI PEN and the horizontal plane through formula (8);

[0177] 3. Judgment condition: If the angle difference between the first angle and the second angle is between 80 degrees and 100 degrees and the acceleration of each axis of the AI PEN does not change much, it can be determined that the user is holding the AI PEN in a writing state on the notebook screen.

[0178] II. Determination condition of the second mode:

[0179] The absolute value of the difference between the first angle between the notebook screen and the horizontal plane and the second angle between the AI PEN and the horizontal plane is between 80 degrees and 100 degrees, and each coordinate axis of the AI PEN has a large data fluctuation, the angle between the screen and the horizontal plane does not change, and the AI PEN is determined to be writing in the space in front of the notebook screen.

[0180] The algorithm logic is as follows:

[0181] 1. Obtain the first angle between the notebook screen and the horizontal plane through formula (7);

[0182] 2. Obtain the second angle between the AI PEN and the horizontal plane through formula (8);

[0183] 3. Judgment condition: If the angle difference between the first angle and the second angle is between 80 degrees and 100 degrees and the acceleration of each axis of the AI PEN changes greatly, it can be determined that the user is holding the AI PEN in a writing state in the air.

[0184] III. Determination condition of the third mode:

[0185] The second angle between the AI PEN and the horizontal plane is between 65 degrees and 80 degrees, and one axis of the three axes of the AI PEN has a small fluctuation, and the AI PEN is determined to be writing on the physical desktop.

[0186] The algorithm logic is as follows:

[0187] 1. Obtain the second angle between the AI PEN and the horizontal plane through formula (8);

[0188] 2. Judgment condition: If the second angle between the AI PEN and the horizontal plane is in other ranges such as 65 degrees to 80 degrees and the AI PEN has a small axis fluctuation, it can be determined that the user holds the AI PEN to write on the physical desktop.

[0189] Accordingly, in the embodiment, when the user writes, as shown in FIG. 1, the input logic of the input track is as follows: Figure 17

[0190] 1. Collection of writing motion data:

[0191] When the user holds the writing (in the air, on the screen or on the desktop), the AI PEN provides the following key sensing data to the processor of the AI PEN:

[0192] Acceleration data: ;

[0193] Angular velocity data: ;

[0194] These data are provided by the IMU inside the AI PEN, and the time interval of each sampling is .

[0195] 2. The processor of the AI PEN determines the writing mode such as the first mode, the second mode or the third mode through the algorithm module, and activates the screen based on the writing mode (i.e. constructs the displacement rotation matrix between the AI PEN and the screen), and then performs coordinate conversion and track derivation:

[0196] i. Initial position:

[0197] Let the coordinate of the initial position of the nib of the AI PEN on the screen be ;

[0198] ii. Displacement calculation:

[0199] According to the acceleration data collected by the IMU (a(t)), the displacement in the own coordinate system (i.e. the space displacement data) is obtained by twice integration, such as formula (1) and formula (2):

[0200] iii. Coordinate system conversion:

[0201] The displacement of the own coordinate system of the AI PEN is converted to the screen coordinate system through the screen rotation matrix R, such as formula (9), to obtain the screen displacement data.

[0202] iv. Update the position on the screen:

[0203] ​​The trajectory point of the nib of the AI PEN on the notebook screen (i.e., screen position data) is obtained through formula (3) and formula (4).

[0204] 3. Mapping to screen pixel coordinates:

[0205] The trajectory point is mapped from actual physical coordinates to pixel coordinates (i.e., target pixel position data) according to the resolution and physical size of the screen through formula (5) and formula (6), thereby realizing trajectory input and displaying to the user.

[0206] The various embodiments are described in a progressive manner in the specification, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0207] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0208] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of both. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0209] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, comprising: obtaining spatial displacement data of a stylus in space; converting the spatial displacement data to screen displacement data of the stylus relative to a display screen according to a writing mode of the stylus relative to the display screen; wherein the writing mode comprises a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen; and displaying a corresponding input track in the display screen according to the screen displacement data. 2.The method of claim 1, wherein the writing mode is determined by: obtaining a first included angle of the display screen relative to a horizontal plane; obtaining a second included angle of the stylus relative to the horizontal plane; and determining the writing mode of the stylus relative to the display screen according to the first included angle, the second included angle, and acceleration data of the stylus. 3.The method of claim 2, wherein: in a case where a difference between the first included angle and the second included angle is within a first angle range and a change in an acceleration component of each coordinate axis in the acceleration data within a unit time is less than or equal to a first threshold value, the writing mode is a mode in which the stylus contacts the display screen for input; in a case where the difference between the first included angle and the second included angle is within the first angle range and the change in the acceleration component of each coordinate axis in the acceleration data within the unit time is greater than or equal to a corresponding second threshold value, respectively, the writing mode is a mode in which the stylus does not contact the display screen but is directed toward the display screen for input; and in a case where the second included angle is within a second angle range and only a change in an acceleration component of one coordinate axis in the acceleration data within the unit time is less than or equal to a third threshold value, the writing mode is a mode in which the stylus contacts a physical desktop for input. 4.The method of claim 2, wherein obtaining the first included angle of the display screen relative to the horizontal plane comprises: collecting first acceleration data of the display screen by a sensor disposed in the display screen; and processing the first acceleration data to obtain the first included angle of the display screen relative to the horizontal plane; and wherein obtaining the second included angle of the stylus relative to the horizontal plane comprises: collecting second acceleration data of the stylus by a sensor disposed in the stylus; and processing the second acceleration data to obtain the second included angle of the stylus relative to the horizontal plane. 5.The method of claim 1, wherein the writing mode corresponds to a displacement rotation matrix from a writing plane of the stylus to a screen plane of the display screen, and different displacement rotation matrices correspond to different writing modes; and converting the spatial displacement data to the screen displacement data of the stylus relative to the display screen comprises: multiplying the spatial displacement data by the displacement rotation matrix corresponding to the writing mode to obtain the screen displacement data of the stylus relative to the display screen. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ wherein, ​ ​ 6. The method of claim 5, wherein the displacement rotation matrix corresponding to the writing mode is determined by: outputting a first prompt information for prompting a clicking operation of at least three target points by the stylus, the target points forming a range of a writing range of the stylus; obtaining point coordinates of the at least three target points clicked by the stylus; and establishing a displacement rotation matrix from a writing plane represented by the point coordinates to a screen plane of the display screen.

7. The method of claim 5, further comprising: in response to a display switching operation, converting the spatial displacement data to a target screen using the displacement rotation matrix corresponding to the writing mode and a screen rotation matrix to obtain screen displacement data of the stylus relative to the target screen; and displaying a corresponding input track in the target screen according to the screen displacement data of the stylus relative to the target screen.

8. The method of claim 1, wherein displaying a corresponding input track in the display screen according to the screen displacement data comprises: obtaining screen position data of the stylus on the display screen according to the screen displacement data; obtaining target pixel position data of an input cursor in the display screen according to the screen position data; and displaying an input track of the input cursor in a display area of the display screen according to initial pixel position data of the input cursor in the display screen and the target pixel position data.

9. A display screen, comprising: a transceiver configured to receive sensing data collected by a deployed sensor of a stylus; and a processor configured to obtain spatial displacement data of the stylus in a space according to the sensing data, convert the spatial displacement data according to a writing mode of the stylus relative to the display screen to obtain screen displacement data of the stylus relative to the display screen, and display a corresponding input track in the display screen according to the screen displacement data, wherein the writing mode comprises a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen.

10. A stylus, comprising: a sensor configured to collect sensing data of the stylus; and a processor configured to obtain spatial displacement data of the stylus in a space according to the sensing data, convert the spatial displacement data according to a writing mode of the stylus relative to a display screen to obtain screen displacement data of the stylus relative to the display screen, wherein the writing mode comprises a mode in which the stylus contacts the display screen or a mode in which the stylus does not contact the display screen; and a transceiver configured to send the screen displacement data to the display screen, and the display screen is configured to display a corresponding input track in the display screen according to the screen displacement data. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​

Images