Electronic device system, electronic device, detection apparatus, method, and storage medium

Abstract

The invention provides an electronic equipment system, electronic equipment, a detection device, a detection method and a storage medium. The detection method comprises the following steps: detecting a first space movement track of an interaction pen; detecting a second space movement track of the wearable device; wherein the wearable device is worn on a hand or an arm of a user holding the interaction pen; and according to the first space movement track and the second space movement track, calculating to obtain a pen point space track of the interaction pen. According to the spatial handwriting detection method for the interactive pen provided by the embodiment of the invention, the first spatial movement track of the interactive pen and the second spatial movement track of the wearable device are respectively detected, and the pen point spatial track of the interactive pen is calculated according to the first spatial movement track and the second spatial movement track; according to the detection method, the trajectory interference of the hand can be removed, and the pen point space trajectory of the interaction pen can be accurately obtained.

Description

Technical Field

[0001] The present application relates to the technical field of electronic device input control, and specifically to an electronic device system, an electronic device, a detection device, a method, and a storage medium. Background Art

[0002] Interactive pen input in conventional technical solutions generally inputs content on the touch screen of an electronic device (such as a mobile phone, tablet computer, etc.) by contacting the touch screen. This method can only input planar patterns but not spatial patterns, and its usage scenarios are limited. Summary of the invention

[0003] A first aspect of an embodiment of the present application provides a spatial handwriting detection method of an interactive pen, the detection method comprising:

[0004] Detecting a first spatial movement trajectory of the interactive pen;

[0005] Detecting a second spatial movement trajectory of the wearable device; wherein the wearable device is worn on a hand or arm of a user holding the interactive pen;

[0006] The spatial trajectory of the pen tip of the interactive pen is calculated according to the first spatial movement trajectory and the second spatial movement trajectory.

[0007] In a second aspect, an embodiment of the present application provides an electronic device system, which includes an electronic device, a wearable device and an interactive pen that are communicatively connected to each other, wherein the electronic device system adopts the detection method described in the above embodiment to obtain the spatial trajectory of the pen tip of the interactive pen.

[0008] In a third aspect, an embodiment of the present application provides a spatial handwriting detection device of an interactive pen, the detection device comprising:

[0009] A first detection module, used to detect a first spatial movement trajectory of the interactive pen;

[0010] A second detection module is used to detect a second spatial movement trajectory of the wearable device; wherein the wearable device is worn on the hand or arm of the user holding the interactive pen;

[0011] A calculation module is used to calculate the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory.

[0012] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0013] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the above embodiments when executing the computer program.

[0014] The spatial handwriting detection method of the interactive pen provided in the embodiment of the present application detects the first spatial movement trajectory of the interactive pen and the second spatial movement trajectory of the wearable device respectively, and calculates the spatial trajectory of the pen tip of the interactive pen based on the first spatial movement trajectory and the second spatial movement trajectory. This detection method can remove the trajectory interference of the hand and thus accurately obtain the spatial trajectory of the pen tip of the interactive pen. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 It is a flowchart of an embodiment of a spatial handwriting detection method of an interactive pen of the present application;

[0017] Figure 2 is a schematic structural diagram of an embodiment of an electronic device system of the present application;

[0018] Figure 3 It is a flowchart of another embodiment of the spatial handwriting detection method of the interactive pen of the present application;

[0019] Figure 4 is a flow chart of an embodiment of obtaining calibration parameters;

[0020] Figure 5 It is a structural schematic diagram of an embodiment of the detection device of the present application;

[0021] Figure 6 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0022] Figure 7 It is a schematic diagram of the structure of the computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0023] The present application is further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present application, but are not intended to limit the scope of the present application. Similarly, the following examples are only some embodiments of the present application rather than all embodiments, and all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0024] The terms "first", "second", and "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", and "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the embodiments of the present application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. The terms "including" and "having" in the embodiments of the present application and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or components inherent to these processes, methods, products, or devices.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] As used herein, "electronic devices" (or simply "terminals") include, but are not limited to, devices configured to receive / send communication signals via a wireline connection (e.g., via a public switched telephone network (PSTN), a digital subscriber line (DSL), a digital cable, a direct cable connection, and / or another data connection / network) and / or via a wireless interface (e.g., for a cellular network, a wireless local area network (WLAN), a digital television network such as a DVB-H network, a satellite network, an AM-FM broadcast transmitter, and / or another communication terminal). Communication terminals configured to communicate via a wireless interface may be referred to as "wireless communication terminals", "wireless terminals" or "mobile terminals". Examples of mobile terminals include, but are not limited to, satellite or cellular phones; personal communication system (PCS) terminals that may combine cellular radiotelephones with data processing, fax, and data communication capabilities; PDAs that may include radiotelephones, pagers, Internet / intranet access, web browsers, organizers, calendars, and / or global positioning system (GPS) receivers; and conventional laptop and / or palmtop receivers or other electronic devices that include radiotelephone transceivers. A mobile phone is an electronic device equipped with a cellular communication module.

[0027] Conventional solutions for spatially identifying interactive pen trajectories cannot remove the interference of hand trajectories, and thus the obtained interactive pen tip trajectory is inaccurate. In order to solve the above problem, the present application embodiment provides a spatial handwriting detection method for an interactive pen, please refer to Figure 1 , Figure 1 It is a flow chart of an embodiment of a spatial handwriting detection method of an interactive pen in the present application; it should be noted that the spatial handwriting detection method of an interactive pen in the present application is used in an electronic equipment system.

[0028] See also Figure 2 , Figure 2It is a schematic diagram of the structure of an embodiment of an electronic device system of the present application, wherein the electronic device system includes an electronic device 10, a wearable device 20 and an interactive pen 30 that are communicatively connected to each other; wherein the electronic device 10 may include devices such as mobile phones, tablet computers, laptop computers, etc.; the wearable device 20 includes any one of smart watches, smart bracelets, and smart rings. The interactive pen 30 is a smart interactive pen that can cooperate with the touch display screen of the electronic device 10 for touch input and can perform spatial input. The detection method in the embodiment of the present application is mainly used for trajectory detection when the interactive pen 30 performs spatial input. In this embodiment, the wearable device 20 and the interactive pen 30 are both provided with spatial displacement sensors (21, 31) and wireless communication modules (22, 32). The spatial displacement sensors (21, 31) are used to detect the spatial positions of the wearable device 20 and the interactive pen 30. Specifically, the spatial displacement sensors (21, 31) can be A+G sensors (a+G-sensor, full name Accelerometer-sensor, is an acceleration sensor that can sense changes in acceleration force. Acceleration force is the force acting on an object during the acceleration process. For example, various movement changes such as shaking (angular momentum), falling, rising, and falling can be converted into electrical signals by a+G-sensor) and optical detection sensors, etc. Any sensor that can detect spatial position and changes can be used, and no specific limitation is made here.

[0029] Optionally, the wireless communication module (22, 32) can be in various forms including a Bluetooth module, a GSM module, and an on-site communication module. The detailed features of this part are within the scope of understanding of those skilled in the art, and are not listed and described in detail here. The wireless communication module (22, 32) is used to realize the communication connection between the wearable device 20 and the interactive pen 30 and the electronic device 10, or between the wearable device 20 and the interactive pen 30.

[0030] In addition, the wearable device 20 and the interactive pen 30 may also be provided with structures such as batteries, control circuit boards, and other sensors. The features of these parts are not described in detail here, and the figure only introduces the components related to the detection method in this application.

[0031] Please continue reading Figure 1 The spatial handwriting detection method of the interactive pen in the embodiment of the present application includes but is not limited to the following steps: Step S100, detecting the first spatial movement trajectory of the interactive pen.

[0032] The first spatial movement trajectory may be specifically detected by utilizing the spatial displacement sensor 31 in the interactive pen 30 .

[0033] Please continue reading Figure 1The spatial handwriting detection method of the interactive pen in the embodiment of the present application also includes step S200, detecting a second spatial movement trajectory of the wearable device, wherein the wearable device is worn on the hand or arm of the user holding the interactive pen.

[0034] The wearable device 20 and the interactive pen 30 need to be located on the same arm or finger of the user, wherein the wearable device 20 worn on the arm or wrist can be a smart watch or bracelet, and the wearable device 20 worn on the finger can be a smart ring. The second spatial movement trajectory can be detected by the spatial displacement sensor 21 in the wearable device 20.

[0035] Please continue reading Figure 1 The spatial handwriting detection method of the interactive pen in the embodiment of the present application also includes step S300, calculating the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory.

[0036] In step S300, specifically, the first spatial movement trajectory t1 and the second spatial movement trajectory t2 may be subjected to a difference operation, and the difference (t1-t2) between the first spatial movement trajectory and the second spatial movement trajectory may be used as the pen tip spatial trajectory. The first spatial movement trajectory t1 and the second spatial movement trajectory t2 may include the sum of multiple motion parameters such as spatial position movement and rotation (angular momentum). The main body of the calculation may be completed by the processor of the electronic device 10, and the wearable device 20 and the interactive pen 30 may send the detected trajectory data to the electronic device 10, and then the electronic device 10 completes the calculation.

[0037] In a specific embodiment, the interactive pen can be connected to the mobile phone via Bluetooth, and the watch can be connected to the mobile phone via Bluetooth; in the 3D painting interface of the mobile phone, the user clicks the button on the interactive pen, or clicks the button on the mobile phone interface to randomly start 3D painting; the A+G sensor in the interactive pen detects the relative displacement and the angular momentum of the pen tip, and sends the data to the mobile phone via Bluetooth; the A+G sensor in the watch synchronously detects the relative displacement and sends the data to the mobile phone via Bluetooth; the mobile phone draws a three-dimensional image 1 (t1) on the mobile phone processor based on the received interactive pen A+G sensor data. The mobile phone draws a three-dimensional image 2 (t2) on the mobile phone processor based on the received watch A+G sensor data. The mobile phone draws the trajectory of the human arm (wrist) based on image 2; the processor of the mobile phone makes a "difference" between image 1 and image 2, that is, subtracts the displacement component in trajectory 2 from trajectory 1, and the trajectory of the interactive pen relative to the finger can be obtained, that is, when the user waves his arm, the change in the direction of the interactive pen tip is accurately calculated. In addition, the jitter of the user's painting can be processed here to make the animation smoother and in line with the user's ideas; the user's painting results can be rendered, and 4D animations (plus time dimension) can be drawn and combined with music, animation, etc. to make the interface display more vivid.

[0038] The spatial handwriting detection method of the interactive pen provided in the embodiment of the present application detects the first spatial movement trajectory of the interactive pen and the second spatial movement trajectory of the wearable device respectively, and calculates the spatial trajectory of the pen tip of the interactive pen based on the first spatial movement trajectory and the second spatial movement trajectory. This detection method can remove the trajectory interference of the hand and thus accurately obtain the spatial trajectory of the pen tip of the interactive pen.

[0039] See also Figure 3 , Figure 3 It is a flow chart of another embodiment of the spatial handwriting detection method of the interactive pen of the present application; the detection method in this embodiment includes but is not limited to the following steps.

[0040] Step S400: detecting a first preset spatial movement trajectory of the interactive pen.

[0041] The first preset spatial movement trajectory may be detected by utilizing the spatial displacement sensor 31 in the interactive pen 30 .

[0042] Step S500: detecting a second preset spatial movement trajectory of the wearable device.

[0043] In step S500 , the detection may be performed by using the spatial displacement sensor 21 in the wearable device 20 .

[0044] Step S600: obtaining calibration parameters according to the first preset spatial movement trajectory and the second preset spatial movement trajectory.

[0045] In this step, the following steps may be specifically included. Figure 4 , Figure 4 1 is a flow chart of an embodiment of obtaining calibration parameters, including: step S610, calculating the trajectory difference between the first preset spatial movement trajectory and the second preset spatial movement trajectory. For example, the first preset spatial movement trajectory is recorded as t10, the second preset spatial movement trajectory is recorded as t20, and the trajectory difference t30 is t10-t20.

[0046] Please continue reading Figure 4 , further comprising step S620, performing a difference operation between the preset parameters and the trajectory difference to obtain a calibration parameter.

[0047] The preset parameter is a parameter preset in the electronic device that needs to identify a specific trajectory, which can be recorded as t40, and the calibration parameter t00=t40-t30.

[0048] Please continue reading Figure 3, the method for detecting the spatial handwriting of the interactive pen in this embodiment further includes step S100 of detecting the first spatial movement trajectory of the interactive pen; step S200 of detecting the second spatial movement trajectory of the wearable device, where the wearable device is worn on the hand or arm of the user holding the interactive pen. Specifically, the first spatial movement trajectory can be detected by using the spatial displacement sensor 31 in the interactive pen 30. The second spatial movement trajectory can be detected by the spatial displacement sensor 21 in the wearable device 20.

[0049] Please continue to refer to Figure 3 , the method for detecting the spatial handwriting of the interactive pen in this embodiment further includes step S700 of calculating the spatial trajectory of the tip of the interactive pen according to the first spatial movement trajectory, the second spatial movement trajectory, and the calibration parameter.

[0050] In a specific embodiment (taking the wearable device 20 as a smart ring as an example), the following process may be included: the interactive pen Bluetooth and the ring Bluetooth are connected to the mobile phone; in the painting and writing interface of the mobile phone, the user clicks the button on the interactive pen or clicks the button on the mobile phone interface to randomly start painting and writing; the A+G sensor in the interactive pen detects the relative displacement and the angular momentum of the pen tip, and sends the data to the mobile phone through Bluetooth; at the same time, the A+G sensor of the ring detects the relative displacement and the angular momentum, and sends the data to the mobile phone through Bluetooth; since the finger will move along with the writing, it is not possible to simply subtract the ring trajectory from the interactive pen trajectory, but training and calibration are required.

[0051] A simple example is: when the user writes the symbol "、", the middle finger ring detects that the momentum in the X direction is a+L, where L is the component of the arm waving, and a is the movement component of the finger when writing "、", and the ring detects the sum of the two. The interactive pen detects that the X direction is a+b+L, that is, the movement amplitude of the interactive pen is greater than that of the finger. At this time, after the processor subtracts the ring trajectory from the interactive pen trajectory, the obtained b is not the real component of the interactive pen writing "、", and the data needs to be compensated to a+b through training (calibration parameter t00). The training method can be that before use, let the user wear the ring and write N Chinese, English, symbols, etc. with the interactive pen according to the instructions on the mobile phone interface. For example: instruct the user to write the "十" character while raising the hand, and then the processor reads the difference b (t30) between the movement trajectories of the interactive pen and the ring, and then according to the recognition trajectory parameter c (t40) required by the "十" character itself, the calibration parameter c-b (t40-t30=t00) can be obtained. The processor needs to calculate the corresponding calibration parameters according to different writing postures (horizontal, vertical, left-falling stroke, right-falling stroke, dot, hook, etc.), and continuously learn and adjust these calibration parameters during the actual use of the user.

[0052] The spatial handwriting detection method of the interactive pen in this embodiment can obtain a more accurate spatial trajectory of the pen tip by performing preset settings before the user uses it to obtain calibration parameters, and then calculating the spatial trajectory of the pen tip in combination with the calibration parameters.

[0053] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of the detection device of the present application, and the detection device 700 includes: a first detection module 710, a second detection module 720 and a calculation module 730. Among them, the first detection module 710 is used to detect the first spatial movement trajectory of the interactive pen; the second detection module 720 is used to detect the second spatial movement trajectory of the wearable device; the wearable device is worn on the hand or arm of the user holding the interactive pen; the calculation module 730 is used to calculate the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory. For the detailed working process of the first detection module 710, the second detection module 720 and the calculation module 730, please refer to the aforementioned method embodiment, which will not be repeated here.

[0054] The present invention also provides an electronic device. Figure 6 , Figure 6 This is a schematic diagram of the structure of an embodiment of an electronic device of the present application, the electronic device includes a memory 810, a processor 820 and a computer program (instructions) stored in the memory and executable on the processor, and the processor 820 implements the following steps when executing the program. Detecting the first spatial movement trajectory of the interactive pen; detecting the second spatial movement trajectory of the wearable device; wherein the wearable device is worn on the hand or arm of the user holding the interactive pen; calculating the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory.

[0055] In addition, the present application also provides a computer-readable storage medium, see Figure 7 , Figure 7 It is a structural diagram of the computer-readable storage medium of the present application, on which a computer-readable program file 91 is stored. When the program is executed by the processor, the following steps are implemented: detecting a first spatial movement trajectory of the interactive pen; detecting a second spatial movement trajectory of the wearable device; wherein the wearable device is worn on the hand or arm of the user holding the interactive pen; and calculating the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory.

[0056] For the detailed process of the method embodiment executed by the electronic device and the computer storage medium, please refer to the relevant description of the aforementioned embodiment, which will not be repeated here.

[0057] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM), etc.

[0058] The above descriptions are only some embodiments of the present application, and do not limit the protection scope of the present application. Any equivalent device or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A spatial handwriting detection method for an interactive pen, characterized in that: The detection method comprises: Detecting a first spatial movement trajectory of the interactive pen; Detecting a second spatial movement trajectory of the wearable device; wherein the wearable device is worn on a hand or arm of a user holding the interactive pen; The spatial trajectory of the pen tip of the interactive pen is calculated according to the first spatial movement trajectory and the second spatial movement trajectory.

2. The detection method according to claim 1, characterized in that: The step of calculating the pen tip spatial trajectory of the interactive pen based on the first spatial movement trajectory and the second spatial movement trajectory specifically includes performing a difference operation on the first spatial movement trajectory and the second spatial movement trajectory, and taking the difference between the first spatial movement trajectory and the second spatial movement trajectory as the pen tip spatial trajectory.

3. The detection method according to claim 1, characterized in that: The detection method also includes, before the step of detecting the first spatial movement trajectory of the interactive pen, the following steps: detecting a first preset spatial movement trajectory of the interactive pen; detecting a second preset spatial movement trajectory of the wearable device; obtaining calibration parameters according to the first preset spatial movement trajectory and the second preset spatial movement trajectory; wherein the step of calculating the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory specifically includes: calculating the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory, the second spatial movement trajectory and the calibration parameters.

4. The detection method according to claim 3, characterized in that: The step of obtaining calibration parameters according to the first preset spatial movement trajectory and the second preset spatial movement trajectory includes calculating a trajectory difference between the first preset spatial movement trajectory and the second preset spatial movement trajectory, and performing a difference operation between preset parameters and the trajectory difference to obtain the calibration parameters.

5. An electronic equipment system, characterized in that: The electronic device system comprises an electronic device, a wearable device and an interactive pen that are communicatively connected to each other, wherein the electronic device system adopts the detection method described in any one of claims 1 to 4 to obtain the spatial trajectory of the pen tip of the interactive pen.

6. The electronic equipment system according to claim 5, characterized in that: The electronic device includes any one of a mobile phone, a tablet computer, and a laptop computer; the wearable device includes any one of a smart watch, a smart bracelet, and a smart ring.

7. The electronic equipment system according to claim 5, characterized in that: The wearable device and the interactive pen are both provided with a spatial displacement sensor and a wireless communication module.

8. A spatial handwriting detection device for an interactive pen, characterized in that: The detection device comprises: A first detection module, used to detect a first spatial movement trajectory of the interactive pen; A second detection module is used to detect a second spatial movement trajectory of the wearable device; wherein the wearable device is worn on the hand or arm of the user holding the interactive pen; A calculation module is used to calculate the spatial trajectory of the pen tip of the interactive pen according to the first spatial movement trajectory and the second spatial movement trajectory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

Images