Handwriting track detection method and device, augmented reality equipment and readable storage medium

Abstract

The invention provides a handwriting track detection method and device, augmented reality equipment and a readable storage medium. The method comprises the following steps: acquiring holding sensing data of a handwriting pen; determining the holding direction of the handwriting pen according to the holding sensing data; acquiring a preset holding angle corresponding to a current user of the handwriting pen, and determining a reference direction on the handwriting pen according to the preset holding angle and the holding direction; acquiring writing pressure sensing data of the handwriting pen; and determining a handwriting track corresponding to the writing pressure sensing data based on the reference direction. The holding habit of the user is determined by setting the preset holding angle, the reference direction is determined based on the holding direction and the preset holding angle, and then the handwriting track is determined based on the reference direction and the writing pressure sensing data, so that the handwriting track can be obtained based on the actual writing habit of the user; the obtained state of the handwriting track is based on the determined reference direction, and accurate detection of the handwriting track can be realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a handwriting trajectory detection method, device, extended reality device and readable storage medium. Background Art

[0002] Currently, the detection of handwriting trajectories usually involves image recognition of the obtained trajectories; however, due to the different writing habits of different users, the handwriting trajectories present different states, and detecting handwriting trajectories in different states may obtain different results, resulting in the problem of low accuracy of handwriting trajectory detection. Summary of the invention

[0003] The main purpose of the present invention is to propose a handwriting trajectory detection method, device, extended reality device and readable storage medium, aiming to solve the problem of low handwriting trajectory detection accuracy in the prior art.

[0004] To achieve the above object, the present invention provides a handwriting trajectory detection method, the handwriting trajectory detection method comprising:

[0005] Get the holding sensing data of the stylus;

[0006] Determining a holding direction of the stylus according to the holding sensing data;

[0007] Acquire a preset holding angle corresponding to a current user of the stylus, and determine a reference direction on the stylus according to the preset holding angle and the holding direction;

[0008] Acquiring writing pressure data of the stylus;

[0009] A handwriting trajectory corresponding to the writing pressure sensing data is determined based on the reference direction.

[0010] Optionally, determining the holding direction of the stylus pen according to the holding sensing data includes:

[0011] Determining a gripping position of the stylus according to the gripping sensing data, wherein the number of the gripping positions is 2;

[0012] The straight line where the gripped position is located is used as the gripping direction of the stylus.

[0013] Optionally, the step of obtaining a preset holding angle corresponding to a current user of the stylus pen includes:

[0014] Acquire a calibration holding direction and a calibration writing direction generated when a current user of the stylus performs writing along a reference trajectory, wherein the reference trajectory includes a trajectory along the reference direction;

[0015] The preset holding angle is obtained according to the calibrated holding direction, the calibrated writing direction and the reference direction.

[0016] Optionally, determining a handwriting trajectory corresponding to the writing pressure sensing data based on the reference direction includes:

[0017] Determine a writing point on the stylus corresponding to the writing pressure data, and a reference point on the stylus corresponding to the reference direction;

[0018] Determining a target rotation angle between the writing point and the reference point;

[0019] Determining the writing direction corresponding to the writing point according to the reference direction and the target rotation angle;

[0020] A handwriting trajectory corresponding to the writing pressure data is determined according to the writing direction.

[0021] Optionally, determining a handwriting trajectory corresponding to the writing pressure data according to the writing direction includes:

[0022] Acquiring motion detection data of the stylus;

[0023] Determining the position of the tip of the stylus pen according to the motion detection data;

[0024] A handwriting trajectory corresponding to the writing pressure data is determined according to the pen tip position and the writing direction.

[0025] Optionally, determining a handwriting trajectory corresponding to the writing pressure data according to the pen tip position and the writing direction includes:

[0026] Determine pressure-sensing sub-data obtained by continuous writing in the writing pressure-sensing data;

[0027] For each of the pressure-sensitive sub-data, the pen tip position corresponding to the initial moment of the pressure-sensitive sub-data is used as the starting point position, and a writing sub-track is generated in the writing direction, wherein the track length in the writing direction is determined based on the pen tip position corresponding to the writing direction;

[0028] The writing sub-trajectories in the pressure-sensitive sub-data are combined to obtain the handwriting trajectory.

[0029] Optionally, the determining the handwriting trajectory corresponding to the writing pressure sensing data based on the reference direction includes:

[0030] Determining a target recognition type corresponding to the writing pressure sensing data;

[0031] Determine a target recognition method corresponding to the target recognition type;

[0032] The handwriting track is identified by the target identification method to obtain writing information corresponding to the handwriting track.

[0033] To achieve the above object, the present invention further provides a handwriting trajectory detection device, which includes a stylus and a data processing device; the stylus includes a gripping sensor module and a writing pressure sensing module, and the data processing device establishes a communication connection with the gripping sensor module and the writing pressure sensing module; wherein:

[0034] The holding sensing module is used to detect holding sensing data of the stylus;

[0035] The writing pressure sensing module is used to detect the writing pressure sensing data of the stylus;

[0036] The data processing device comprises:

[0037] A first acquisition module, used to acquire the grip sensing data of the stylus;

[0038] A first determining module, configured to determine a holding direction of the stylus according to the holding sensing data;

[0039] A second acquisition module, configured to acquire a preset holding angle corresponding to a current user of the stylus pen, and determine a reference direction on the stylus pen according to the preset holding angle and the holding direction;

[0040] A third acquisition module, used to acquire writing pressure data of the stylus;

[0041] The second determination module is used to determine the handwriting trajectory corresponding to the writing pressure data based on the reference direction.

[0042] To achieve the above-mentioned purpose, the present invention also provides an extended reality device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the handwriting trajectory detection method as described above.

[0043] To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the handwriting trajectory detection method described above are implemented.

[0044] The present invention proposes a handwriting trajectory detection method, device, extended reality device and readable storage medium, which obtains the holding sensing data of the stylus; determines the holding direction of the stylus according to the holding sensing data; obtains the preset holding angle corresponding to the current user of the stylus, and determines the reference direction on the stylus according to the preset holding angle and the holding direction; obtains the writing pressure data of the stylus; and determines the handwriting trajectory corresponding to the writing pressure data based on the reference direction. The user's holding habits are determined by setting a preset holding angle, the reference direction is determined based on the holding direction and the preset holding angle, and then the handwriting trajectory is determined based on the reference direction and the writing pressure data, so that the handwriting trajectory can be obtained based on the user's actual writing habits, and the state of the obtained handwriting trajectory is based on the determined reference direction, which can achieve accurate detection of the handwriting trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 It is a flowchart of the first embodiment of the handwriting trajectory detection method of the present invention;

[0048] Figure 2 Schematic diagram of holding detection points in the handwriting trajectory detection method of the present invention;

[0049] Figure 3 It is a schematic diagram of the arrangement of the holding detection module and the writing pressure sensing module in the handwriting trajectory detection method of the present invention;

[0050] Figure 4 Schematic diagram of writing detection points in the handwriting trajectory detection method of the present invention;

[0051] Figure 5 It is a schematic diagram of the structure of the stylus of the present invention;

[0052] Figure 6 It is a schematic diagram of the module structure of the extended reality device of the present invention. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.

[0054] The present invention provides a handwriting trajectory detection method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a handwriting trajectory detection method according to the present invention. The method comprises the following steps:

[0055] Step S10, obtaining the grip sensing data of the stylus;

[0056] The stylus is a device for generating handwriting tracks; that is, the user obtains handwriting tracks by writing with the stylus. It should be noted that the specific type of the stylus can be set based on actual needs, such as a purely electronic stylus, that is, the stylus cannot directly write on an object, such as a paper surface, to obtain ink marks, but can obtain handwriting tracks in the form of data through subsequent steps; the stylus can also realize ink writing, such as writing on an object, such as a paper surface, to obtain ink marks, and at the same time, can also obtain handwriting tracks in the form of data through subsequent steps.

[0057] The grip sensing data is used to indicate the user's grip of the stylus; the grip sensing data can be detected by the grip sensing module set on the stylus; it is understandable that when the user uses the stylus to write, the user needs to hold the stylus, and the data detected by the grip sensing module is different based on the different positions of the user's grip, so the user's grip state of the stylus can be known through the grip sensing data; the specific type of the grip sensing module can be set based on actual needs, such as the grip sensing module can perform pressure detection or touch detection, etc., and the corresponding grip sensing data can indicate the pressure distribution or touch position on the stylus. This embodiment and subsequent embodiments are described by taking the grip sensing module as a pressure sensor as an example, and other types of grip sensing modules can be implemented by analogy.

[0058] Step S20, determining a holding direction of the stylus pen according to the holding sensing data;

[0059] The holding direction indicates the position where the user holds the stylus. It can be understood that the holding direction refers to the direction of the stylus. If the user holds the stylus unchanged, the holding direction will not change when the stylus is in motion. When the user holds the stylus differently, the holding direction will change accordingly.

[0060] When the user holds the stylus pen to write, the holding condition may change. Therefore, it is necessary to update the holding direction of the stylus pen based on the acquired holding sensing data in real time to update the related parameters subsequently obtained from the holding direction.

[0061] Step S30, obtaining a preset holding angle corresponding to the current user of the stylus pen, and determining a reference direction on the stylus pen according to the preset holding angle and the holding direction;

[0062] The reference direction is the reference direction of the handwriting track; the reference direction is the direction of the handwriting track on the stylus, such as the reference direction can indicate the horizontal writing direction and the vertical writing direction of the handwriting track; it can be understood that although the reference direction indicates the direction on the stylus, the state of the reference direction in space does not change with the movement of the stylus, such as when a user writes in a horizontal direction on a paper with horizontal lines with the horizontal lines as a reference, taking writing from left to right as an example, the reference direction at this time can be a direction parallel to the horizontal lines and from left to right on the paper, and the reference direction can also be set to a direction perpendicular to the horizontal lines and from top to bottom on the paper; after the reference direction is determined, other directions on the paper can be determined based on the reference direction, therefore, the reference direction can actually be set to a direction at any angle on the paper; however, for the convenience of calculation and implementation, in this embodiment, the reference direction is set to the horizontal writing direction, and the subsequent description is made using the horizontal writing direction as the reference direction, and other methods can be implemented by analogy.

[0063] The preset holding angle is the angle between the holding direction corresponding to the current user and the reference direction; users usually have relatively fixed pen-holding habits, therefore, when the same user is writing, the relative position between the holding position of the stylus and the writing direction is relatively fixed, that is, the angle between the holding direction and the reference direction is relatively fixed; therefore, in this embodiment, a preset holding angle is set to indicate the user's pen-holding habits, so that after the holding direction is determined, the preset holding angle can be used to deduce the user's current writing reference direction.

[0064] The above content points out that the reference direction is for the handwriting trajectory, which does not change with the movement of the stylus. Therefore, if you need to obtain an accurate handwriting trajectory, you need to determine the reference direction on the stylus. Figure 2 , Figure 2A1, B1, C1, D1, E1, F1, G1, and H1 in the figure respectively refer to different holding detection points in the holding sensing data of the stylus, and A1, E1, C1, G1, B1, F1, D1, and H1 are set in a counterclockwise order; Figure 2 Take the example of 8 holding detection points with a difference of 45° between each holding detection point; at this time, if the holding direction corresponding to the holding sensing data is E1-F1, and the preset holding angle is 45°, then the corresponding reference direction at this time is A1 pointing to B1; it can be understood that at this time, the direction from A1 to B1 is parallel to the writing direction of the handwriting trajectory; thereafter, if the holding direction corresponding to the holding sensing data changes to C1-D1, since the user's preset holding angle is fixed, the reference direction on the stylus changes to E1 pointing to F1, at this time, the writing direction of the handwriting trajectory remains unchanged, but the position corresponding to the reference direction on the stylus changes, and the reference direction on the stylus is still parallel to the writing direction of the handwriting trajectory.

[0065] It should be noted that the above description takes 8 grip detection points as an example. In actual applications, more or fewer grip detection points can be set. The more grip detection points there are, the more accurate the grip direction detection is, the higher the accuracy requirement for the sensor is, and the higher the data processing capability requirement is. Conversely, the fewer grip detection points there are, the less accurate the grip direction detection is, the lower the accuracy requirement for the sensor is, and the lower the data processing capability requirement is. Therefore, the number of grip detection points to be set can be selected based on actual needs. For the setting of specific grip detection points, a sensor with multi-point pressure detection can be used, or a pressure sensor can be set at each grip detection point.

[0066] Step S40, obtaining writing pressure data of the stylus;

[0067] The writing pressure data is the pressure data generated by the tip of the stylus when writing; the writing pressure data is detected based on the writing pressure module, and the writing pressure module can use a pressure sensor; it can be understood that when the stylus is writing, a force in a corresponding direction will be generated based on the movement direction of the tip of the pen. Generally, if a pressure sensor is set on the tip of the pen, the direction indicated by the writing pressure data detected by the pressure sensor is consistent with the direction of movement of the tip of the pen; if a pressure sensor is set around the tip of the pen, the direction indicated by the writing pressure data detected by the pressure sensor is opposite to the direction of movement of the tip of the pen.

[0068] Step S50: determining a handwriting trajectory corresponding to the writing pressure data based on the reference direction.

[0069] The reference direction indicates the writing direction, and the writing pressure data indicates the movement direction of the pen tip. Therefore, a handwriting trajectory with a clear writing direction can be obtained through the reference direction and the writing pressure data.

[0070] It can be understood that the reference direction is determined based on the grip sensor data, and the grip sensor data is detected by the grip sensor module; the writing pressure sensor data is detected by the writing pressure sensor module; therefore, in order to make the reference direction and the writing pressure sensor data have a unified expression, the position settings of the grip sensor module and the writing pressure sensor module are unified in this embodiment; see Figure 3 and Figure 4 The gripping sensor module and the writing pressure sensing module are arranged to overlap in a vertical direction. Figure 4 A2, B2, C2, D2, E2, F2, G2, and H2 in the figure respectively refer to different writing detection points in the writing pressure data of the stylus, and A2, E2, C2, G2, B2, F2, D2, and H2 are set in a counterclockwise order; Figure 4 A2 and Figure 2 The projection of A1 in the vertical direction is the same, and the same applies to other positions. The projections of the holding detection points and writing detection points corresponding to the same letters in the holding sensor module and the writing pressure sensing module are the same in the vertical direction; when the reference direction is determined to be A1 pointing to B1, then correspondingly, the reference direction can also be determined to be A2 pointing to B2. When determining the handwriting trajectory, the reference direction of A2 pointing to B2 can be determined in combination with the writing pressure sensing data.

[0071] For the convenience of subsequent description, the structure of the stylus is now described. The specific structure of the stylus that needs to be described can be set based on actual needs on the basis of satisfying the functions to be implemented in this application. Only an example is given here; see Figure 5The stylus includes a gripping sensor module 1, a writing pressure sensing module 2, a pen tip 3, a shell 4, a PCB5, a SOC, a motion sensor, and a pen cap 6; wherein the pen tip 3 and the pen cap 6 are respectively arranged at the two ends of the shell, the pen tip 3 is coaxially connected by a cylinder and a cone, the tip of the cone faces the outside of the shell 4, the writing pressure sensing module 2 is a hollow cylinder, and the cylinder of the pen tip 3 is matched with the hollow writing pressure sensing module 2; a partition 41 is arranged in the shell 4 to divide the internal space into two parts, the pen tip 3 and the writing pressure sensing module 2 are arranged on one side, and other devices are arranged on the other side; The holding sensor module 1 is arranged on the side of the shell 4, and the specific setting position can be set based on the position where the user may hold it; the PCB5 is arranged in the shell 4 and fixed in the shell 4 by the fixing column 51; the SOC and the motion sensor are arranged on the PCB5, and the holding sensor module 1 and the writing pressure sensing module 2 respectively establish communication connections with the SOC, so that the SOC can receive the detection data of the holding sensor module 1 and the writing pressure sensing module 2; a button 61 is set on the pen cap 6, and the button 61 can be functionally defined, and the button 61 establishes a communication connection with the SOC.

[0072] Among them, the solution of the present application can be executed by SOC, and SOC can obtain the handwriting trajectory based on the relevant data obtained, and then display or recognize the handwriting trajectory. It can also translate, search, solve problems and other processing based on the recognition results. The specific application can be set based on actual needs, and the handwriting trajectory can also be output to other devices.

[0073] The solution of the present application can also be executed by devices other than the stylus, such as an extended reality device. The SOC outputs the acquired relevant data to the extended reality device, such as via Bluetooth. After receiving the relevant data sent by the SOC, the extended reality device obtains the handwriting trajectory, and then displays or recognizes the handwriting trajectory. It can also translate, search, solve problems, etc. based on the recognition results. The specific application can be set based on actual needs.

[0074] This embodiment determines the user's holding habits by setting a preset holding angle, determines a reference direction based on the holding direction and the preset holding angle, and then determines the handwriting trajectory based on the reference direction and the writing pressure data, so that the handwriting trajectory can be obtained based on the user's actual writing habits. The state of the obtained handwriting trajectory is based on the determined reference direction, which can achieve accurate detection of the handwriting trajectory.

[0075] Further, in the second embodiment of the handwriting trajectory detection method of the present invention proposed based on the first embodiment of the present invention, the step S20 includes the steps of:

[0076] Step S21, determining the gripping position of the stylus according to the gripping sensing data, wherein the number of the gripping positions is 2;

[0077] Step S22: taking the straight line where the gripped position is located as the gripping direction of the stylus.

[0078] When holding a stylus, the user generally uses multiple fingers to fix the stylus. Therefore, the stylus may be held in multiple positions, such as holding the stylus with the thumb, index finger and middle finger. However, the holding direction needs to be determined in the present application. Therefore, only two holding positions need to be determined. In order to accurately determine the holding positions, the two positions with the greatest pressure can be determined as the holding positions through the holding sensing data, and then the straight line where the two holding positions are located is used as the holding direction.

[0079] See again Figure 2 If the gripping positions corresponding to the gripping sensing data are E1 and F1 respectively, the corresponding gripping direction is E1-F1; it can be understood that the gripping positions can be two gripping detection points set relatively or not relatively. For example, if the gripping positions corresponding to the gripping sensing data are E1 and B1 respectively, the corresponding gripping direction is E1-B1.

[0080] In other embodiments, in order to improve the accuracy of the holding direction, the structure of the stylus pen may be adjusted; for example, the outer shell of the stylus pen is cylindrical, and the holding sensor module sets two relative and parallel planes at the corresponding position on the outer shell. When the user holds the stylus pen, he will hold it through the two planes, so that the holding direction is always the normal direction of the two planes, thereby reducing the deviation of the holding direction detection and improving the accuracy of the subsequent reference direction determination.

[0081] It should be noted that the stylus is not necessarily perpendicular to the writing plane during the writing process. Therefore, in practical applications, the posture of the stylus can be detected by a motion sensor, so that the holding direction, reference direction, etc. can be projected onto the writing plane based on the posture of the stylus to achieve unified processing.

[0082] Furthermore, in a third embodiment of the handwriting trajectory detection method of the present invention proposed based on the first embodiment of the present invention, the step before step S30 includes the following steps:

[0083] Step S60, obtaining a calibration holding direction and a calibration writing direction generated when the current user of the stylus writes along a reference trajectory, wherein the reference trajectory includes a trajectory along the reference direction;

[0084] Step S70, obtaining the preset holding angle according to the calibrated holding direction, the calibrated writing direction and the reference direction.

[0085] The current user is the user who is holding the stylus; it is understandable that different users have different writing habits, and their corresponding preset holding angles are also different, so it is necessary to set the corresponding preset holding angle for the specific user.

[0086] The calibration holding direction is the holding direction of the current user when writing the reference trajectory; the calibration holding direction is determined in the same manner as the aforementioned holding direction.

[0087] The calibration writing direction is the direction in which the pen tip moves when the current user writes along the reference trajectory; the calibration writing direction can be determined by the writing pressure data collected at this time.

[0088] When using the stylus, the user can calibrate the preset holding angle; the calibration can be triggered manually by the user, or a trigger rule can be set, such as when switching users, when the stylus has not been used for a preset usage time, etc.

[0089] The reference trajectory is a pre-set trajectory that instructs the user to write; it is understandable that in order to facilitate the user to accurately draw the reference trajectory, the reference trajectory needs to be set in a relatively standard manner, such as setting the reference trajectory to a horizontal line from left to right, a vertical line from top to bottom, and a diagonal line from the upper left to the lower right; when specifically obtaining the calibration holding direction, text display or voice prompts can be used to remind the user to write the reference trajectory in turn, thereby obtaining the calibration holding direction corresponding to each reference trajectory.

[0090] The direction corresponding to the reference track is fixed. For example, a horizontal line from left to right corresponds to the horizontal direction from left to right. Figure 2 For example, the reference direction is set to the horizontal direction from left to right. If the calibrated writing direction is A1 to B1, then A1 to B1 is the reference direction. At this time, if the calibrated holding direction is E1 to F1, the preset holding angle is determined to be the angle between the two straight lines E1-F1 and A1-B1, that is, 45°.

[0091] For example, the reference track is a vertical line from top to bottom, which corresponds to the vertical direction from top to bottom. Figure 2 For example, at this time, the reference direction is set to the horizontal direction from left to right. If the calibrated writing direction is D1 to C1, it means that D1 to C1 is the vertical direction. Therefore, A1 to B1 perpendicular to it is the reference direction; at this time, if the calibrated holding direction is E1 to F1, the preset holding angle is determined to be the angle between the two straight lines E1-F1 and A1-B1, that is, 45°.

[0092] For example, the reference track is a vertical line from the upper left to the lower right, which corresponds to the oblique direction from the upper left to the lower right. Figure 2For example, the reference direction is set to the horizontal direction from left to right. If the calibrated writing direction is H1 to G1, it means that H1 to G1 is an oblique direction. Therefore, A1 to B1 with an angle of 45° therewith is the reference direction. At this time, if the calibrated holding direction is E1 to F1, the preset holding angle is determined to be the angle between the two straight lines E1-F1 and A1-B1, that is, 45°.

[0093] In actual calibration, multiple different reference tracks may be set simultaneously to jointly determine the preset holding angle, such as by averaging, weighted averaging, etc.

[0094] When there is a preset holding angle, the newly obtained preset holding angle is compared with the existing preset holding angle. If the difference between the two is greater than the preset angle difference, the existing preset holding angle is updated to the new preset holding angle. If the difference between the two is less than or equal to the preset angle difference, the existing preset holding angle can be maintained.

[0095] Multiple calibrations may also be performed, and the successively obtained preset holding angles may be compared. When the differences among the preset holding angles for a consecutive preset number of times are less than or equal to the preset angle difference, the last obtained preset holding angle may be updated as the final preset holding angle.

[0096] Further, in a fourth embodiment of the handwriting trajectory detection method of the present invention proposed based on the first embodiment of the present invention, step S50 includes the following steps:

[0097] Step S51, determining the writing point corresponding to the writing pressure data on the stylus, and the reference point corresponding to the reference direction on the stylus;

[0098] Step S52, determining a target rotation angle between the writing point and the reference point;

[0099] Step S53, determining the writing direction corresponding to the writing point according to the reference direction and the target rotation angle;

[0100] Step S54, determining a handwriting trajectory corresponding to the writing pressure data according to the writing direction.

[0101] The writing point is the writing detection point corresponding to the writing direction; the reference point is the writing detection point corresponding to the reference direction; Figure 2For example, A1 to B1 is the horizontal writing direction, that is, the reference direction; if the writing direction of the stylus is vertically upward, the corresponding writing point is D1, and if the writing direction of the stylus is horizontally to the right, the corresponding writing point is B1; since the reference direction has been determined to be A1 to B1, the reference point is B1; when the writing point is D1, the straight line D1-C1 where the writing point D1 is located is reached after rotating 90° counterclockwise from B1-A1, wherein the target writing angle is the angle between B1-A1 and D1-C1; since the reference direction has been determined to be A1 to B1, combined with the target rotation angle of 90°, it can be determined that the writing direction is the direction after A1 to B1 is rotated 90° counterclockwise, that is, C1 to D1, corresponding to the vertically upward writing direction.

[0102] Based on the changes in the writing point during the writing process, the writing direction at each moment can be determined, and then the handwriting trajectory can be obtained.

[0103] It should be noted that, during the writing process, since the reference direction does not change with the stylus posture, while the writing detection point on the stylus changes position with the stylus posture, when the stylus posture changes, such as when it is rotated, the reference point corresponding to the reference direction may change; if the horizontal writing direction is used as the reference direction, the corresponding reference points at the two moments are B1 and F1 respectively. If the writing direction is vertically upward at both moments, the writing point generated at the corresponding moment of the B1 reference point is D1, and the writing point generated at the corresponding moment of the F1 reference point is H1; therefore, during the writing process, the reference direction on the stylus needs to be updated in real time, and the writing direction is determined based on the real-time updated reference direction.

[0104] Furthermore, the step S54 comprises the steps of:

[0105] Step S541, obtaining motion detection data of the stylus;

[0106] Step S542, determining the tip position of the stylus pen according to the motion detection data;

[0107] Step S543, determining a handwriting trajectory corresponding to the writing pressure data according to the pen tip position and the writing direction.

[0108] The motion detection data is used to indicate the posture and changes of the stylus; the motion detection data can be detected by a motion sensor, such as a 9-axis sensor.

[0109] The writing direction can indicate the direction of the strokes in the handwriting trajectory, but the determination of the handwriting trajectory also requires the clarification of the length of each stroke; therefore, in this embodiment, the position of the stylus tip is detected through motion detection data, so that the movement of the pen tip can be clarified, and the data of the stroke length can be supplemented, which is then combined with the writing direction to obtain the handwriting trajectory.

[0110] The specific method of determining the pen tip position based on motion detection data can be set based on actual needs.

[0111] Furthermore, the step S543 includes the steps of:

[0112] Step S5431, determining the pressure-sensing sub-data obtained by continuous writing in the writing pressure-sensing data;

[0113] Step S5432, for each of the pressure-sensitive sub-data, taking the pen tip position corresponding to the initial moment of the pressure-sensitive sub-data as the starting point position, and generating a writing sub-track in the writing direction, wherein the track length in the writing direction is determined based on the pen tip position corresponding to the writing direction;

[0114] Step S5433, combining the writing sub-trajectories in the pressure-sensitive sub-data to obtain the handwriting trajectory.

[0115] The pressure-sensitive sub-data is the pressure-sensitive data generated by the stylus from the continuous putting down of the pen to the lifting of the pen; it can be understood that the writing sub-trajectory corresponding to the pressure-sensitive sub-data is continuous.

[0116] The pen tip position corresponding to the initial moment of the pressure-sensitive sub-data is the position where the pen is put down, and the writing sub-trajectory starts from this. Therefore, this position is used as the starting position of the writing sub-trajectory. After the pen is put down, starting from the starting position, as the writing process proceeds, the pen tip moves. During the movement, the changes in the writing direction and the pen tip position will be measured. The writing direction is the same as the extension direction of the handwriting trajectory at the same time, and the change in the pen tip position corresponds to the extended length of the handwriting trajectory. Therefore, the writing sub-trajectory can be determined by the writing direction and the pen tip position.

[0117] Specifically, Figure 4 For example, A2 to B2 is the reference direction, and the reference direction is the horizontal writing direction;

[0118] When the pressure-sensitive sub-data detects that the writing direction is always B2 to A2, it means that the corresponding writing sub-trace is written from right to left. At this time, by determining the moving distance of the pen tip position, a horizontal line written from right to left can be determined.

[0119] When the pressure-sensitive sub-data detects that the writing direction is always D2 to C2, it means that the corresponding writing sub-trace is written from top to bottom. At this time, by determining the moving distance of the pen tip position, a vertical line written from top to bottom can be determined.

[0120] When the pressure-sensitive sub-data detects that the writing direction is first A2 to B2 and then turns to D2 to C2, it means that the corresponding writing sub-trace is first written from left to right and then from top to bottom. At this time, the moving distance corresponding to the pen tip position in different writing directions can be determined to determine the The writing sub-track of .

[0121] It should be noted that the writing direction without change in the pen tip position can be discarded. For example, the writing direction mentioned above is first A2 to B2, and then changes to D2 to C2. In the process of changing from A2 to B2 to D2 to C2, the writing direction may be detected as H2 to G2, but at this time no trajectory is generated in the H2 to G2 direction, and the corresponding pen tip position has not changed. Therefore, the writing direction of H2 to G2 can be discarded, and the writing sub-trajectory can be determined only by A2 to B2 and D2 to C2.

[0122] For the case where multiple writing sub-trajectories are combined into a writing trajectory, such as two writing sub-trajectories, the two writing sub-trajectories are written one after another. At the same time, the two writing sub-trajectories are not generated continuously, that is, there is at least one pen-lifting-pen-dropping process between the two writing sub-trajectories:

[0123] The first writing sub-trace determines a horizontal line written from right to left;

[0124] If the second writing sub-track generates a vertical line written from top to bottom with the center of the first writing sub-track as the starting point, a writing track of "T" can be obtained;

[0125] If the second writing sub-track generates a vertical line written from top to bottom with the right end point of the first writing sub-track as the starting point, we can get The writing track of The above-mentioned writing sub-track Although the structure is the same, the writing of sub-tracks is written in one stroke, and here It is composed of two strokes.

[0126] The above is only an example. In actual applications, a single writing sub-track may include more writing directions, and a writing track may be composed of more writing sub-tracks.

[0127] Further, in a fifth embodiment of the handwriting trajectory detection method of the present invention proposed based on the first embodiment of the present invention, the step S50 includes the following steps:

[0128] Step S80, determining the target recognition type corresponding to the writing pressure sensing data;

[0129] Step S90, determining a target recognition method corresponding to the target recognition type;

[0130] Step S100, identifying the handwriting trajectory by the target recognition method to obtain writing information corresponding to the handwriting trajectory.

[0131] The target recognition type is used to indicate the type of content presented by the handwriting trajectory; for example, the target recognition type can be text or picture; it is understandable that there are differences in the recognition methods for text and picture, such as recognition through different recognition models; therefore, in order to improve the recognition accuracy of handwriting trajectories, different target recognition methods are set for handwriting trajectories of different target recognition types, so as to accurately obtain the corresponding writing information; the target recognition type can be input by the user, such as switching between image and text recognition by pressing the buttons on the stylus; another example is switching through voice or other interactive methods set on the stylus; another example is switching through the relevant settings on the extended reality device.

[0132] The writing information indicates the specific content of the handwriting track; for example, the writing information corresponding to a text-type handwriting track may be in an electronic format of the text; the writing information corresponding to a picture-type handwriting track may be in an image format of the picture or the meaning of the picture.

[0133] After the written information is determined, the written information can be further applied, such as searching, translating, solving problems, etc.

[0134] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0136] The present application also provides a handwriting trajectory detection device for implementing the above-mentioned handwriting trajectory detection method, the handwriting trajectory detection device comprises a stylus and a data processing device; the stylus comprises a gripping sensor module and a writing pressure sensing module, the data processing device establishes a communication connection with the gripping sensor module and the writing pressure sensing module; wherein:

[0137] The holding sensing module is used to detect holding sensing data of the stylus;

[0138] The writing pressure sensing module is used to detect the writing pressure sensing data of the stylus;

[0139] The data processing device comprises:

[0140] A first acquisition module, used to acquire the grip sensing data of the stylus;

[0141] A first determining module, configured to determine a holding direction of the stylus according to the holding sensing data;

[0142] A second acquisition module, configured to acquire a preset holding angle corresponding to a current user of the stylus pen, and determine a reference direction on the stylus pen according to the preset holding angle and the holding direction;

[0143] A third acquisition module, used to acquire writing pressure data of the stylus;

[0144] The second determination module is used to determine the handwriting trajectory corresponding to the writing pressure data based on the reference direction.

[0145] The handwriting trajectory detection device determines the user's holding habits by setting a preset holding angle, determines a reference direction based on the holding direction and the preset holding angle, and further determines the handwriting trajectory based on the reference direction and writing pressure data, so that the handwriting trajectory can be obtained based on the user's actual writing habits. The state of the obtained handwriting trajectory is based on the determined reference direction, which can achieve accurate detection of the handwriting trajectory.

[0146] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the first determination module in this embodiment can be used to execute step S20 in the embodiment of the present application, the second acquisition module in this embodiment can be used to execute step S30 in the embodiment of the present application, the third acquisition module in this embodiment can be used to execute step S40 in the embodiment of the present application, and the second determination module in this embodiment can be used to execute step S50 in the embodiment of the present application.

[0147] Furthermore, the first determining module includes:

[0148] A first determining submodule, configured to determine a gripping position of the stylus according to the grip sensing data, wherein the number of gripping positions is 2;

[0149] The first execution submodule is configured to use the straight line where the gripped position is located as the gripping direction of the stylus.

[0150] Furthermore, the data processing device further includes:

[0151] a fourth acquisition module, configured to acquire a calibration holding direction and a calibration writing direction generated when a current user of the stylus performs writing along a reference trajectory, wherein the reference trajectory includes a trajectory along the reference direction;

[0152] The first execution module is used to obtain the preset holding angle according to the calibrated holding direction, the calibrated writing direction and the reference direction.

[0153] Furthermore, the second determining module includes:

[0154] A second determination submodule, used to determine a writing point on the stylus corresponding to the writing pressure data, and a reference point on the stylus corresponding to the reference direction;

[0155] A third determination submodule is used to determine a target rotation angle between the writing point and the reference point;

[0156] A fourth determination submodule, configured to determine a writing direction corresponding to the writing point according to the reference direction and the target rotation angle;

[0157] The fifth determination submodule is used to determine the handwriting trajectory corresponding to the writing pressure data according to the writing direction.

[0158] Further, the fifth determining submodule includes:

[0159] A first acquisition unit, used to acquire motion detection data of the stylus;

[0160] A first determining unit, configured to determine a tip position of the stylus pen according to the motion detection data;

[0161] A second determining unit is used to determine a handwriting trajectory corresponding to the writing pressure data according to the pen tip position and the writing direction.

[0162] Further, the second determining unit includes:

[0163] A first determining subunit, configured to determine pressure-sensing sub-data obtained by continuous writing in the writing pressure-sensing data;

[0164] A first generating sub-unit is used for, for each of the pressure-sensitive sub-data, taking the pen tip position corresponding to the initial moment of the pressure-sensitive sub-data as the starting position, and generating a writing sub-trace in the writing direction, wherein the length of the trajectory in the writing direction is determined based on the pen tip position corresponding to the writing direction;

[0165] The first combining subunit is used to combine the writing sub-trajectories in the pressure-sensing sub-data to obtain the handwriting trajectory.

[0166] Furthermore, the data processing device comprises:

[0167] A third determination module is used to determine the target recognition type corresponding to the writing pressure data;

[0168] A fourth determination module, used to determine a target recognition method corresponding to the target recognition type;

[0169] The first recognition module is used to recognize the handwriting track by using the target recognition method to obtain writing information corresponding to the handwriting track.

[0170] Reference Figure 6 In terms of hardware structure, the extended reality device may include components such as a communication module 10, a memory 20, and a processor 30. In the extended reality device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0171] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other extended reality devices, servers or Internet of Things devices, such as televisions, etc.

[0172] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as obtaining the grip sensing data of the stylus), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0173] The processor 30 is the control center of the extended reality device. It uses various interfaces and lines to connect various parts of the entire extended reality device. By running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, it performs various functions of the extended reality device and processes data, thereby monitoring the extended reality device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0174] although Figure 6 Although not shown, the above-mentioned extended reality device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 6 The extended reality device structure shown in the figure does not constitute a limitation of the extended reality device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0175] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 6 The memory 20 in the extended reality device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a TV, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0176] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0177] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0178] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A handwriting trajectory detection method, characterized in that: The handwriting trajectory detection method comprises: Get the holding sensing data of the stylus; Determining a holding direction of the stylus according to the holding sensing data; Acquire a preset holding angle corresponding to a current user of the stylus, and determine a reference direction on the stylus according to the preset holding angle and the holding direction; Acquiring writing pressure data of the stylus; A handwriting trajectory corresponding to the writing pressure sensing data is determined based on the reference direction.

2. The handwriting trajectory detection method according to claim 1, characterized in that: The determining the holding direction of the stylus according to the holding sensing data comprises: Determining a gripping position of the stylus according to the gripping sensing data, wherein the number of the gripping positions is 2; The straight line where the gripped position is located is used as the gripping direction of the stylus.

3. The handwriting trajectory detection method according to claim 1, characterized in that: The step of obtaining the preset holding angle corresponding to the current user of the stylus pen includes: Acquire a calibration holding direction and a calibration writing direction generated when a current user of the stylus performs writing along a reference trajectory, wherein the reference trajectory includes a trajectory along the reference direction; The preset holding angle is obtained according to the calibrated holding direction, the calibrated writing direction and the reference direction.

4. The handwriting trajectory detection method according to claim 1, characterized in that: The determining the handwriting trajectory corresponding to the writing pressure sensing data based on the reference direction comprises: Determine a writing point on the stylus corresponding to the writing pressure data, and a reference point on the stylus corresponding to the reference direction; Determining a target rotation angle between the writing point and the reference point; Determining the writing direction corresponding to the writing point according to the reference direction and the target rotation angle; A handwriting trajectory corresponding to the writing pressure data is determined according to the writing direction.

5. The handwriting trajectory detection method according to claim 4, characterized in that: Determining the handwriting trajectory corresponding to the writing pressure data according to the writing direction includes: Acquiring motion detection data of the stylus; Determining the position of the tip of the stylus pen according to the motion detection data; A handwriting trajectory corresponding to the writing pressure data is determined according to the pen tip position and the writing direction.

6. The handwriting trajectory detection method according to claim 5, characterized in that: The step of determining the handwriting trajectory corresponding to the writing pressure sensing data according to the pen tip position and the writing direction comprises: Determine pressure-sensing sub-data obtained by continuous writing in the writing pressure-sensing data; For each of the pressure-sensitive sub-data, the pen tip position corresponding to the initial moment of the pressure-sensitive sub-data is used as the starting point position, and a writing sub-track is generated in the writing direction, wherein the track length in the writing direction is determined based on the pen tip position corresponding to the writing direction; The writing sub-trajectories in the pressure-sensitive sub-data are combined to obtain the handwriting trajectory.

7. The handwriting trajectory detection method according to claim 1, characterized in that: The step of determining the handwriting trajectory corresponding to the writing pressure sensing data based on the reference direction includes: Determining a target recognition type corresponding to the writing pressure sensing data; Determine a target recognition method corresponding to the target recognition type; The handwriting track is identified by the target identification method to obtain writing information corresponding to the handwriting track.

8. A handwriting trajectory detection device, characterized in that: The handwriting trajectory detection device includes a stylus and a data processing device; the stylus includes a gripping sensor module and a writing pressure sensing module, and the data processing device establishes a communication connection with the gripping sensor module and the writing pressure sensing module; wherein: The holding sensing module is used to detect holding sensing data of the stylus; The writing pressure sensing module is used to detect the writing pressure sensing data of the stylus; The data processing device comprises: A first acquisition module, used to acquire the grip sensing data of the stylus; A first determining module, configured to determine a holding direction of the stylus according to the holding sensing data; A second acquisition module, configured to acquire a preset holding angle corresponding to a current user of the stylus pen, and determine a reference direction on the stylus pen according to the preset holding angle and the holding direction; A third acquisition module, used to acquire writing pressure data of the stylus; The second determination module is used to determine the handwriting trajectory corresponding to the writing pressure data based on the reference direction.

9. An extended reality device, characterized in that: The extended reality device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the handwriting trajectory detection method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the handwriting trajectory detection method according to any one of claims 1 to 7 are implemented.

Images