Method and device for obtaining handwriting information

Abstract

The present application discloses a method and device for acquiring handwriting information, the method comprising: acquiring multiple groups of collected data collected from the pen at multiple collection moments during the writing process of the pen, wherein each group of collected data includes the motion-related data of the pen and the initial image obtained by photographing the pen tip; using the motion-related data, determining the position representation information of the pen tip at each collection moment; based on the position representation information at each collection moment, transforming the initial image at each collection moment to the same space to obtain the transformed image at each collection moment; wherein the transformed image at each collection moment is used to identify the writing handwriting of the pen. Through the above method, the present application can be written on ordinary paper without relying on a matching handwriting book, thereby realizing complete handwriting recognition and recording.

Description

Technical Field

[0001] The present application relates to the technical field of mobile equipment, and in particular to a method and device for acquiring handwriting information. Background Art

[0002] With the development of mobile office and online learning, more and more people are recording notes on electronic devices. The conventional practice is to write directly on mobile devices, such as mobile phones and tablets, or use special pens with dedicated notebooks to record and save notes.

[0003] For example, the existing method of using a special pen and a dedicated notebook to record and save notes mostly requires printing a specific OID (Optical Identify) code on the matching notebook paper. The OID code contains the XY coordinates of the location relative to the paper. When using the pen to write, the camera at the end of the pen determines the current position of the handwriting by identifying the OID code. By storing these continuous XY coordinates, the continuous handwriting can be restored on the mobile device, thereby realizing the recognition and recording of the handwriting.

[0004] However, the above method can only be used on a notebook printed with a matching dot matrix code, and has great limitations in use. The notebook cannot be replaced at will, and when the notebook is used up, a dedicated notebook needs to be purchased. Summary of the invention

[0005] The main technical problem solved by the present application is to provide a method and device for obtaining handwriting information, which can realize the recognition of handwriting without the need for specific writing paper.

[0006] In order to solve the above-mentioned technical problems, the first aspect of the present application provides a method for acquiring handwriting information, which method includes: acquiring multiple groups of collected data obtained from the pen at multiple collection moments during the pen writing process, wherein each group of collected data includes motion-related data of the pen and an initial image obtained by photographing the pen tip; using the motion-related data, determining the position representation information of the pen tip at each collection moment; based on the position representation information at each collection moment, transforming the initial image at each collection moment to the same space to obtain a transformed image at each collection moment; wherein the transformed image at each collection moment is used to identify the writing handwriting of the pen.

[0007] In order to solve the above technical problems, the second aspect of the present application provides a device for acquiring handwriting information. The device includes a processor, an image acquisition component, a sensor component and other components. The processor uses motion-related data to determine the position representation information of the pen tip at each acquisition time, and based on the position representation information at each acquisition time, transforms the initial image at each acquisition time to the same space to obtain the transformed image at each acquisition time, and identifies the transformed image at each acquisition time to obtain the writing handwriting of the pen; the image acquisition component is used to perform image acquisition on the pen tip to obtain the initial image; and the sensor component is used to acquire the motion-related data of the pen.

[0008] The beneficial effects of the present application are as follows: Different from the prior art, the present application first obtains multiple groups of collected data collected from the pen at multiple collection moments during the writing process of the pen, wherein each group of collected data includes the motion-related data of the pen and the initial image obtained by photographing the pen tip; then the motion-related data is used to determine the position representation information of the pen tip at each collection moment; based on the position representation information at each collection moment, the initial image at each collection moment is transformed to the same space to obtain the changed image at each collection moment, wherein the transformed image at each collection moment is used to identify the writing notes of the pen. The above method obtains the position representation information of the pen tip at each collection moment, and based on the position representation information at each collection moment, the initial image at each collection moment is transformed to the same space, thereby the handwriting of the pen can be obtained by using the image recognition transformed to the same space, thereby achieving the identifiability of the handwriting, and the above method does not need to rely on specific writing paper, and can be applied to more writing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of an implementation method of a handwriting information acquisition method of the present application;

[0010] Figure 2 It is a flowchart of another implementation method of a method for obtaining handwriting information of the present application;

[0011] Figure 3 It is a flowchart of an implementation method of obtaining the transformed image at each acquisition moment of the present application;

[0012] Figure 4 It is a flowchart of another implementation method of a method for obtaining handwriting information of the present application;

[0013] Figure 5 It is a flowchart of an implementation method of obtaining multiple sets of collected data in the present application;

[0014] Figure 6 It is a schematic diagram of the framework of a handwriting information acquisition device of the present application;

[0015] Figure 7It is a structural schematic diagram of an implementation method of a handwriting information acquisition device of the present application. DETAILED DESCRIPTION

[0016] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] It can be understood that, in addition to explicitly describing the execution subject of the following embodiment methods of the present application, other embodiment methods can be executed by the following writing pen or other terminal devices.

[0018] See also Figure 1 , Figure 1 This is a flow chart of an implementation method of a handwriting information acquisition method of the present application. The method comprises:

[0019] S110: Acquire multiple groups of collected data obtained from collecting the pen at multiple collection moments during the pen writing process, wherein each group of collected data includes movement-related data of the pen and an initial image obtained by photographing the pen tip.

[0020] During the writing process, the image captured by the pen tip is collected in real time or at predetermined intervals, and the motion-related data of the pen at the time of collection is obtained. The motion-related data of the pen can be understood as data that can represent the motion state of the pen, such as but not limited to one or more of the acceleration data, angular velocity data and magnetic field data of the pen.

[0021] Specifically, the pen is provided with an image acquisition component and a sensor component for acquiring motion-related data. The image acquisition component is used to acquire images of the pen tip at multiple acquisition moments to obtain initial images corresponding to the multiple acquisition moments, and the sensor component is used to acquire images at multiple acquisition moments to obtain motion-related data corresponding to the multiple acquisition moments. In a specific application scenario, the data of the motion state of the pen includes acceleration data, angular velocity data, and magnetic field data of the pen. Correspondingly, the sensor component includes an acceleration sensor, a gyroscope, and a magnetic sensor. Specifically, the acceleration sensor is used to acquire acceleration data, the gyroscope is used to acquire angular velocity data, and the magnetic field data is acquired by the magnetic sensor. It can be understood that, considering that the motion-related data of the pen is used to determine the corresponding position information of the pen, a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetic sensor can be used to respectively acquire acceleration data, angular velocity data, and magnetic field data in the three coordinate axis directions of a predetermined coordinate system (such as a world coordinate system).

[0022] After the collected data is collected, the method of the present application is executed by a pen or other terminal device to realize handwriting recognition.

[0023] S120: Determine the position representation information of the pen tip at each acquisition moment by using the motion-related data.

[0024] By combining the acceleration data, angular velocity data, and magnetic field data collected by the pen at each collection moment, the position representation information of the pen tip at each collection moment can be obtained. Among them, the position representation information can be information indicating the position of the pen tip, such as an attitude angle and / or position. In the following embodiments, the position representation information is taken as an example of the attitude angle of the pen tip in a preset coordinate system, but it should be understood that the position representation information can also be the spatial coordinates of the pen tip in a preset coordinate system, and the preset coordinate system can be any pre-set coordinate system, such as a coordinate system established with the position of the pen tip at the first collection moment of this writing process as the origin, and the coordinate system can be a three-dimensional or two-dimensional coordinate system.

[0025] S130: Based on the position representation information of each acquisition moment, the initial image at each acquisition moment is transformed into the same space to obtain a transformed image at each acquisition moment; wherein the transformed image at each acquisition moment is used to identify the writing handwriting of the pen.

[0026] Among them, the position representation information of each acquisition moment indicates the position of the initial image collected by the pen at the corresponding acquisition moment. Therefore, based on the position representation information of each acquisition moment, the initial image at each acquisition moment can be transformed to the same space. For example, the initial images at non-first acquisition moments in the writing process are transformed to the space where the initial image at the first acquisition moment is located. In one embodiment, the above-mentioned space transformation is realized by rotation. For example, the attitude angle of each acquisition moment in the coordinate system with the pen tip at the first acquisition as the origin is obtained, that is, the attitude angle at the first acquisition moment is zero. Based on the attitude angle of each acquisition moment, the initial images at non-first acquisition moments in the writing process are rotated to the space of the initial image at the first acquisition moment, that is, the initial image at the first acquisition moment does not need to be rotated, and the initial images at non-first acquisition moments are rotated according to the attitude angle.

[0027] The transformed images at each acquisition moment obtained by spatial transformation are in the same space, so the relevant handwriting can be identified from the transformed images at each acquisition moment, and the handwriting information of the pen tip during the writing process can be obtained by combining the handwriting in the transformed images at each acquisition moment. In one embodiment, the transformed images at each acquisition moment can be spliced ​​to obtain a spliced ​​image, which contains the handwriting information of the pen tip during the writing process. By recognizing the spliced ​​image, the handwriting of the pen tip during the writing process contained in it can be obtained. Among them, the recognition method can be but is not limited to using OCR (Optical Character Recognition) recognition, or using a trained handwriting recognition model for recognition. The above-mentioned handwriting can be but is not limited to handwriting such as text, drawings or paintings.

[0028] In one embodiment, since the position representation information at each acquisition moment can express the position of the pen tip at each acquisition moment, the position representation information at each acquisition moment can be used to directly calculate the reference handwriting of the pen tip during the writing process, and the handwriting obtained by the above-mentioned transformation image recognition is adjusted based on the reference handwriting or the accuracy of the recognized handwriting is verified. For example, if there is a large deviation between the recognized handwriting and the reference handwriting, the recognized handwriting is considered to be inaccurate.

[0029] Using the above method, during the writing process, the pen uses an image acquisition component to acquire images of the pen tip at each acquisition time, and uses a sensor component to acquire acceleration data, angular velocity data, and magnetic field data of the pen at each acquisition time. Based on the data acquired at each acquisition time, the position information of the pen tip at each acquisition time can be obtained. Based on the position information of the pen tip at each acquisition time, the images at each acquisition time are transformed to the same space for splicing, and the writing handwriting can be obtained by identifying the spliced ​​images. There is no need to rely on a specific notebook, and the writing handwriting can be obtained by writing on any paper.

[0030] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another embodiment of a method for obtaining handwriting information of the present application. In this embodiment, the method for obtaining handwriting information includes:

[0031] S210: Acquire multiple groups of collected data obtained by collecting the pen at multiple collection moments during the writing process of the pen, wherein each group of collected data includes movement-related data of the pen and an initial image obtained by photographing the pen tip.

[0032] S220: Determine position characterization information of the pen tip at each acquisition moment using the motion-related data, wherein the motion-related data includes at least two types of sensing data among acceleration data, angular velocity data, and magnetic field data.

[0033] In this embodiment, the acceleration data may include the acceleration value of the pen in the XYZ direction in the three-dimensional space at the corresponding collection time, that is, the acceleration value of the pen in the X direction, the acceleration value of the pen in the Y direction, and the acceleration value of the pen in the Z direction at each collection time; the angular velocity data may include the angular velocity value of the pen in the XYZ direction in the three-dimensional space at the corresponding collection time, that is, the angular velocity value of the pen in the X direction, the angular velocity value of the pen in the Y direction, and the angular velocity value of the pen in the Z direction at each collection time; the magnetic field data may include the magnetic field strength of the pen in the XYZ direction in the three-dimensional space at the corresponding collection time, that is, the magnetic field strength of the pen in the X direction, the magnetic field strength in the Y direction, and the magnetic field strength in the Z direction at each collection time.

[0034] S230: Fusing at least two types of sensor data to obtain position representation information of the pen tip at each acquisition moment.

[0035] The motion-related data of the pen may include at least two types of data among acceleration data, angular velocity data and magnetic field data of the pen at multiple acquisition moments.

[0036] In one embodiment, the position representation information of the pen at the collection time is obtained by using the acceleration data, angular velocity data and magnetic field data of the pen at the collection time, and the position representation information can be represented by the attitude angle. Specifically, a three-dimensional coordinate system is established with the pen tip as the origin, and each collection time is respectively used as the current collection time. The acceleration value of the pen in the X direction, Y direction and Z direction at the current collection time is obtained by using the acceleration sensor. The acceleration value represents the speed change of the pen at the current collection time relative to the previous collection time of the pen. The previous collection time of the pen can be defined as the zero time, and the current collection time of the pen can be defined as the t time. The acceleration values ​​of the pen in the X direction, Y direction and Z direction in a time period such as 0-t time are integrated to obtain the speed function of the pen in the X direction, Y direction and Z direction within the time period of 0-t. The speed value of the pen in the X direction, Y direction and Z direction at the current collection time can be obtained by substituting the t time into the speed function.

[0037] The angular velocity sensor is used to obtain the angular velocity values ​​of the X, Y and Z directions of the pen's position at the time of collection. The previous collection time of the pen can be defined as time zero, and the current collection time of the pen can be defined as time t. The angular velocity values ​​of the X, Y and Z directions of the pen's position within the time 0-t are integrated to obtain the function of the rotation angle of the X, Y and Z directions of the pen's position within the time 0-t. The rotation angle value of the pen's current collection time relative to the previous collection time can be obtained by substituting time t into the rotation angle function. It should be noted that in this embodiment, in order to ensure that the collected data is more accurate, the time interval between the previous collection time of the pen and the current collection time of the pen is extremely short, which can be 2 milliseconds or even shorter.

[0038] The geomagnetic sensor is used to obtain the magnetic field strength in the X, Y and Z directions of the pen at each acquisition moment. The current direction of the pen tip can be determined based on the magnetic field strength. Combined with the direction of the pen tip at the previous acquisition moment, the movement direction of the pen tip at the current acquisition moment relative to the previous acquisition moment can be obtained. Combined with the speed value, rotation angle value and movement direction of the pen tip in the X, Y and Z directions at the current acquisition moment, the current position characterization information of the pen tip can be obtained. Specifically, the rotation angle value can be optimized using the speed value and movement direction, and the above-mentioned attitude angle can be obtained from the optimized rotation angle value.

[0039] In another embodiment, a common posture fusion algorithm such as the Mahony algorithm is used to obtain the position representation information of the pen tip at each acquisition time, and the acceleration data and angular velocity data are fused to calculate the quaternion, or the acceleration data, angular velocity data and magnetic field data are fused to calculate the quaternion, and the posture angle is calculated by the quaternion. In this embodiment, the posture angle is used to represent the position representation information of the pen tip at each acquisition time. It can be understood that in other embodiments, other algorithms can also be used to obtain the position representation information of the pen tip at each acquisition time, which is not limited here.

[0040] S240: Based on the position representation information of each acquisition moment, the initial image at each acquisition moment is transformed into the same space to obtain a transformed image at each acquisition moment; wherein the transformed image at each acquisition moment is used to identify the writing handwriting of the pen.

[0041] After obtaining the position representation information of the pen tip at each acquisition moment, the initial image at each acquisition moment is transformed to the same space based on the position representation information. Specifically, all images that are not acquired for the first time can be transformed to the space where the image acquired for the first time is located. The transformation method can be rotation, and the transformed images are spliced. The spliced ​​images are identified to obtain the pen's writing handwriting.

[0042] See also Figure 3 , Figure 3 This is a flow chart of an implementation method of obtaining the transformed image at each acquisition moment in the present application. The acquisition method includes:

[0043] S310: Acquire multiple groups of collected data obtained from collecting the pen at multiple collection moments during the writing process of the pen, wherein each group of collected data includes movement-related data of the pen and an initial image obtained by photographing the pen tip.

[0044] S320: Determine the position representation information of the pen tip at each acquisition moment by using the motion-related data, wherein the position representation information is the attitude angle of the pen tip in a preset coordinate system at each acquisition moment.

[0045] In this embodiment, the preset coordinate system refers to a three-dimensional coordinate system established with the position of the pen tip at the first acquisition moment as the origin; the attitude angle is used to represent the spatial change of the pen coordinate system relative to the preset coordinate system. Among them, the pen coordinate system refers to a three-dimensional coordinate system established with the pen tip as the origin at each acquisition moment.

[0046] The specific implementation of steps S310 and S320 can refer to the description of the corresponding positions in the above embodiments, which will not be repeated here.

[0047] S330: For each acquisition moment, the initial image at the acquisition moment is rotated using the posture angle corresponding to the acquisition moment to obtain a transformed image at each acquisition moment based on a preset coordinate system.

[0048] Obtaining the posture angle of the pen at each acquisition moment is equivalent to obtaining the position of the pen tip at each acquisition moment and the rotation angle value of the pen tip at each acquisition moment relative to the first acquisition moment. Based on the rotation angle value and the position of the pen tip, the image at each acquisition moment is rotated and moved to the preset coordinate system, and the transformed image at each acquisition moment based on the preset coordinate system can be obtained.

[0049] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of another embodiment of a method for obtaining handwriting information of the present application. In this embodiment, the method for obtaining handwriting information may further include:

[0050] After the transformed images at each acquisition time are obtained in the above step S130, S240 or S330, the acquisition method may further include:

[0051] S440: Crop the transformed image, wherein adjacent sides of the transformed images at adjacent acquisition moments after cropping are parallel.

[0052] Since the angle may change during the movement of the pen, the angle of the camera image will also change. Therefore, after the initial image at each acquisition moment is transformed to the same space, there may be an angle relationship between the transformed image and the previous transformed image, that is, the adjacent edges of adjacent transformed images are not parallel, and the adjacent edges cannot be docked to achieve the splicing of the adjacent transformed images. Therefore, the transformed image needs to be cropped so that the edges of the cropped transformed image are perpendicular or parallel to the coordinate axis of the preset coordinate system, that is, the adjacent edges of the cropped adjacent transformed images are parallel, wherein the adjacent edges are the edges of the transformed image used for splicing with the adjacent transformed image. In one embodiment, the edges used for splicing of the transformed image at the first acquisition moment are used as a reference, so that the edges used for splicing of the cropped transformed image at the non-first acquisition moment are parallel to the edges used for splicing of the transformed image at the first acquisition moment. In the case where the edges used for splicing of the transformed image at the first acquisition moment are perpendicular to the specified coordinate axis of the preset coordinate system, all the transformed images are cropped so that their edges used for splicing are perpendicular to the above-mentioned specified coordinate axis.

[0053] S450: stitching the cropped transformed images at each acquisition time to obtain a stitched image.

[0054] The transformed images at each acquisition moment are stitched together. If the edge of the previous transformed image used for stitching is parallel to the edge of the next transformed image used for stitching in the preset coordinate system, they can be stitched together directly. If the edge of the previous transformed image used for stitching is angularly related to the edge of the next transformed image used for stitching in the preset coordinate system, the next transformed image must be cropped first to ensure that the edge of the cropped image used for stitching is parallel in size to the edge of the previous transformed image used for stitching before stitching can be performed.

[0055] S460: Recognize the spliced ​​image to obtain the handwriting in the spliced ​​image, or send the spliced ​​image to a handwriting recognition device.

[0056] The spliced ​​image can be recognized by the execution device such as a pen or other terminal, or the spliced ​​image can be sent to a handwriting recognition device other than the execution device such as a server, etc., and recognized by the handwriting recognition device.

[0057] In this embodiment, based on the position representation information of each acquisition moment, the initial image at each acquisition moment is transformed to the same space to obtain the transformed image at each acquisition moment, and the cropping and splicing of the transformed image at each acquisition moment are all performed by the pen tip. In other embodiments, the position representation information of each acquisition moment can also be sent to the terminal device, and the terminal device transforms the initial image at each acquisition moment to the same space to obtain the transformed image at each acquisition moment, and the transformed image at each acquisition moment is cropped and spliced.

[0058] See also Figure 5 , Figure 5 This is a flow chart of an implementation method of obtaining multiple sets of collected data in this application. This method embodiment is executed by a pen, and specifically includes:

[0059] S510: After the pen is powered, if it is detected that the pressure detection component of the pen receives pressure from the pen tip, it is determined to enter the writing mode.

[0060] Among them, the press detection component includes a press detection switch, and the press detection switch can be a sensor or a switch. In one embodiment, the press detection switch is a sensor, and a threshold is set for the sensor according to actual needs, and the threshold is related to the force of pressing the pen tip. When pressing the pen tip, it can be determined whether the pen needs to enter the writing mode based on whether the size of the pressing force reaches the threshold. If the size of the pressing force reaches the set threshold, the pen enters the writing mode. In another embodiment, the press detection switch is a switch, and when the pen tip is pressed, the switch is turned on and the pen enters the writing mode.

[0061] S520: After the pen enters the writing mode, the image acquisition component of the pen is used to acquire images of the pen tip at multiple acquisition moments to obtain initial images corresponding to the multiple acquisition moments.

[0062] Among them, the image acquisition component of the pen includes a camera, which takes pictures of the image at the tip of the pen. The shooting range of the camera is set according to actual needs and is not limited here.

[0063] S530: After the pen enters the writing mode, the sensing component of the pen is used to collect data at multiple collection moments to obtain motion-related data corresponding to the multiple collection moments.

[0064] The sensing components of the pen include an accelerometer, a gyroscope and a magnetic sensor. The accelerometer is used to obtain the acceleration values ​​of the pen at multiple collection times; the gyroscope is used to obtain the angular velocity values ​​of the pen at multiple collection times; and the magnetic sensor is used to obtain the magnetic field strength of the pen at multiple collection times.

[0065] After the pen is powered and enters the writing mode, step S520 may be executed first to obtain initial images corresponding to multiple acquisition moments, or step S530 may be executed first to obtain motion-related data corresponding to multiple acquisition moments. The execution order of steps S520 and S530 is not limited here.

[0066] In this embodiment, the acquisition of multiple sets of collected data collected from the pen at multiple collection moments during the writing process of the pen is performed when the pen is in the writing mode, wherein the multiple sets of collected data include the initial images corresponding to the pen at multiple collection moments and the corresponding motion-related data. The motion-related data includes but is not limited to the acceleration data, angular velocity data and magnetic field data of the pen at each collection moment. After the pen collects the above-mentioned motion-related data and initial image, the above embodiment can be used to obtain a transformed image using the motion-related data and the initial image, thereby realizing recognition of the writing handwriting of the pen.

[0067] After the pen enters the writing mode, if it is detected that the press detection component does not receive a press from the pen tip within a preset time, the writing mode is exited. The preset time is set as needed. In one embodiment, the preset time can be one minute. If the press detection component of the pen does not receive information that the pen tip is pressed again within one minute, the writing mode is exited. At this time, the pen no longer collects new motion-related data and initial images, and further no longer executes the relevant steps of the above embodiment to achieve the acquisition of handwriting information.

[0068] See also Figure 6 , Figure 6 : is a schematic diagram of the framework of a handwriting information acquisition device of the present application. The handwriting information acquisition device 60 includes a processor 61 and a memory 62 coupled to each other, the memory 62 stores program instructions, and the processor 61 is used to execute the program instructions stored in the memory 62 to implement the steps in the above-mentioned handwriting information acquisition method embodiment. The processor 61 can also be called a CPU (Central Processing Unit). The processor 61 may be an integrated circuit chip with signal processing capabilities. The processor 61 can also be a general-purpose processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 61 can be implemented by multiple integrated circuit chips.

[0069] In the above embodiment, the processor is used to obtain multiple groups of collected data obtained by collecting the pen at multiple collection moments during the pen writing process, wherein each group of collected data includes motion-related data of the pen and an initial image obtained by photographing the pen tip; and using the motion-related data, determine the position representation information of the pen tip at each collection moment, based on the position representation information at each collection moment, transform the initial image at each collection moment to the same space to obtain the transformed image at each collection moment, and identify the transformed image at each collection moment to obtain the writing handwriting of the pen.

[0070] See also Figure 7 , Figure 7 It is a structural schematic diagram of an implementation method of a handwriting information acquisition device of the present application.

[0071] The handwriting information acquisition device 60 can be a pen or other terminal device. Specifically, in one embodiment, the handwriting information acquisition device is a pen 70, which includes a pen housing 71, a pen core 72, an image acquisition component 73 and a sensor component 74 arranged in the pen housing 71, wherein the image acquisition component 73 and the sensor component 74 are connected to the processor 61, the image acquisition component 73 is used to acquire an image of the pen tip to obtain an initial image, and the sensor component 74 is used to acquire the movement-related data of the pen.

[0072] In one embodiment, the image acquisition component 73 may be a camera, which is installed at the end of the pen. The end of the pen is a semi-cone, and the interior of the semi-cone is hollow. The camera is installed inside the semi-cone. The pen body will tilt during writing, and the camera will capture the image of the pen tip through the blank area of ​​the pen end. The sensor component 74 may be a detection chip, which includes at least two of an acceleration sensor 741, a magnetic sensor 742, and a gyroscope 743. The acceleration sensor 741 is used to detect the acceleration of the current pen movement, and may be a three-axis acceleration sensor; the magnetic sensor 742 is used to detect the magnetic field strength of the current pen, and may be a three-axis magnetic sensor; the gyroscope 743 is used to detect the angular velocity of the current pen movement, and may be a three-axis gyroscope.

[0073] Please continue reading Figure 7 The pen 70 further includes a pressure detection component 75, a wireless communication component 76, a battery 77, and / or a connection interface 78, an indicator light (not shown), a power switch (not shown), etc., which are arranged outside the pen housing 71. In a specific embodiment, one or more of these components can be arranged in the structure of the pen 70 as required.

[0074] In one embodiment, the pen 70 further includes a press detection component 75, a wireless communication component 76 and a battery 77 arranged in the pen housing 71. The press detection component 75 is arranged at the end of the pen core 72 away from the pen tip, and is used to detect whether the pen tip is pressed. The press detection component 75 includes a press detection switch. When the press detection component 75 detects that the pen tip is pressed and the pressing force reaches a preset value, the press detection switch will be turned on to put the pen 70 into writing mode; the wireless communication component 76 is arranged in the pen housing 71, and the wireless communication component 76 is used to communicate with an external device. Through the wireless communication component 76, the image captured by the image acquisition component 73 of the pen 70, and / or the sensor data collected by the sensor component 74 of the pen 70, and / or the spliced ​​image can be sent to the external device; the battery 77 is arranged in the pen housing 71, and is used to power the pen 70.

[0075] In another embodiment, the pen 70 further includes a connection interface 78, an indicator light (not shown in the figure), and a power switch (not shown in the figure) disposed outside the pen housing 71. The connection interface 78 is disposed outside the pen housing 71, connected to the processor 61, and used to connect an external connection line, and the battery disposed in the pen housing 71 can be charged through the connection interface 78; the power switch is disposed outside the pen housing 71, and the power switch can be a push switch or a key switch, which is not limited here. After the power switch is turned on, the battery 77 disposed in the pen housing 71 starts to supply power to the pen 70, and when the power switch is turned off, the battery 77 disposed in the pen housing 71 stops supplying power to the pen 70; the indicator light disposed outside the pen housing 71 is connected to the processor 61, and the processor 61 controls the indicator light to turn on or off. The indicator light can turn on automatically when the power switch of the pen 70 is turned on, indicating that the pen 70 has entered the working state. It can also turn on when the press detection component 75 of the pen 70 detects pressure, used to indicate that the pen 70 has entered the writing mode. In other embodiments, the indicator light can also indicate two working states, such as the indicator light is displayed in red when the power switch of the pen 70 is turned on, and the indicator light is displayed in blue when the press detection component 75 of the pen 70 detects pressure.

[0076] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for obtaining handwriting information, It is characterized in that The method: Acquire multiple groups of collected data collected from the pen at multiple collection moments during the writing process of the pen, wherein each group of collected data includes movement-related data of the pen and an initial image obtained by photographing the pen tip; Determine the position representation information of the pen tip at each of the acquisition moments by using the motion-related data; Based on the position representation information of each acquisition moment, the initial image at each acquisition moment is transformed into a preset coordinate system to obtain a transformed image at each acquisition moment; wherein the transformed image at each acquisition moment is used to identify the handwriting of the pen; The transformed image is cropped, wherein adjacent sides of the transformed image at adjacent acquisition moments after cropping are parallel; splicing the cropped transformed images at each of the acquisition moments to obtain a spliced ​​image; The stitched image is recognized to obtain the handwriting in the stitched image, or the stitched image is sent to a handwriting recognition device.

2. The method according to claim 1, It is characterized in that The motion-related data includes at least two sensor data of acceleration data, angular velocity data and magnetic field data; The using the motion-related data to determine the position representation information of the pen tip at each acquisition time includes: The at least two types of sensor data are fused and calculated to obtain the position representation information of the pen tip at each of the acquisition moments.

3. The method according to claim 1, It is characterized in that The position representation information at each of the acquisition moments is the attitude angle of the pen tip in the preset coordinate system at each of the acquisition moments; The step of transforming the initial image at each acquisition moment into the preset coordinate system based on the position representation information at each acquisition moment to obtain the transformed image at each acquisition moment includes: For each of the acquisition moments, the initial image at the acquisition moment is rotated using the posture angle corresponding to the acquisition moment to obtain a transformed image at the acquisition moment based on the preset coordinate system.

4. The method according to claim 3, It is characterized in that The preset coordinate system is established with the position of the pen tip at the first acquisition moment as the origin.

5. The method according to claim 1, It is characterized in that The method is performed by the pen; The acquiring of multiple sets of collected data collected from the pen at multiple collection moments during the writing process of the pen includes: Using the image acquisition component of the pen to acquire images of the pen tip at the multiple acquisition moments respectively, to obtain initial images corresponding to the multiple acquisition moments; and The sensing component of the pen is used to collect data at the multiple collection moments respectively, so as to obtain motion-related data corresponding to the multiple collection moments.

6. The method according to claim 5, It is characterized in that The method further comprises: After the pen is powered, if it is detected that the pressure detection component of the pen receives a pressure from the pen tip, it is determined to enter the writing mode, wherein the acquisition of multiple sets of collected data collected from the pen at multiple collection moments during the writing process of the pen is performed in the writing mode; After entering the writing mode, if it is detected that the pressure detection component does not receive pressure from the pen tip within a preset time period, the writing mode is exited.

7. A device for acquiring handwriting information, It is characterized in that comprising a processor and a memory connected to each other, wherein: The memory is used to store program instructions; The processor is used to execute the program instructions to implement the method according to any one of claims 1 to 6.

8. The device according to claim 7, It is characterized in that The acquisition device is a pen, which further comprises a pen housing, a pen core arranged in the pen housing, an image acquisition component and a sensor component, wherein the image acquisition component and the sensor component are connected to the processor, wherein: The image acquisition component is used to acquire an initial image from the pen tip, and the sensor component is used to acquire movement-related data of the pen.

9. The device according to claim 8, It is characterized in that The sensing component includes at least two of an acceleration sensor, a magnetic sensor and a gyroscope; And / or, the pen further includes at least one of the following: A pressure detection component is arranged in the pen housing, and the pressure detection component is arranged at an end of the pen core away from the pen tip, and is used to detect whether the pen tip is pressed; A wireless communication component disposed in the pen housing, the wireless communication component being used to communicate with an external device; A battery disposed in the pen housing, the battery being used to power the pen; A connection interface provided outside the pen housing, the connection interface being connected to the processor and used for connecting an external connection line; an indicator light disposed outside the pen housing, the indicator light being connected to the processor; A power switch is arranged outside the pen housing.

Images