Written character recognition method, computer device and storage medium

Abstract

The present application provides a method for recognizing written characters, a computer device and a storage medium, wherein the method comprises: obtaining a current frame writing image of a writing pen, wherein the writing pen is covered with at least one preset marker, and the relative position between the preset marker and the pen tip of the writing pen is fixed; determining the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image; and determining the characters written by the writing pen according to the multiple pen tip positions corresponding to different frame writing images. This can improve the accuracy of written character recognition while reducing hardware costs.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method for recognizing written text, a computer device, and a storage medium. Background Art

[0002] Nowadays, there are many products related to calligraphy practice on the market, most of which are exploring the use of paper and pen. In summary, they mainly include: (1) using an ordinary pen, placing a pressure sensor under the paper, and identifying the writing trajectory through the pressure sensor; (2) attaching a camera to the pen, using a deep learning algorithm, and identifying the writing trajectory through the camera. For method (1), since the paper is easy to move, the recognition of the writing trajectory by placing a pressure sensor under the paper will not be accurate enough; for method (2), installing a camera on the pen will make the pen larger and heavier, and the use of deep learning algorithms also requires high camera pixels and computing power, high hardware costs, and poor user experience.

[0003] Therefore, how to improve the accuracy of written text recognition while reducing hardware costs has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a method for recognizing written text, a computer device, and a storage medium, which can improve the accuracy of written text recognition while reducing hardware costs.

[0005] In a first aspect, the present application provides a method for recognizing written text, the method comprising:

[0006] Acquire a current frame writing image of a writing pen, wherein the writing pen is covered with at least one preset marker, and a relative position between the preset marker and the tip of the writing pen is fixed;

[0007] Determining the position of the pen tip of the writing pen in the writing image of the current frame according to the preset marker in the writing image of the current frame;

[0008] The text written by the writing pen is determined according to the multiple pen tip positions corresponding to different frames of writing images.

[0009] In a second aspect, the present application further provides a computer device, the computer device comprising:

[0010] Memory and processor;

[0011] Wherein, the memory is connected to the processor and is used to store programs;

[0012] The processor is used to implement the steps of the written text recognition method as described in any one of the embodiments of the present application by running the program stored in the memory.

[0013] In a third aspect, the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the written text recognition method as described in any one of the embodiments of the present application.

[0014] The written text recognition method, computer device and storage medium disclosed in the present application obtain the current frame writing image of the writing pen, wherein the writing pen is covered with at least one preset marker, and the relative position between the preset marker and the pen tip of the writing pen is fixed. According to the preset marker in the current frame writing image, the pen tip position of the writing pen in the current frame writing image is determined, so that multiple pen tip positions corresponding to different frames of writing images are obtained, and the text written by the writing pen is determined according to the multiple pen tip positions. There is no need to attach a camera to the pen, nor is there a need to lay a pressure sensor under the paper, thus reducing the hardware cost; moreover, it will not be affected by the movement of the paper during the writing process, and the pen tip position is accurately located based on the preset marker, thus improving the accuracy of written text recognition.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is a schematic diagram of the steps of a method for recognizing written characters provided in an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of a preset marker provided in an embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of the steps of another method for recognizing written characters provided in an embodiment of the present application;

[0020] Figure 4 This is a schematic diagram of the steps of another method for recognizing written characters provided in an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of a step of determining the position of the pen tip of the writing pen in the current frame writing image according to the preset marker in the current frame writing image provided by an embodiment of the present application;

[0022] Figure 6 This is a schematic diagram of the steps of another method for recognizing written characters provided in an embodiment of the present application;

[0023] Figure 7 It is a schematic diagram of a step of determining whether the writing pen has been put down in the current frame writing image provided by an embodiment of the present application;

[0024] Figure 8 It is a schematic diagram of a step of determining the text written by the writing pen according to a plurality of pen tip positions corresponding to different frames of writing images provided by an embodiment of the present application;

[0025] Fig. 9 This is a schematic diagram of the steps of another method for recognizing written characters provided in an embodiment of the present application;

[0026] Fig.10 It is a schematic block diagram of a computer device provided in an embodiment of the present application.

[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0030] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] It should be understood that in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with substantially the same functions and effects. For example, the first callback function and the second callback function are only used to distinguish different callback functions, and their order is not limited. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit them to be different.

[0032] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] Nowadays, there are many products related to calligraphy practice on the market, most of which are exploring the use of paper and pen. In summary, they mainly include: (1) using an ordinary pen, placing a pressure sensor under the paper, and identifying the writing trajectory through the pressure sensor; (2) attaching a camera to the pen, using a deep learning algorithm, and identifying the writing trajectory through the camera. For method (1), since the paper is easy to move, the recognition of the writing trajectory by placing a pressure sensor under the paper will not be accurate enough; for method (2), installing a camera on the pen will make the pen larger and heavier, and the use of deep learning algorithms also requires high camera pixels and computing power, high hardware costs, and poor user experience.

[0034] To this end, embodiments of the present application provide a method for recognizing written text, a computer device, and a storage medium, so as to improve the accuracy of written text recognition while reducing hardware costs.

[0035] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0036] See also Figure 1 , Figure 1 The present invention is a flowchart of a method for recognizing written text provided in an embodiment of the present invention. The method for recognizing written text can be applied to a terminal device, wherein the terminal device can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, a wearable device or a smart screen, a smart TV, a projector, a reading pen, a computer device, etc., but is not limited thereto.

[0037] like Figure 1 As shown, the method for recognizing written text can be applied to a computer device, and the method for recognizing written text applied to a computer device includes steps S101 to S103.

[0038] S101. Acquire a current frame writing image of a writing pen, wherein the writing pen is covered with at least one preset marker, and a relative position between the preset marker and the tip of the writing pen is fixed.

[0039] Exemplarily, a unique preset marker is pre-designed, and the preset marker is different from some conventional symbols. Figure 2 As shown in FIG. 1 , the preset marker is a combination of a four-pointed star and an arrow. It should be noted that the preset marker may also be different from Figure 2 This application does not specifically limit other markers than the preset markers shown.

[0040] At least one preset marker is attached to the writing pen, for example, a plurality of preset markers are attached around the pen barrel at one end of the writing pen near the pen tip. In this way, the preset marker can be captured when shooting at different angles. After the preset marker is attached to the writing pen, the relative position of the preset marker and the pen tip of the writing pen is fixed.

[0041] Afterwards, when writing text with the writing pen, a camera or other imaging device is used to capture the current frame of writing image of the writing pen. The imaging device may be any device with a photographing function in the current writing scene.

[0042] Exemplarily, the shooting cycle is preset, for example, the shooting cycle is set to 500ms. It is understandable that the shooting cycle can be flexibly set according to actual conditions, and no specific limitation is made here.

[0043] According to the shooting cycle, the current frame writing image of the writing pen is shot periodically. For example, the current frame writing image of the writing pen is shot once every 500ms.

[0044] S102: Determine the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image.

[0045] Since the relative position between the preset marker and the pen tip of the writing pen is fixed, the position of the pen tip of the writing pen in the current frame writing image can be determined according to the position of the preset marker in the current frame writing image.

[0046] In some embodiments, Figure 3 As shown, step S104 may be included before step S102, and step S102 may include sub-step S1021.

[0047] S104: Identify the preset marker on the current frame writing image.

[0048] S1021. Determine the pen tip position of the writing pen in the current frame writing image according to the recognized preset marker.

[0049] Exemplarily, identifying the preset marker on the current frame writing image may include:

[0050] Inputting the current frame writing image into a neural network model to identify the preset marker; and / or

[0051] The current frame writing image is compared with a preset marker reference image to identify the preset marker.

[0052] In some embodiments, a CNN (Convolutional Neural Networks) model for pre-setting marker recognition is pre-built, and the current frame writing image is input into the CNN model to recognize the preset marker and output a corresponding recognition result. The recognition result includes no recognition of the preset marker, recognition of one preset marker, recognition of multiple preset markers, etc.

[0053] In other embodiments, the preset marker is photographed in advance to generate a preset marker reference image, and the current frame writing image is compared with the preset marker reference image, that is, the two images are compared to identify whether the current frame writing image contains the preset marker.

[0054] Exemplarily, comparing the current frame writing image with a preset marker reference image to identify the preset marker may include:

[0055] Acquire first image feature information of the current frame writing image, and acquire second image feature information of the preset marker reference image;

[0056] Determining a degree of matching between the first image feature information and the second image feature information;

[0057] If the matching degree between the first image feature information and the second image feature information is greater than or equal to a preset threshold, determining that the preset marker is recognized in the current frame writing image;

[0058] If the matching degree between the first image feature information and the second image feature information is less than the preset threshold, it is determined that the preset marker is not recognized in the current frame writing image.

[0059] A preset threshold corresponding to the image feature information matching is preset in advance, and the preset threshold can be flexibly set according to actual conditions and is not specifically limited here.

[0060] Among them, the image feature information includes but is not limited to Multi Channel HOG (Histogram of Oriented Gradient) feature information, SIFT (Scale-invariant feature transform) feature information, etc.

[0061] For example, the Multi Channel HOG feature information of the writing image of the current frame is obtained, and the Multi Channel HOG feature information of the preset marker reference image is obtained. The Multi Channel HOG feature information of the writing image of the current frame is compared with the Multi Channel HOG feature information of the preset marker reference image to determine the matching degree between the Multi Channel HOG feature information of the writing image of the current frame and the Multi Channel HOG feature information of the preset marker reference image.

[0062] If the degree of match between the Multi Channel HOG feature information of the current frame writing image and the Multi Channel HOG feature information of the preset marker reference image is greater than or equal to a preset threshold, it is determined that at least one preset marker is recognized in the current frame writing image.

[0063] If the degree of match between the Multi Channel HOG feature information of the current frame writing image and the Multi Channel HOG feature information of the preset marker reference image is less than a preset threshold, it is determined that the preset marker is not recognized in the current frame writing image.

[0064] If a preset marker is identified, the pen tip position of the writing pen in the current frame writing image is determined directly based on the identified preset marker.

[0065] If multiple preset markers are identified, one preset marker is selected from the multiple preset markers, and then the pen tip position of the writing pen in the current frame writing image is determined based on the selected preset marker.

[0066] In some embodiments, selecting a preset marker from the plurality of preset markers may include:

[0067] According to the area of ​​each of the identified preset markers, a preset marker with the largest area is selected from the plurality of preset markers.

[0068] The area of ​​the preset marker includes but is not limited to the pixel area of ​​the preset marker.

[0069] For example, still Figure 2Taking the preset marker of the four-pointed star and arrow combination shown in FIG. 1 as an example, if multiple preset markers of the four-pointed star and arrow combination are identified in the current frame writing image, the pixel area of ​​each group of four-pointed stars is calculated. Then, according to the obtained pixel areas, the four-pointed star and arrow combination with the largest pixel area is selected as a valid four-pointed star and arrow combination, and the others are invalid four-pointed star and arrow combinations.

[0070] It should be noted that, in addition to selecting the preset marker with the largest area from multiple preset markers based on area as listed above, a preset marker can also be selected from multiple preset markers in other ways, and this application does not make any specific limitations.

[0071] Afterwards, the position of the pen tip of the writing pen in the current frame writing image is determined according to the position of the selected valid four-pointed star and arrow combination.

[0072] In some embodiments, Figure 4 As shown, step S104 may include step S105 and step S106.

[0073] S105, if the preset marker is not recognized, start timing;

[0074] S106: When the timing reaches a preset time, return to execute step S101.

[0075] Exemplarily, a preset duration is pre-set, wherein the preset duration can be flexibly set according to actual conditions and is not specifically limited here. For example, the preset duration is set to 500ms.

[0076] For example, when the preset marker is not recognized, the timing starts, and after a preset time, such as 500ms, the current frame is recorded, the current frame writing image corresponding to the writing pen is photographed, and a new current frame writing image is obtained again. Then the above operation is repeated, and the preset marker is recognized again for the new current frame writing image, which will not be repeated here.

[0077] In some embodiments, the written text recognition method may further include:

[0078] If the preset marker is not recognized, the number of consecutive times the preset marker is not recognized is accumulated;

[0079] When the number of times reaches a first preset number threshold, the step of acquiring the current frame writing image of the writing pen is terminated.

[0080] When the preset marker is not recognized, a new current frame writing image is acquired again, the preset marker is recognized for the new current frame writing image, and the number of consecutive failures to recognize the preset marker is accumulated. When the accumulated number of consecutive failures to recognize the preset marker has not reached the first preset number threshold, the above operation is repeated to recognize the preset marker for the new current frame writing image. When the accumulated number of consecutive failures to recognize the preset marker reaches the first preset number threshold, it means that the user may have stopped writing. At this time, there is no need to repeat the operation of acquiring the current frame writing image of the writing pen, and the operation of recognizing the preset marker is terminated.

[0081] It is understandable that the first preset number threshold can be flexibly set according to actual conditions and is not specifically limited here.

[0082] In some embodiments, Figure 5 As shown, step S102 may include sub-steps S1022 to S1025.

[0083] S1022: Fit each point corresponding to the preset marker to generate a fitting line segment corresponding to the preset marker, and determine a first length of the fitting line segment.

[0084] For example, still Figure 2 Taking the preset marker of the four-pointed star and arrow combination as an example, based on the position of each point in the arrow, each point is fitted to obtain the fitting line segment corresponding to the arrow, and then the line segment length of the fitting line segment is calculated. For the convenience of description, the line segment length of the fitting line segment is referred to as the first length below.

[0085] S1023. Obtain a second length corresponding to the preset marker, and a distance between the preset marker and the tip of the writing pen.

[0086] Exemplarily, the length corresponding to the preset marker, the distance between the preset marker and the pen tip of the writing pen and other information are stored in a preset storage address in advance. The length corresponding to the preset marker and the distance between the preset marker and the pen tip of the writing pen are obtained by accessing the storage address. For ease of description, the length corresponding to the preset marker is referred to as the second length below.

[0087] For example, still Figure 2 Taking the preset marker of the combination of a four-pointed star and an arrow as an example, the overall straight length of the arrow and the distance between the arrow tip and the pen tip of the writing pen are obtained.

[0088] S1024: Determine an extension distance according to the first length, the second length, and the distance.

[0089] Exemplarily, the first length of the fitting line segment is multiplied by the distance between the preset marker and the tip of the writing pen, and then divided by the second length corresponding to the preset marker to obtain the extended distance.

[0090] For example, assuming that the first length of the fitting line segment is mf, the second length corresponding to the preset marker is m, and the distance between the preset marker and the tip of the writing pen is d, the extended distance ml is calculated according to the calculation formula ml=mf*d / m.

[0091] S1025: Extend the preset marker in a direction pointing to the pen tip by the extension distance, and determine the extension end point as the pen tip position.

[0092] For example, still taking the above example, the arrow is extended along the direction of the arrow, and the extension distance is mf*d / m. After the extension, the extension end point is obtained, and the extension end point is the pen tip position of the writing pen.

[0093] S103, determining the text written by the writing pen according to the multiple pen tip positions corresponding to different frames of writing images.

[0094] Repeating the operations of step S101 and step S102 above can determine multiple pen tip positions corresponding to different frames of writing images during the writing process of the writing pen. The determined multiple pen tip positions are summarized to determine the text written by the writing pen.

[0095] In some embodiments, Figure 6 As shown, step S102 may include step S107 and step S108, and step S103 may include sub-step S1031.

[0096] S107: Determine whether the writing pen is put down in the writing image of the current frame.

[0097] Since the writing pen may be in the writing state or in the non-writing state in the acquired current frame writing image, and determining the text written by the writing pen only requires the position of the writing pen tip in the writing state. Therefore, in addition to determining the position of the writing pen tip, it is also determined whether the writing pen is in the writing state in the current frame writing image.

[0098] In some embodiments, Figure 7 As shown, step S107 may include sub-step S1071 and sub-step S1072.

[0099] S1071. According to the pen tip position, capture a ROI image from the writing image of the current frame.

[0100] Exemplarily, a ROI (region of interest) image centered at the pen tip position of the writing pen is captured from the current frame writing image.

[0101] For example, the image size of the ROI image is preset, and based on the image size, the ROI image of the image size centered at the pen tip position of the writing pen is intercepted.

[0102] S1072, input the ROI image into a binary classification network model, and output whether the writing pen has been put down.

[0103] Among them, the binary classification network model includes but is not limited to a binary classification network model based on ResNet (Residual Network).

[0104] A large number of sample images of writing pen put down / not put down are obtained in advance, the sample images of writing pen put down / not put down are input into the constructed binary classification network model, the binary classification network model is trained, and a trained binary classification network model is obtained.

[0105] Afterwards, the obtained ROI image is input into the trained binary classification network model, and the writing pen puts down information, or outputs the writing pen does not put down information.

[0106] It should be noted that in addition to the above-mentioned binary classification network model, other methods can also be used to determine whether the writing pen has been put down, and this application does not impose specific restrictions. For example, image recognition technology is used to perform image recognition processing on the ROI image to determine whether the writing pen has been put down.

[0107] S108. If the writing pen is put down in the current frame writing image, then the pen tip position corresponding to the current frame writing image is recorded.

[0108] If it is determined that the writing pen has been put down in the writing image of the current frame, then the position of the pen tip corresponding to the writing image of the current frame is recorded, that is, the position of the pen tip is saved.

[0109] On the contrary, if it is determined that the writing pen has not been put down in the current frame writing image, the record and save operation is not performed. A new current frame writing image corresponding to the writing pen is obtained by a new shooting device, so as to determine the new pen tip position of the writing pen according to the new current frame writing image.

[0110] S1031. Determine the text written by the writing pen according to the multiple pen tip positions corresponding to different frames of recorded writing images.

[0111] For example, suppose that ten frames of writing images are captured and the pen tip position of the first frame, the pen tip position of the third frame, the pen tip position of the fourth frame, the pen tip position of the fifth frame, the pen tip position of the eighth frame, the pen tip position of the ninth frame and the tenth frame are recorded, among which the position of the writing pen is not recorded in the second frame, the sixth frame and the seventh frame because the writing pen has not been put to paper.

[0112] The text written by the writing pen is determined based on the pen tip position of the first frame writing image, the pen tip position of the third frame writing image, the pen tip position of the fourth frame writing image, the pen tip position of the fifth frame writing image, the pen tip position of the eighth frame writing image, the pen tip position of the ninth frame writing image and the pen tip position of the tenth frame writing image.

[0113] In some embodiments, before determining the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of recorded writing images, the method may include:

[0114] If the number of times that the writing pen has not been put down continuously reaches a second preset number threshold, determining that the writing pen has been lifted;

[0115] Determining the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of recorded writing images may include:

[0116] After determining that the writing pen is lifted, the text written by the writing pen is determined based on a plurality of pen tip positions corresponding to different frames of recorded writing images.

[0117] If it is determined that the writing pen has not been put down in the current frame writing image, the position of the writing pen tip in the current frame writing image is not recorded, but the number of times the writing pen has not been put down in a row is accumulated.

[0118] Then, a new image is captured by the camera device to obtain a new current frame writing image corresponding to the writing pen, so as to determine the new pen tip position of the writing pen according to the new current frame writing image, and determine whether the writing pen is put down in the new current frame writing image, and repeat the above operation. If the accumulated number of times the writing pen has not put down the pen reaches a second preset number threshold, it is determined that the writing pen is lifted, and the user's writing operation may be finished.

[0119] It is understandable that the first preset number threshold can be flexibly set according to actual conditions and is not specifically limited here.

[0120] After determining that the writing pen is lifted, the text written by the writing pen is determined based on a plurality of pen tip positions corresponding to different frames of recorded writing images.

[0121] In some embodiments, Figure 8As shown, step S103 may include sub-step S1032 and sub-step S1033.

[0122] S1032. Generate a writing trajectory of the writing pen according to the multiple pen tip positions corresponding to different frames of writing images.

[0123] Exemplarily, according to the multiple pen tip positions corresponding to the different frames of recorded writing images, and according to the order in which the frame of writing image corresponding to each pen tip position is taken, the multiple pen tip positions are connected in order to generate the writing trajectory of the writing pen.

[0124] S1033. Perform text recognition according to the writing trajectory to determine the text written by the writing pen.

[0125] Exemplarily, the generated writing trace is subjected to OCR (Optical Character Recognition) recognition to determine the text written by the writing pen.

[0126] It should be noted that in addition to the OCR recognition methods listed above, text recognition of writing trajectories can also be performed in other ways, which are not specifically limited in this application.

[0127] In some embodiments, Fig. 9 As shown, step S109 may be included after step S103.

[0128] S109: judging the writing level of the text written by the writing pen according to the writing trajectory, and / or outputting writing suggestions.

[0129] That is, the text written by the writing pen is recognized, and the writing level of the user is judged according to the handwriting corresponding to the text.

[0130] For application scenarios where users practice calligraphy, corresponding writing suggestions can be output based on the handwriting, so that users can achieve better calligraphy practice results based on the writing suggestions.

[0131] The method for recognizing written text disclosed in the above embodiment obtains the current frame writing image of the writing pen, wherein the writing pen is covered with at least one preset marker, and the relative position between the preset marker and the pen tip of the writing pen is fixed. According to the preset marker in the current frame writing image, the pen tip position of the writing pen in the current frame writing image is determined, and in this way, multiple pen tip positions corresponding to different frames of writing images are obtained, and the text written by the writing pen is determined according to the multiple pen tip positions. There is no need to attach a camera to the pen, nor is there a need to lay a pressure sensor under the paper, thereby reducing the hardware cost; moreover, it will not be affected by the movement of the paper during the writing process, and the pen tip position is accurately located based on the preset marker, thereby improving the accuracy of written text recognition.

[0132] See also Fig.10 , Fig.10 is a schematic block diagram of a computer device provided in an embodiment of the present application. Fig.10 As shown, the computer device 300 includes one or more processors 301 and a memory 302. The processor 301 and the memory 302 are connected via a bus, such as an I2C (Inter-integrated Circuit) bus.

[0133] Among them, one or more processors 301 work individually or collectively to execute the steps of the written text recognition method provided in the above embodiment.

[0134] Specifically, the processor 301 may be a micro-controller unit (MCU), a central processing unit (CPU) or a digital signal processor (DSP).

[0135] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0136] The processor 301 is used to run the computer program stored in the memory 302, and implement the steps of the written text recognition method provided in the above embodiment when executing the computer program.

[0137] Exemplarily, the processor 301 is used to run the computer program stored in the memory 302, and implement the following steps when executing the computer program:

[0138] Acquire a current frame writing image of a writing pen, wherein the writing pen is covered with at least one preset marker, and a relative position between the preset marker and the tip of the writing pen is fixed;

[0139] Determining the position of the pen tip of the writing pen in the writing image of the current frame according to the preset marker in the writing image of the current frame;

[0140] The text written by the writing pen is determined according to the multiple pen tip positions corresponding to different frames of writing images.

[0141] In some embodiments, after determining the position of the pen tip of the writing pen in the writing image of the current frame, the processor 301 is configured to implement:

[0142] Determine whether the writing pen is put down in the current frame writing image;

[0143] If the writing pen is put down in the current frame writing image, then the pen tip position corresponding to the current frame writing image is recorded;

[0144] When the processor 301 determines the text written by the writing pen according to the plurality of pen tip positions corresponding to the writing images of different frames, it is used to implement:

[0145] The text written by the writing pen is determined according to the multiple pen tip positions corresponding to different frames of recorded writing images.

[0146] In some embodiments, before determining the position of the pen tip of the writing pen in the current frame writing image according to the preset marker in the current frame writing image, the processor 301 is configured to implement:

[0147] Recognizing the preset marker on the current frame writing image;

[0148] When the processor 301 determines the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image, it is used to implement:

[0149] The pen tip position of the writing pen in the current frame writing image is determined according to the recognized preset marker.

[0150] In some embodiments, when implementing the recognition of the preset marker on the current frame writing image, the processor 301 is used to implement:

[0151] Inputting the current frame writing image into a neural network model to identify the preset marker; or

[0152] The current frame writing image is compared with a preset marker reference image to identify the preset marker.

[0153] In some embodiments, when the processor 301 compares the current frame writing image with a preset marker reference image to identify the preset marker, it is configured to implement:

[0154] Acquire first image feature information of the current frame writing image, and acquire second image feature information of the preset marker reference image;

[0155] Determining a degree of matching between the first image feature information and the second image feature information;

[0156] If the matching degree between the first image feature information and the second image feature information is greater than or equal to a preset threshold, determining that the preset marker is recognized in the current frame writing image;

[0157] If the matching degree between the first image feature information and the second image feature information is less than the preset threshold, it is determined that the preset marker is not recognized in the current frame writing image.

[0158] In some embodiments, after implementing the recognition of the preset marker on the current frame writing image, the processor 301 is used to implement:

[0159] If the preset marker is not recognized, start timing;

[0160] When the timing reaches the preset time length, the process returns to the step of obtaining the current frame writing image of the writing pen.

[0161] In some embodiments, the processor 301 is further configured to implement:

[0162] If the preset marker is not recognized, the number of consecutive times the preset marker is not recognized is accumulated;

[0163] When the number of times reaches a first preset number threshold, the step of acquiring the current frame writing image of the writing pen is terminated.

[0164] In some embodiments, when the processor 301 determines the pen tip position of the writing pen in the current frame writing image according to the recognized preset marker, it is configured to implement:

[0165] If multiple preset markers are identified, selecting one preset marker from the multiple preset markers;

[0166] The pen tip position corresponding to the current frame writing image is determined according to the selected preset marker.

[0167] In some embodiments, when implementing the selecting of a preset marker from a plurality of preset markers, the processor 301 is configured to implement:

[0168] According to the area of ​​each of the identified preset markers, a preset marker with the largest area is selected from the plurality of preset markers.

[0169] In some embodiments, when the processor 301 determines the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image, it is used to implement:

[0170] Fitting each point corresponding to the preset marker to generate a fitting line segment corresponding to the preset marker, and determining a first length of the fitting line segment;

[0171] Acquire a second length corresponding to the preset marker, and a distance between the preset marker and the tip of the writing pen;

[0172] Determine an extension distance according to the first length, the second length, and the distance;

[0173] The preset marker is extended by the extension distance in a direction pointing to the pen tip, and the extension end point is determined as the pen tip position.

[0174] In some embodiments, when the processor 301 implements the step of determining whether the writing pen in the current frame writing image has been put down, it is configured to implement:

[0175] According to the pen tip position, intercepting the ROI image from the current frame writing image;

[0176] The ROI image is input into a binary classification network model, and outputs whether the writing pen has been put down.

[0177] In some embodiments, before determining the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of recorded writing images, the processor 301 is configured to implement:

[0178] If the number of times that the writing pen has not been put down continuously reaches a second preset number threshold, determining that the writing pen has been lifted;

[0179] When the processor 301 determines the text written by the writing pen according to the plurality of pen tip positions corresponding to the recorded writing images of different frames, it is used to implement:

[0180] After determining that the writing pen is lifted, the text written by the writing pen is determined based on a plurality of pen tip positions corresponding to different frames of recorded writing images.

[0181] In some embodiments, when the processor 301 determines the text written by the writing pen according to the plurality of pen tip positions corresponding to the writing images of different frames, it is configured to implement:

[0182] Generating a writing trajectory of the writing pen according to the plurality of pen tip positions corresponding to different frames of writing images;

[0183] According to the writing trajectory, text recognition is performed to determine the text written by the writing pen.

[0184] In some embodiments, after generating the writing trajectory of the writing pen, the processor 301 is used to implement:

[0185] According to the writing trajectory, the writing level of the text written by the writing pen is judged, and / or writing suggestions are output.

[0186] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the written text recognition method provided in the above embodiment.

[0187] The computer-readable storage medium may be an internal storage unit of the computer device described in any of the above embodiments, such as a hard disk or memory of the terminal device. The computer-readable storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device.

[0188] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for recognizing written characters, characterized in that: The method comprises: Acquire a current frame writing image of a writing pen, wherein the writing pen is covered with at least one preset marker, and a relative position between the preset marker and the tip of the writing pen is fixed; Determining the position of the pen tip of the writing pen in the writing image of the current frame according to the preset marker in the writing image of the current frame; Determining the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of writing images; The step of determining the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image includes: Fitting each point corresponding to the preset marker to generate a fitting line segment corresponding to the preset marker, and determining a first length of the fitting line segment; Acquire a second length of the preset marker and a distance between the preset marker and the tip of the writing pen; Determine an extension distance according to the first length, the second length, and the distance; The preset marker is extended by the extension distance in a direction pointing to the pen tip, and the extension end point is determined as the pen tip position.

2. The method according to claim 1, characterized in that After determining the pen tip position of the writing pen in the current frame writing image, the method includes: Determine whether the writing pen is put down in the current frame writing image; If the writing pen is put down in the current frame writing image, then the pen tip position corresponding to the current frame writing image is recorded; The step of determining the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of writing images includes: The text written by the writing pen is determined according to the multiple pen tip positions corresponding to different frames of recorded writing images.

3. The method according to claim 1, characterized in that Before determining the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image, the method includes: Recognizing the preset marker on the current frame writing image; The step of determining the pen tip position of the writing pen in the current frame writing image according to the preset marker in the current frame writing image includes: The pen tip position of the writing pen in the current frame writing image is determined according to the recognized preset marker.

4. The method according to claim 3, characterized in that The identifying the preset marker on the current frame writing image includes: Inputting the current frame writing image into a neural network model to identify the preset marker; or The current frame writing image is compared with a preset marker reference image to identify the preset marker.

5. The method according to claim 4, characterized in that The step of comparing the current frame writing image with a preset marker reference image to identify the preset marker includes: Acquire first image feature information of the current frame writing image, and acquire second image feature information of the preset marker reference image; Determining a degree of matching between the first image feature information and the second image feature information; If the matching degree between the first image feature information and the second image feature information is greater than or equal to a preset threshold, determining that the preset marker is recognized in the current frame writing image; If the matching degree between the first image feature information and the second image feature information is less than the preset threshold, it is determined that the preset marker is not recognized in the current frame writing image.

6. The method according to claim 3, characterized in that After the preset marker is identified on the current frame writing image, the method includes: If the preset marker is not recognized, start timing; When the timing reaches the preset time length, the process returns to the step of obtaining the current frame writing image of the writing pen.

7. The method according to claim 6, characterized in that The method further comprises: If the preset marker is not recognized, the number of consecutive times the preset marker is not recognized is accumulated; When the number of times reaches a first preset number threshold, the step of acquiring the current frame writing image of the writing pen is terminated.

8. The method according to claim 3, characterized in that Determining the pen tip position of the writing pen in the current frame writing image according to the recognized preset marker includes: If multiple preset markers are identified, selecting one preset marker from the multiple preset markers; The pen tip position corresponding to the current frame writing image is determined according to the selected preset marker.

9. The method according to claim 8, characterized in that The step of selecting a preset marker from a plurality of preset markers comprises: According to the area of ​​each of the identified preset markers, a preset marker with the largest area is selected from the plurality of preset markers.

10. The method according to claim 2, characterized in that The determining whether the writing pen is put down in the current frame writing image comprises: According to the pen tip position, intercepting the ROI image from the current frame writing image; The ROI image is input into a binary classification network model, and outputs whether the writing pen has been put down.

11. The method according to claim 2, characterized in that Before determining the text written by the writing pen according to the plurality of pen tip positions corresponding to the different frames of writing images recorded, the method includes: If the number of times that the writing pen has not been put down continuously reaches a second preset number threshold, determining that the writing pen has been lifted; The determining of the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of recorded writing images comprises: After determining that the writing pen is lifted, the text written by the writing pen is determined based on a plurality of pen tip positions corresponding to different frames of recorded writing images.

12. The method according to any one of claims 1 to 11, characterized in that The step of determining the text written by the writing pen according to the plurality of pen tip positions corresponding to different frames of writing images includes: Generating a writing trajectory of the writing pen according to the plurality of pen tip positions corresponding to different frames of writing images; According to the writing trajectory, text recognition is performed to determine the text written by the writing pen.

13. The method according to claim 12, characterized in that After the writing trajectory of the writing pen is generated, the method includes: According to the writing trajectory, the writing level of the text written by the writing pen is judged, and / or writing suggestions are output.

14. A computer device, characterized in that: The computer device comprises: Memory and processor; Wherein, the memory is connected to the processor and is used to store programs; The processor is used to implement the steps of the written text recognition method as described in any one of claims 1 to 13 by running the program stored in the memory.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the method for recognizing written text as described in any one of claims 1 to 13.

Images