Stroke sequence recognition method and device and computer device

Abstract

The application discloses a stroke sequence recognition method and device and computer equipment, and mainly lies in that the stroke sequence recognition can be carried out in the daily writing process of students, and the writing problems of students can be corrected in time. The method comprises the following steps: in response to the rotation of a support device of a tablet computer by a preset angle, an image sequence in a learning scene is captured by using a shooting device arranged at one end of the support device; the image sequence is subjected to segmentation processing to obtain a hand image sequence and a writing tool image sequence; the hand image sequence and the writing tool image sequence are respectively subjected to registration to obtain a first motion displacement of a hand and a second motion displacement of a writing tool; and the stroke sequence of a current writing character is determined based on the first motion displacement and the second motion displacement.

Description

Technical Field

[0001] This invention relates to the field of intelligent education, and in particular to a stroke order recognition method, apparatus, and computer equipment. Background Technology

[0002] More and more students are choosing tablets for online classes and targeted training in single subjects, using tablets as virtual teachers. Therefore, the smart features of tablets are becoming increasingly important.

[0003] Currently, students typically need to install dedicated stroke practice software on their tablets to identify stroke order when practicing strokes. However, this method of stroke order recognition relies on specific software and cannot be performed during students' daily writing process, thus hindering the timely correction of students' writing problems. Summary of the Invention

[0004] This invention provides a stroke order recognition method, device, and computer equipment, which mainly enables stroke order recognition during students' daily writing process, thus facilitating timely correction of students' writing problems.

[0005] According to a first aspect of the present invention, a stroke order recognition method is provided, comprising:

[0006] In response to the tablet computer's support device rotating at a preset angle, an image sequence of the learning scenario is captured using a camera device built into one end of the support device.

[0007] The image sequence is segmented to obtain a hand image sequence and a writing tool image sequence;

[0008] The hand image sequence and the writing tool image sequence are registered respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool;

[0009] Based on the first motion displacement and the second motion displacement, the stroke order of the currently written character is determined.

[0010] Optionally, the step of segmenting the image sequence to obtain a hand image sequence and a writing tool image sequence includes:

[0011] The image sequence is segmented into hand segments to obtain the hand image sequence;

[0012] Determine the image center corresponding to any hand image in the hand image sequence;

[0013] Based on the image center and the preset cutout distance, the hand image sequence is cut out to obtain the writing tool image sequence.

[0014] Optionally, the registration of the hand image sequence and the writing tool image sequence to obtain the first motion displacement of the hand and the second motion displacement of the writing tool includes:

[0015] The hand image sequence is registered to obtain the registration point corresponding to any hand image in the sequence; based on the position information of the registration point corresponding to any hand image, the first motion displacement of the hand is determined; and,

[0016] The writing tool image sequence is registered to obtain the registration point corresponding to any writing tool image in the sequence; the second motion displacement of the writing tool is determined based on the position information of the registration point corresponding to any writing tool image.

[0017] Optionally, determining the stroke order of the currently written character based on the first motion displacement and the second motion displacement includes:

[0018] Calculate the displacement difference between the first motion displacement and the second motion displacement;

[0019] If the displacement difference is less than the preset displacement difference, then the position information of the pen tip in any writing tool image is determined;

[0020] The stroke order of the written character is determined based on the position information of the pen tip in any image of a writing tool.

[0021] Optionally, determining the position information of the pen tip in any image of a writing instrument includes:

[0022] Edge detection is performed on the writing tool image sequence to obtain the edge shape in any one of the writing tool images;

[0023] Determine the lower endpoint corresponding to the edge shape, and determine the position information of the lower endpoint as the position information of the pen tip in any writing tool image.

[0024] Optionally, the method further includes:

[0025] The written characters are identified, and the standard stroke order corresponding to the identified written characters is determined;

[0026] Compare the stroke order with the standard stroke order;

[0027] Based on the comparison results, the corresponding feedback information is displayed on the display device.

[0028] Optionally, recognizing the written characters includes:

[0029] Based on the position information of the pen tip, calculate the offset distance of the pen tip in any two adjacent images of the writing tool;

[0030] From the plurality of said offset distances, determine a target offset distance that is greater than a preset offset distance, and determine the two images corresponding to the target offset distance;

[0031] For the two images corresponding to the target offset distance, character recognition is performed on the written characters under the pen tip in the previous image.

[0032] According to a second aspect of the present invention, a stroke order recognition device is provided, comprising:

[0033] The acquisition unit is used to capture a sequence of images in the learning scenario in response to the rotation of the support device of the tablet computer at a preset angle, using a shooting device built into one end of the support device.

[0034] A segmentation unit is used to segment the image sequence to obtain a hand image sequence and a writing tool image sequence;

[0035] A registration unit is used to register the hand image sequence and the writing tool image sequence respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool;

[0036] The determining unit is used to determine the stroke order of the currently written character based on the first motion displacement and the second motion displacement.

[0037] Optionally, the segmentation unit includes: a segmentation module, a first determination module, and a matting module.

[0038] The segmentation module is used to segment the image sequence into hand segments to obtain the hand image sequence;

[0039] The first determining module is used to determine the image center corresponding to any hand image in the hand image sequence;

[0040] The image cutout module is used to perform image cutout processing on the hand image sequence based on the image center and a preset cutout distance to obtain the writing tool image sequence.

[0041] Optionally, the configuration unit is specifically configured to perform a registration operation on the hand image sequence to obtain a registration point corresponding to any hand image in the hand image sequence; determine a first motion displacement of the hand based on the position information of the registration point corresponding to the arbitrary hand image; and perform a registration operation on the writing tool image sequence to obtain a registration point corresponding to any writing tool image in the writing tool image sequence; and determine a second motion displacement of the writing tool based on the position information of the registration point corresponding to the arbitrary writing tool image.

[0042] Optionally, the determining unit includes: a first calculation module and a second determining module.

[0043] The first calculation module is used to calculate the displacement difference between the first motion displacement and the second motion displacement;

[0044] The second determining module is used to determine the position information of the pen tip in any image of a writing tool if the displacement difference is less than a preset displacement difference.

[0045] The second determining module is further configured to determine the stroke order of the written character based on the position information of the pen tip in any image of a writing tool.

[0046] Optionally, the second determining module is specifically used to perform edge detection on the writing tool image sequence to obtain the edge shape in any one of the writing tool images; determine the lower endpoint corresponding to the edge shape, and determine the position information of the lower endpoint as the position information of the pen tip in the any one of the writing tool images.

[0047] Optionally, the device further includes: an identification unit, a comparison unit, and a display unit.

[0048] The recognition unit is used to recognize the written characters and determine the standard stroke order corresponding to the recognized written characters;

[0049] The comparison unit is used to compare the stroke order with the standard stroke order.

[0050] The display unit is used to display corresponding feedback information in the display device based on the comparison results.

[0051] Optionally, the recognition unit is specifically configured to calculate the offset distance of the pen tip in any two adjacent writing tool images based on the position information of the pen tip; determine a target offset distance greater than a preset offset distance from a plurality of offset distances, and determine two images corresponding to the target offset distance; and perform character recognition on the written characters under the pen tip in the previous image for the two images corresponding to the target offset distance.

[0052] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0053] In response to the tablet computer's support device rotating at a preset angle, an image sequence of the learning scenario is captured using a camera device built into one end of the support device.

[0054] The image sequence is segmented to obtain a hand image sequence and a writing tool image sequence;

[0055] The hand image sequence and the writing tool image sequence are registered respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool;

[0056] Based on the first motion displacement and the second motion displacement, the stroke order of the currently written character is determined.

[0057] According to a fourth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the following steps:

[0058] In response to the tablet computer's support device rotating at a preset angle, an image sequence of the learning scenario is captured using a camera device built into one end of the support device.

[0059] The image sequence is segmented to obtain a hand image sequence and a writing tool image sequence;

[0060] The hand image sequence and the writing tool image sequence are registered respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool;

[0061] Based on the first motion displacement and the second motion displacement, the stroke order of the currently written character is determined.

[0062] This invention provides a stroke order recognition method that, compared to using stroke practice software, can respond to the rotation of a tablet's support device at a preset angle and capture an image sequence of the learning scenario using a camera device built into one end of the support device. The image sequence is then segmented to obtain a hand image sequence and a writing tool image sequence. Simultaneously, the hand and writing tool image sequences are registered to obtain a first motion displacement of the hand and a second motion displacement of the writing tool. Finally, based on the first and second motion displacements, the stroke order of the currently written character is determined. Therefore, this invention can perform stroke recognition during students' daily writing process without requiring separate stroke practice software, thus facilitating timely correction of students' writing problems during daily practice. Attached Figure Description

[0063] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0064] Figure 1 This diagram illustrates a stroke order recognition method according to an embodiment of the present invention.

[0065] Figure 2 This invention provides a schematic flowchart of another stroke order recognition method.

[0066] Figure 3 This diagram illustrates the structure of a stroke order recognition device according to an embodiment of the present invention.

[0067] Figure 4 This invention provides a schematic diagram of the structure of another stroke order recognition device according to an embodiment of the invention.

[0068] Figure 5 A schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0069] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0070] Currently, students typically need to install dedicated stroke practice software on their tablets to recognize stroke order when practicing strokes. However, this method of stroke order recognition relies on fixed software and cannot be performed during students' daily writing process, thus hindering the timely correction of students' writing problems.

[0071] To address the aforementioned problems, embodiments of the present invention provide a stroke order recognition method that can automatically recognize the stroke order of characters when students write them on paper. Figure 1 As shown, the method includes:

[0072] 101. In response to the tablet computer's support device rotating at a preset angle, an image sequence of the learning scenario is captured using a shooting device built into one end of the support device.

[0073] The image sequence is a sequence of images captured using the camera device of a tablet computer, and the images include an arm, a hand, writing tools, and writing content.

[0074] This invention is primarily applicable to scenarios where students are writing characters on paper and need to recognize the strokes of those characters. The executing entity for this invention is the tablet computer disclosed above.

[0075] In this embodiment of the invention, after the supporting device rotates by a preset angle, the shooting device and the display device are activated. The shooting device can capture images of the student's writing area on the supporting surface to obtain an image sequence of the student. Specifically, the shooting device can record images of the student's writing area, such as continuously for 5 seconds, and then decompose these 5 seconds of video footage into multiple frames to obtain an image sequence of the learning scenario. In addition, the shooting device can also capture images of the student's writing area at predetermined time intervals, such as capturing the student's writing area once every 0.5 seconds, obtaining an image sequence captured within 3 consecutive seconds. Thus, by following the above method, an image sequence of the learning scenario can be obtained, so as to determine the stroke order of the student's writing based on the image sequence.

[0076] 102. The image sequence is segmented to obtain a hand image sequence and a writing tool image sequence.

[0077] In this embodiment of the invention, after acquiring the image sequence, it is necessary to perform image segmentation to obtain a hand image sequence and a writing tool image sequence. Specifically, step 102 includes: segmenting the image sequence by hand to obtain the hand image sequence; determining the image center corresponding to any hand image in the hand image sequence; and performing image matting processing on the hand image sequence based on the image center and a preset matting distance to obtain the writing tool image sequence. The preset matting distance can be set according to actual business needs.

[0078] Specifically, firstly, a preset hand segmentation model is used to segment the hands in multiple images of the image sequence, resulting in a hand image sequence. Each hand image includes a complete hand and a hand gripping a writing tool. Furthermore, the preset hand segmentation model can be a deep learning-based model, such as U-Net, or an improved version thereof, or a progressive dense V-network (PDV-NET) or other hand segmentation models. The methods for training U-Net, its improved version, or PDV-NET or other hand segmentation models are known to those skilled in the art, and are not described in detail in this embodiment.

[0079] Furthermore, after obtaining the hand image sequence, it is necessary to acquire the writing tool image sequence. This writing tool image includes part of the hand (such as fingers) and the writing tool itself, which can specifically refer to a pen, pencil, brush, or other writing implement. Specifically, first, the image center corresponding to each of the multiple hand images is determined. Then, using this image center as the center and a preset cutout distance as the radius, images containing the writing tool are extracted from the hand images to obtain the writing tool image sequence.

[0080] 103. Register the hand image sequence and the writing tool image sequence respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool.

[0081] In this embodiment of the invention, after acquiring the hand image sequence and the writing tool image sequence, image registration needs to be performed on the hand image sequence and the writing tool image sequence respectively, so as to determine the first motion displacement of the hand and the second motion displacement of the writing tool based on the registration result. For this process, step 103 specifically includes: performing a registration operation on the hand image sequence to obtain a registration point corresponding to any hand image in the hand image sequence; determining the first motion displacement of the hand based on the position information of the registration point corresponding to the arbitrary hand image; and performing a registration operation on the writing tool image sequence to obtain a registration point corresponding to any writing tool image in the writing tool image sequence; determining the second motion displacement of the writing tool based on the position information of the registration point corresponding to the arbitrary writing tool image.

[0082] Specifically, a first preset registration model is used to register a sequence of hand images, obtaining one-to-one registration points in multiple hand images. These registration points are then connected to obtain the first motion displacement of the hand. Similarly, a second preset registration model is used to register a sequence of writing tool images, obtaining one-to-one registration points in multiple writing tool images. These registration points are then connected to obtain the second motion displacement of the writing tool. The first and second preset registration models can be registration models based on the traditional SIFT algorithm or other flexible registration models, or they can be registration models based on deep learning, such as using the VGG network (VGG-net) in deep learning. Methods for training VGG-net or its improved models are known to those skilled in the art, and this embodiment of the invention will not provide a detailed description.

[0083] It should be noted that, in the embodiments of the present invention, the first movement displacement of the hand can be determined first, and then the second movement displacement of the writing tool can be determined, or the second movement displacement of the writing tool can be determined first, and then the first movement displacement of the hand can be determined. The embodiments of the present invention do not impose specific limitations on the order of determining the first movement displacement and the second movement displacement.

[0084] 104. Based on the first motion displacement and the second motion displacement, determine the stroke order of the currently written character.

[0085] In this embodiment of the invention, after determining the first movement displacement of the hand and the second movement displacement of the writing tool, it is determined whether the first movement displacement of the hand and the second movement displacement of the writing tool are consistent. If they are consistent, the stroke order of the character currently being written by the student can be determined based on the second movement displacement of the writing tool. If they are inconsistent, it indicates that there is an error in the calculation and the stroke order cannot be determined.

[0086] This invention provides a stroke order recognition method that, compared to using stroke practice software, can respond to the rotation of a tablet's support device by a preset angle and capture an image sequence of the learning scenario using a camera device built into one end of the support device. The image sequence is then segmented to obtain a hand image sequence and a writing tool image sequence. Simultaneously, the hand image sequence and the writing tool image sequence are registered to obtain a first motion displacement of the hand and a second motion displacement of the writing tool. Finally, based on the first and second motion displacements, the stroke order of the currently written character is determined. Therefore, this invention allows for stroke recognition during students' daily writing process without requiring separate stroke practice software, thus facilitating timely correction of students' writing problems during daily practice.

[0087] Furthermore, to better illustrate the above-described stroke order recognition process, as a refinement and extension of the above embodiments, this invention provides another stroke order recognition method, such as... Figure 2 As shown, the method includes:

[0088] 201. In response to the tablet computer's support device rotating at a preset angle, an image sequence of the learning scenario is captured using a shooting device built into one end of the support device.

[0089] In this embodiment of the invention, the process of obtaining the image sequence to be processed is exactly the same as step 101, and will not be repeated here.

[0090] 202. The image sequence is segmented to obtain a hand image sequence and a writing tool image sequence, and the hand image sequence and the writing tool image sequence are registered to obtain the first motion displacement of the hand and the second motion displacement of the writing tool.

[0091] In this embodiment of the invention, in addition to obtaining the first motion displacement of the student's hand and the second motion displacement of the writing tool through image segmentation and image registration as described in Embodiment 1, a preset image recognition model can also be used to recognize the hand and the writing tool separately. Specifically, multiple images in an image sequence can be input into a preset YOLO image recognition model for hand and writing tool recognition, obtaining the hand and writing tool in each image, as well as the position information of the hand and the writing tool in each image. Further, based on the position information of the hand and the writing tool in multiple images, the first motion displacement of the hand and the second motion displacement of the writing tool can be obtained. The method for training the YOLO image recognition model is known to those skilled in the art, and this embodiment of the invention will not provide a detailed description.

[0092] 203. Calculate the displacement difference between the first motion displacement and the second motion displacement. If the displacement difference is less than a preset displacement difference, determine the position information of the pen tip in any writing tool image.

[0093] The preset displacement difference can be set according to actual business needs, and the specific value of the preset displacement difference is not limited in this embodiment of the invention. In order to further improve the accuracy of stroke recognition, the stroke order can be determined by the movement trajectory of the pen tip. Specifically, after obtaining the first movement displacement of the hand and the second movement displacement of the writing tool, the displacement difference between the first and second movement displacements is calculated. If the displacement difference is greater than or equal to the preset displacement difference, it is determined that the movement trajectories of the hand and the writing tool are inconsistent, and no further calculation is performed. If the displacement difference is less than the preset displacement difference, it is determined that the movement trajectories of the hand and the writing tool are consistent, and the position information of the pen tip in each writing tool image is determined, so as to determine the stroke order based on the position information of the pen tip in each writing tool image.

[0094] As an optional implementation, the method for determining the position information of the pen tip in each writing tool image includes: performing edge detection on the writing tool image sequence to obtain the edge shape in any one of the writing tool images; determining the lower endpoint corresponding to the edge shape, and determining the position information of the lower endpoint as the position information of the pen tip in the any one of the writing tool images.

[0095] Specifically, an edge detection algorithm is used to perform edge detection on the writing tool image sequence to obtain the edge shape in each writing tool image. This edge shape includes the edge shape of part of the hand (such as fingers) and the edge shape of the writing tool. Then, the upper and lower endpoints of the edge shapes are determined. Since the pen tip is in close contact with the paper surface, the position information of the lower endpoint is determined as the position information of the pen tip. Thus, the position information of the pen tip in the writing tool image sequence can be obtained in the above manner. The edge detection algorithm is a commonly used algorithm by those skilled in the art, and the embodiments of this invention will not describe it in detail.

[0096] 204. Determine the stroke order of the written character based on the position information of the pen tip in any image of a writing tool.

[0097] In the embodiments of the present invention, after determining the position information of the pen tip in each writing tool image sequence, the movement trajectory of the pen tip can be inferred based on the position information of the pen tip in each writing tool image sequence, and thus the stroke order of the characters written by the student can be obtained, such as the final determined stroke order being to write "horizontal" first, then "left-falling" and finally "right-falling".

[0098] 205. Recognize the written characters and determine the standard stroke order corresponding to the recognized written characters.

[0099] For the embodiments of the present invention, after identifying the stroke order written by the student, it is necessary to compare it with the corresponding standard stroke order so as to give the student corresponding feedback on the display device according to the comparison result. Before comparing the stroke order, it is necessary to identify what character the student is writing so as to query the standard stroke order corresponding to the character from the preset stroke library. For the specific process of character recognition, as an optional implementation manner, the method includes: calculating the offset distance of the pen tip in any two adjacent writing tool images according to the position information of the pen tip; determining the target offset distance greater than the preset offset distance from multiple offset distances, and determining the two images corresponding to the target offset distance; for the two images corresponding to the target offset distance, performing character recognition on the writing character under the pen tip in the previous image. The preset offset distance can be set according to actual business needs, and the embodiments of the present invention do not limit the specific value of the preset offset distance.

[0100] Specifically, after determining the position information of the pen tip in each writing tool image, calculate the offset distance of the pen tip in any two adjacent writing tool images. Since during the process of character writing, the position of the pen tip writing the same character usually does not deviate too far, and when starting to write the next character, the pen tip usually needs to move a relatively long distance. Therefore, the target offset distance greater than the preset offset distance can be determined from multiple offset distances, and the two adjacent images corresponding to the target offset distance can be determined. In these two images, the character under the pen tip in the previous image has just been written, and the pen tip in the latter image has switched to the next character. Therefore, perform OCR recognition on the character under the pen tip in the previous image. For example, if the character written by the student is recognized as "big". Further, according to the recognized writing character, query the preset stroke library to determine the standard stroke order corresponding to the writing character.

[0101] In a specific application scenario, if the number of target offset distances greater than the preset offset distance is one, it means that the student has completed writing one character. At this time, only the previous image of the two images corresponding to one target offset distance needs to be subjected to character recognition; if the number of target offset distances greater than the preset offset distance is at least two, it means that the student has completed writing at least two characters. At this time, it is necessary to perform character recognition on the previous images of the two images corresponding to at least two target offset distances respectively.

[0102] It should be noted that the writing characters in the embodiments of the present invention are not limited to Chinese characters, but can also be English characters, or other types of characters with a writing order.

[0103] 206. Compare the stroke order with the standard stroke order, and display corresponding feedback information on the display device according to the comparison result.

[0104] For the embodiments of the present invention, the recognized stroke order is compared with the standard stroke order. For example, the recognized stroke order is "horizontal", "right-falling stroke", "left-falling stroke", while the standard stroke order of the character "大" is "horizontal", "left-falling stroke", "right-falling stroke". At this time, the display device will prompt the student that the stroke order is incorrect through voice or text, and show the student the correct stroke order to correct the student's writing problem in time.

[0105] Another stroke order recognition method provided by the embodiments of the present invention, compared with the method of using stroke practice software to recognize the stroke order, can respond to the rotation of the support device of the tablet computer by a preset angle, and use the shooting device built in one end of the support device to shoot an image sequence in the learning scenario; and perform segmentation processing on the image sequence to obtain a hand image sequence and a writing tool image sequence; at the same time, perform registration on the hand image sequence and the writing tool image sequence respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool; finally, based on the first motion displacement and the second motion displacement, determine the stroke order of the currently written character. It can be seen that the embodiments of the present invention can perform stroke recognition during the student's daily writing process without a separate stroke practice software, so it is convenient to correct the student's writing problems in time during the daily writing process.

[0106] Furthermore, as Figure 1 a specific implementation of, the embodiments of the present invention provide a stroke order recognition device, as Figure 3 shown, the device includes: an acquisition unit 31, a segmentation unit 32, a registration unit 33 and a determination unit 34.

[0107] The acquisition unit 31 can be used to respond to the rotation of the support device of the tablet computer by a preset angle, and use the shooting device built in one end of the support device to shoot an image sequence in the learning scenario.

[0108] The segmentation unit 32 can be used to perform segmentation processing on the image sequence to obtain a hand image sequence and a writing tool image sequence.

[0109] The registration unit 33 can be used to perform registration on the hand image sequence and the writing tool image sequence respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool.

[0110] The determination unit 34 can be used to determine the stroke order of the currently written character based on the first motion displacement and the second motion displacement.

[0111] In a specific application scenario, the segmentation unit 32, as Figure 4 shown, includes: a segmentation module 321, a first determination module 322 and a matte extraction module 323.

[0112] The segmentation module 321 can be used to segment the image sequence into hand segments to obtain the hand image sequence.

[0113] The first determining module 322 can be used to calculate the image center corresponding to any hand image in the hand image sequence.

[0114] The image cutout module 323 can be used to perform image cutout processing on the hand image sequence based on the image center and a preset cutout distance to obtain the writing tool image sequence.

[0115] In a specific application scenario, the registration unit 33 can be used to perform a registration operation on the hand image sequence to obtain a registration point corresponding to any hand image in the hand image sequence; determine the first motion displacement of the hand based on the position information of the registration point corresponding to the arbitrary hand image; and perform a registration operation on the writing tool image sequence to obtain a registration point corresponding to any writing tool image in the writing tool image sequence; and determine the second motion displacement of the writing tool based on the position information of the registration point corresponding to the arbitrary writing tool image.

[0116] In a specific application scenario, the determining unit 34 includes: a first calculation module 341 and a second determining module 342.

[0117] The first calculation module 341 can be used to calculate the displacement difference between the first motion displacement and the second motion displacement.

[0118] The second determining module 342 can be used to determine the position information of the pen tip in any writing tool image if the displacement difference is less than a preset displacement difference.

[0119] The second determining module 342 can also be used to determine the stroke order of the written character based on the position information of the pen tip in any image of a writing tool.

[0120] Furthermore, the second determining module 342 can be specifically used to perform edge detection on the writing tool image sequence to obtain the edge shape in any one of the writing tool images; determine the lower endpoint corresponding to the edge shape, and determine the position information of the lower endpoint as the position information of the pen tip in the any one of the writing tool images.

[0121] In specific application scenarios, the device further includes: an identification unit 35, a comparison unit 36, and a display unit 37.

[0122] The recognition unit 35 can be used to recognize the written characters and determine the standard stroke order corresponding to the recognized written characters.

[0123] The comparison unit 36 ​​can be used to compare the stroke order with the standard stroke order.

[0124] The display unit 37 can be used to display corresponding feedback information in the display device based on the comparison results.

[0125] Furthermore, the recognition unit 35 can be specifically used to calculate the offset distance of the pen tip in any two adjacent writing tool images based on the position information of the pen tip; determine a target offset distance greater than a preset offset distance from a plurality of offset distances, and determine two images corresponding to the target offset distance; and perform character recognition on the written characters under the pen tip in the previous image for the two images corresponding to the target offset distance.

[0126] It should be noted that other corresponding descriptions of the functional modules involved in the stroke order recognition device provided in this embodiment of the invention can be found in [reference]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0127] Based on the above, Figure 1 Accordingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: in response to the rotation of a tablet computer's support device by a preset angle, capturing an image sequence of a learning scenario using a camera device built into one end of the support device; segmenting the image sequence to obtain a hand image sequence and a writing tool image sequence; registering the hand image sequence and the writing tool image sequence to obtain a first motion displacement of the hand and a second motion displacement of the writing tool; and determining the stroke order of the currently written character based on the first motion displacement and the second motion displacement.

[0128] Based on the above, Figure 1 The method shown and as Figure 3 The embodiment of the device shown in the invention also provides a physical structure diagram of a computer device, such as... Figure 5As shown, the computer device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor. Both the memory 42 and the processor 41 are mounted on a bus 43. When the processor 41 executes the program, it performs the following steps: responding to a preset angle rotation of the tablet's support device, it captures an image sequence of the learning scenario using a camera device built into one end of the support device; it segments the image sequence to obtain a hand image sequence and a writing tool image sequence; it registers the hand image sequence and the writing tool image sequence to obtain a first motion displacement of the hand and a second motion displacement of the writing tool; and based on the first motion displacement and the second motion displacement, it determines the stroke order of the currently written character.

[0129] The embodiments of the present invention can perform stroke recognition during students' daily writing process without the need for separate stroke practice software, thus facilitating timely correction of students' writing problems during daily writing.

[0130] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0131] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A stroke order recognition method, characterized in that, include: In response to the tablet computer's support device rotating at a preset angle, an image sequence of the learning scenario is captured using a camera device built into one end of the support device. The image sequence is segmented to obtain a hand image sequence and a writing tool image sequence; The hand image sequence and the writing tool image sequence are registered respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool; Based on the first motion displacement and the second motion displacement, calculate the displacement difference between the first motion displacement and the second motion displacement; If the displacement difference is less than the preset displacement difference, the edge detection algorithm is used to perform edge detection on the writing tool image sequence to obtain the edge shape in each writing tool image. The edge shape includes the edge shape of part of the hand and the edge shape of the writing tool. Then the upper and lower endpoints of the edge shape are determined respectively. Since the pen tip is close to the paper surface, the position information of the lower endpoint is determined as the position information of the pen tip in any writing tool image. The stroke order of the written character is determined based on the position information of the pen tip in any image of a writing tool.

2. The method according to claim 1, characterized in that, The segmentation process of the image sequence to obtain a hand image sequence and a writing tool image sequence includes: The image sequence is segmented into hand segments to obtain the hand image sequence; Determine the image center corresponding to any hand image in the hand image sequence; Based on the image center and the preset cutout distance, the hand image sequence is cut out to obtain the writing tool image sequence.

3. The method according to claim 1, characterized in that, The step of registering the hand image sequence and the writing tool image sequence to obtain the first motion displacement of the hand and the second motion displacement of the writing tool includes: The hand image sequence is registered to obtain the registration point corresponding to any hand image in the sequence; based on the position information of the registration point corresponding to any hand image, the first motion displacement of the hand is determined; and, The writing tool image sequence is registered to obtain the registration point corresponding to any writing tool image in the sequence; the second motion displacement of the writing tool is determined based on the position information of the registration point corresponding to any writing tool image.

4. The method according to claim 1, characterized in that, The method further includes: The written characters are identified, and the standard stroke order corresponding to the identified written characters is determined; Compare the stroke order with the standard stroke order; Based on the comparison results, the corresponding feedback information is displayed on the display device.

5. The method according to claim 4, characterized in that, Recognizing the written characters includes: Based on the position information of the pen tip, calculate the offset distance of the pen tip in any two adjacent images of the writing tool; From the plurality of said offset distances, determine a target offset distance that is greater than a preset offset distance, and determine the two images corresponding to the target offset distance; For the two images corresponding to the target offset distance, character recognition is performed on the written characters under the pen tip in the previous image.

6. A stroke order recognition device, characterized in that, include: The acquisition unit is used to capture a sequence of images in the learning scenario in response to the rotation of the support device of the tablet computer at a preset angle, using a shooting device built into one end of the support device. A segmentation unit is used to segment the image sequence to obtain a hand image sequence and a writing tool image sequence; A registration unit is used to register the hand image sequence and the writing tool image sequence respectively to obtain the first motion displacement of the hand and the second motion displacement of the writing tool; The determining unit is used to calculate the displacement difference between the first motion displacement and the second motion displacement based on the first motion displacement and the second motion displacement; If the displacement difference is less than the preset displacement difference, the edge detection algorithm is used to perform edge detection on the writing tool image sequence to obtain the edge shape in each writing tool image. The edge shape includes the edge shape of part of the hand and the edge shape of the writing tool. Then the upper and lower endpoints of the edge shape are determined respectively. Since the pen tip is close to the paper surface, the position information of the lower endpoint is determined as the position information of the pen tip in any writing tool image. The stroke order of the written character is determined based on the position information of the pen tip in any image of a writing tool.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images