Text recognition method and device, computer readable storage medium and scanning pen

Abstract

The application belongs to the technical field of computer-aided education, and particularly relates to a text recognition method and device, a computer readable storage medium and a scanning pen. The method comprises the following steps: detecting a preset scanning preparation action; if the scanning preparation action is detected, collecting a first scanning image set through a camera of the scanning pen; detecting a preset scanning execution action; if the scanning execution action is detected, collecting a second scanning image set through the camera; splicing the first scanning image set and the second scanning image set to obtain a target scanning image; and performing text recognition on the target scanning image to obtain a text recognition result. In the application, image scanning can be performed in advance after the scanning preparation action is detected, and the images scanned in advance and the images scanned normally are spliced and then recognized, so that the missing scanning content caused by the delay and the too fast scanning speed can be effectively compensated, and complete scanning content can be obtained.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer-aided education, and particularly relates to a text recognition method and device, a computer readable storage medium, and a scanning pen. BACKGROUND

[0002] A scanning pen is a handheld electronic device. A user can hold the scanning pen to scan and input text or images, and can also perform operations such as text recognition, storage, and editing. The scanning pen is widely used in scenarios such as student learning, travel translation, and court recording.

[0003] In the prior art, a scanning pen is generally triggered by a user pressing the scanning pen. However, there is a delay from the time when the scanning pen is pressed to the time when the scanning pen responds. After the scanning pen responds, there is also a delay from the time when the scanning pen is turned on to the time when the camera and the fill light are turned on. The user often slides the scanning pen immediately after pressing the scanning pen to scan. When the scanning speed is too fast, the content scanned at the front may not be scanned in time, and complete scanning content cannot be obtained. SUMMARY

[0004] Therefore, embodiments of the present application provide a text recognition method, device, computer readable storage medium, and scanning pen to solve the problem that the prior art text recognition method causes missing scanning content due to a delay and a too-fast scanning speed.

[0005] A first aspect of embodiments of the present application provides a text recognition method applied to a scanning pen, which can include the following steps.

[0006] Detecting a preset scanning preparation action;

[0007] If the scanning preparation action is detected, a first scanning image set is collected by a camera of the scanning pen;

[0008] Detecting a preset scanning execution action;

[0009] If the scanning execution action is detected, a second scanning image set is collected by the camera;

[0010] Image splicing is performed on the first scanning image set and the second scanning image set to obtain a target scanning image;

[0011] Text recognition is performed on the target scanning image to obtain a text recognition result.

[0012] In a specific implementation manner of the first aspect, the detecting of the preset scanning preparation action can include the following steps.

[0013] The scanning preparation action is detected by a motion state sensor of the scanning pen;

[0014] or, detecting a trigger signal corresponding to the scan preparation action.

[0015] In an implementation form of the first aspect, the capturing, by the camera of the scanning pen, the first set of scan images can include:

[0016] measuring, by a distance sensor of the scanning pen, a distance between the scanning pen and the object to be scanned;

[0017] adjusting a focal length of the camera according to the distance;

[0018] capturing, by the camera with the adjusted focal length, a real-time scan image of the object to be scanned, and adding the real-time scan image into the first set of scan images.

[0019] In an implementation form of the first aspect, the image stitching of the first set of scan images and the second set of scan images to obtain the target scan image can include:

[0020] selecting, from the first set of scan images, a preset number of last-captured scan images as a third set of scan images;

[0021] removing, from the third set of scan images, scan images with a brightness less than a preset brightness threshold to obtain a fourth set of scan images;

[0022] image stitching of the fourth set of scan images and the second set of scan images to obtain the target scan image.

[0023] In an implementation form of the first aspect, the image stitching of the fourth set of scan images and the second set of scan images to obtain the target scan image can include:

[0024] extracting image feature points of each frame of scan image in the fourth set of scan images and the second set of scan images;

[0025] pairwise matching each frame of scan image according to the image feature points of the frame of scan image to obtain a matching confidence of each image pair;

[0026] selecting, from the each frame of scan image, each frame of matching scan image according to the matching confidence of each image pair;

[0027] image stitching of the each frame of matching scan image to obtain the target scan image.

[0028] In an implementation form of the first aspect, the image stitching of the each frame of matching scan image to obtain the target scan image can include:

[0029] calculate a rotation matrix of each of the matching scanned images respectively;

[0030] perform image transformation on the matching scanned images according to the rotation matrix of each of the matching scanned images to obtain a transformed image of each frame;

[0031] perform waveform correction on the transformed image of each frame to obtain a corrected image of each frame;

[0032] stitch the corrected image of each frame to obtain a stitched image;

[0033] perform multi-band fusion on the stitched image to obtain the target scanned image.

[0034] In a specific implementation manner of the first aspect, after performing text recognition on the target scanned image to obtain a text recognition result, the method can further include:

[0035] perform semantic completeness analysis on a first sentence in the text recognition result to determine whether the first sentence is a complete sentence;

[0036] if the first sentence is not a complete sentence, delete the first sentence from the text recognition result.

[0037] A second aspect of the embodiment of the application provides a text recognition device applied to a scanning pen, which can include:

[0038] a first detection module configured to detect a preset scanning preparation action;

[0039] a first scanning module configured to, if the scanning preparation action is detected, collect a first scanned image set by using a camera device of the scanning pen;

[0040] a second detection module configured to detect a preset scanning execution action;

[0041] a second scanning module configured to, if the scanning execution action is detected, collect a second scanned image set by using the camera device;

[0042] an image stitching module configured to stitch the first scanned image set and the second scanned image set to obtain a target scanned image;

[0043] a text recognition module configured to perform text recognition on the target scanned image to obtain a text recognition result.

[0044] In an implementation form of the second aspect, the first detecting module can be specifically configured to: detect the scan preparation action through a motion state sensor of the scan pen; or detect a trigger signal corresponding to the scan preparation action.

[0045] In an implementation form of the second aspect, the first scanning module can be specifically configured to: measure a distance between the scan pen and the object to be scanned through a distance sensor of the scan pen; adjust a focal length of the camera according to the distance; and collect a real-time scan image of the object to be scanned through the camera after the focal length is adjusted, and add the real-time scan image into the first scan image set.

[0046] In an implementation form of the second aspect, the image stitching module can include:

[0047] an image selecting sub-module configured to: select a preset number of last collected scan images from the first scan image set as a third scan image set; and remove a scan image with a brightness less than a preset brightness threshold from the third scan image set to obtain a fourth scan image set;

[0048] an image stitching sub-module configured to stitch the fourth scan image set and the second scan image set to obtain the target scan image.

[0049] In an implementation form of the second aspect, the image stitching sub-module can include:

[0050] a feature point extracting unit configured to extract image feature points of each frame of scan image in the fourth scan image set and the second scan image set;

[0051] a feature matching unit configured to pair each frame of scan image with another frame of scan image according to the image feature points of the each frame of scan image to obtain a matching confidence of each image pair;

[0052] a matching screening unit configured to screen each frame of matching scan image from the each frame of scan image according to the matching confidence of each image pair;

[0053] an image stitching unit configured to stitch the each frame of matching scan image to obtain the target scan image.

[0054] In an implementation form of the second aspect, the image stitching unit can include:

[0055] a rotation matrix calculating sub-unit configured to calculate a rotation matrix of the each frame of matching scan image respectively;

[0056] an image transformation subunit, configured to perform image transformation on the frames of the scanned images according to a rotation matrix of the frames of the matched scanned images, to obtain frames of transformed images;

[0057] a waveform correction subunit, configured to perform waveform correction on the frames of the transformed images, to obtain frames of corrected images;

[0058] an image splicing subunit, configured to splice the frames of the corrected images, to obtain a spliced image;

[0059] an image fusion subunit, configured to perform multi-band fusion on the spliced image, to obtain the target scanned image.

[0060] In an implementation manner of the second aspect, the text recognition apparatus can further include:

[0061] a semantic completeness analysis module, configured to perform semantic completeness analysis on a first sentence in the text recognition result, to determine whether the first sentence is a complete sentence; and if the first sentence is not a complete sentence, delete the first sentence from the text recognition result.

[0062] A third aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of any of the above text recognition methods.

[0063] A fourth aspect of the embodiments of the present application provides a scanning pen, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor, when executing the computer program, implements the steps of any of the above text recognition methods.

[0064] A fifth aspect of the embodiments of the present application provides a computer program product, which, when executed on a scanning pen, causes the scanning pen to perform the steps of any of the above text recognition methods.

[0065] The beneficial effects of the embodiments of the present application compared with the prior art are that the embodiments of the present application detect a preset scanning preparation action; if the scanning preparation action is detected, a first scanning image set is collected through a camera of a scanning pen; a preset scanning execution action is detected; if the scanning execution action is detected, a second scanning image set is collected through the camera; the first scanning image set and the second scanning image set are spliced to obtain a target scanning image; and the target scanning image is subjected to text recognition to obtain a text recognition result. In the embodiments of the present application, image scanning can be performed in advance after the scanning preparation action is detected, and the images scanned in advance and the images scanned normally (i.e. the images scanned after the scanning execution action is detected) are spliced and then recognized, so that the missing scanning content caused by the delay and the too fast scanning speed can be effectively compensated, and complete scanning content can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0067] Figure 1 A schematic diagram of a specific use scenario of a scanning pen;

[0068] Figure 2 An embodiment flowchart of a text recognition method in the embodiments of the present application;

[0069] Figure 3 A schematic flowchart of splicing the first scanning image set and the second scanning image set;

[0070] Figure 4 A schematic flowchart of splicing the matched scanning images;

[0071] Figure 5 An embodiment structure diagram of a text recognition device in the embodiments of the present application;

[0072] Figure 6 A schematic block diagram of a scanning pen in the embodiments of the present application. DETAILED DESCRIPTION

[0073] In order to make the objectives, characteristics, advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0074] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0075] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0076] It should be further understood that the term "and / or" as used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0077] As used in the present application specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]" depending on the context.

[0078] In addition, in the description of the present application, the terms "first", "second", "third" and the like are only used for differentiation in description and cannot be understood as indicating or implying relative importance.

[0079] The text recognition method provided by the embodiments of the present application can be applied to a scanning pen, Figure 1 As shown in the figure, a user can scan a to-be-scanned object by holding the scanning pen, where the to-be-scanned object can be a book, a page, or other text carriers.

[0080] In view of the problem that the existing text recognition method causes missing of scanned content due to delay and too fast scanning speed, the embodiment of the present application can perform early scanning, and recognize the image obtained by splicing the image scanned in advance and the image scanned normally, so that the missing of scanned content caused by delay and too fast scanning speed can be effectively compensated, and complete scanned content can be obtained.

[0081] Please refer to Figure 2 An embodiment of the text recognition method in the embodiment of the present application can include the following steps.

[0082] In step S201, a scanning preparation action is detected.

[0083] The scanning preparation action can be an action of holding the scanning pen and approaching the object to be scanned from the air. In the embodiment of the present application, the scanning preparation action can be detected by various action state sensors of the scanning pen.

[0084] For example, the acceleration of the scanning pen can be measured by the acceleration sensor of the scanning pen, and the scanning preparation action can be detected according to the acceleration. Specifically, if the acceleration is greater than a preset acceleration threshold value continuously within a preset detection time length, it is determined that the scanning preparation action is detected, otherwise, it is determined that the scanning preparation action is not detected. The specific values of the detection time length and the acceleration threshold value can be set according to actual conditions, and the embodiment of the present application does not make specific limitation thereto.

[0085] For another example, the distance between the scanning pen and the object to be scanned can be measured by the distance sensor of the scanning pen, and the scanning preparation action can be detected according to the distance. Specifically, if the distance continuously decreases within the detection time length, it is determined that the scanning preparation action is detected, otherwise, it is determined that the scanning preparation action is not detected.

[0086] For another example, the acceleration of the scanning pen can be measured by the acceleration sensor of the scanning pen, and the distance between the scanning pen and the object to be scanned can be measured by the distance sensor of the scanning pen, and the scanning preparation action can be detected according to the acceleration and the distance. Specifically, if the acceleration is greater than the acceleration threshold value continuously within the detection time length, and the distance continuously decreases, it is determined that the scanning preparation action is detected, otherwise, it is determined that the scanning preparation action is not detected.

[0087] In the embodiment of the present application, a trigger signal corresponding to the scanning preparation action can also be detected. For example, a corresponding switch or button can be preset in the scanning pen, and when the user opens the switch or clicks the button, the trigger signal is generated. If the scanning pen detects this trigger signal, it is determined that the scanning preparation action is detected, otherwise, it is determined that the scanning preparation action is not detected.

[0088] If the scan preparation action is not detected, step S201 is continuously performed, i.e., the scan preparation action is continuously detected, and if the scan preparation action is detected, step S202 is performed.

[0089] In step S202, a first scan image set is collected by the camera of the scanning pen.

[0090] The period from the detection of the scan preparation action to the detection of the scan execution action is a pre-scan period. During the pre-scan period, light is relatively sufficient and there is no obstruction, and image collection can be periodically performed by the camera of the scanning pen to obtain each frame of scan image in sequence. The set of these scan images is the first scan image set.

[0091] The camera in the embodiments of the present application can be a zoom camera, i.e., the focal length of the camera can be flexibly adjusted according to actual conditions. Taking an arbitrary image collection process in the pre-scan period as an example, first, the distance between the scanning pen and the object to be scanned can be measured by a distance sensor, and then the focal length of the camera is adjusted according to the distance to ensure that a clear image can be collected. Finally, the real-time scan image of the object to be scanned is collected by the camera with the adjusted focal length, and the real-time scan image is added to the first scan image set. In this way, the focal length of the camera can be adjusted accordingly as the distance between the scanning pen and the object to be scanned changes, so that each collected scan image has high clarity.

[0092] In step S203, a preset scan execution action is detected.

[0093] The scan execution action can be an action triggered by the user by pressing or other ways. In the embodiments of the present application, various sensors of the scanning pen can be used to detect the scan execution action. For example, the pressure sensor of the scanning pen can be used to measure the pressure received by the scanning pen, and the scan execution action can be detected according to the pressure. Specifically, if the pressure continuously exceeds a preset pressure threshold within a detection time period, it is determined that the scan execution action is detected, otherwise, it is determined that the scan execution action is not detected. The specific value of the pressure threshold can be set according to actual conditions, which is not limited in the embodiments of the present application.

[0094] If the scan execution action is not detected, step S203 is continuously performed, i.e., the scan execution action is continuously detected, and if the scan execution action is detected, step S204 is performed.

[0095] In step S204, a second scan image set is collected by the camera.

[0096] The period from the detection of the scanning action until the end of the scanning is the normal scanning period. During the normal scanning period, the LED fill light of the scanning pen can be turned on, and the scanning pen's camera device can periodically acquire images, obtaining each frame of scanned images in sequence. The set of these scanned images is the second scanned image set.

[0097] During normal scanning periods, the distance between the scanning pen and the object being scanned remains essentially constant, eliminating the need to adjust the focus of the camera device.

[0098] Step S205: Stitch the first scanned image set and the second scanned image set together to obtain the target scanned image.

[0099] In one specific implementation of this application, all scanned images in the first scanned image set and the second scanned image set can be stitched together to obtain the target scanned image.

[0100] Considering the limitations of the scanning pen's memory and computing resources, in one specific implementation of this application embodiment, a preset number of scanned images acquired last can be selected from the first scanned image set as the third scanned image set, and the third scanned image set and the second scanned image set can be stitched together to obtain the target scanned image.

[0101] The specific number of scanned images selected from the first scanned image set can be set according to the actual situation. For example, it can be set to 10 frames, 8 frames or other values. This application embodiment does not make a specific limitation on this.

[0102] Considering that the supplementary lighting is not yet turned on during the pre-scanning period, some scanned images may be captured that are too dark, affecting the subsequent image stitching effect. In one specific implementation of this application, scanned images with brightness lower than a preset brightness threshold can be removed from the third scanned image set to obtain a fourth scanned image set. The fourth scanned image set and the second scanned image set are then stitched together to obtain the target scanned image. The specific value of the brightness threshold can be set according to the actual situation, and this application does not impose a specific limitation on it.

[0103] like Figure 3 As shown, stitching together the fourth scanned image set and the second scanned image set can specifically include the following steps:

[0104] Step S2051: Extract image feature points from each frame of the scanned images in the fourth scanned image set and the second scanned image set.

[0105] In the embodiments of the present application, the extraction of the image feature points can be performed by any one of the feature extraction algorithms in the prior art, including but not limited to Scale Invariant Feature Transform (SIFT), Speeded Up Robust Feature (SURF), and ORB (Oriented FAST and Rotated BRIEF) and the like, and the embodiments of the present application do not make specific limitation in this regard.

[0106] In step S2052, the image pairs are matched in pairs according to the image feature points of each frame of the scanning images, and the matching confidence of each image pair is obtained.

[0107] The total number of the scanning images in the fourth scanning image set and the second scanning image set is denoted as N, and the matching in pairs is performed, and a total of image pairs can be obtained.

[0108] Taking any one of the image pairs as an example, two frames of images in the image pair are denoted as image A and image B, and for any one of the image feature points in image A (denoted as a target point), the first two image feature points with the closest Euclidean distance to the target point in image B can be found and denoted as a nearest neighbor point and a second nearest neighbor point, and then the ratio between the distance of the nearest neighbor point and the distance of the second nearest neighbor point is calculated, and if the ratio is less than a preset ratio threshold, it is determined that the target point and the nearest neighbor point are a pair of matching points, otherwise, it is determined that there is no image feature point in image B that matches the target point. The distance of the nearest neighbor point is the Euclidean distance between the target point and the nearest neighbor point, the distance of the second nearest neighbor point is the Euclidean distance between the target point and the second nearest neighbor point, and the specific value of the ratio threshold can be set according to actual conditions, and the embodiments of the present application do not make specific limitation in this regard. By traversing all the image feature points in image A, each pair of matching points in image A and image B can be found, and it is easy to understand that the more the number of the matching points, the higher the matching confidence of the image pair.

[0109] In step S2053, the matching scanning images are selected from the scanning images according to the matching confidence of each image pair.

[0110] Specifically, the image pairs with a matching confidence less than a preset confidence threshold can be first removed, and the specific value of the confidence threshold can be set according to actual conditions, and the embodiments of the present application do not make specific limitation in this regard.

[0111] Then, the remaining image pairs are processed by a union set, and images existing in association are merged into a set. For example, if the remaining image pairs are (image A, image B), (image B, image C), (image C, image D), (image D, image E), (image E, image F), and (image G, image H), the images existing in association are merged into a set by processing the union set, and two sets (image A, image B, image C, image D, image E, image F) and (image G, image H) are obtained, wherein the images in the largest set are the matched scanning images screened.

[0112] In step S2054, the matched scanning images are stitched to obtain a target scanning image.

[0113] As shown in FIG. 5, step S2054 can include the following processes. Figure 4

[0114] In step S20541, a rotation matrix of each matched scanning image is calculated.

[0115] In a specific implementation of the embodiment, camera parameter estimation can be performed on each matched scanning image first to obtain a rough rotation matrix, and then a precise rotation matrix is estimated by using a bundle adjustment method.

[0116] In step S20542, each scanning image is transformed according to the rotation matrix of each matched scanning image to obtain a transformed image.

[0117] For any matched scanning image, each coordinate point of the image can be converted by using the rotation matrix to obtain a corresponding transformed image.

[0118] In step S20543, each transformed image is corrected in waveform to obtain a corrected image.

[0119] Since each transformed image is not necessarily horizontal or vertical, slight tilt can cause an airplane curve in the finally obtained stitched image. Therefore, the embodiment can correct each transformed image in waveform in a horizontal or vertical direction to obtain a corrected image.

[0120] In step S20544, each corrected image is stitched to obtain a stitched image.

[0121] After the image transformation and the waveform correction, each corrected image is transformed to the same coordinate system, and each corrected image is stitched according to the position coordinates of the image in the coordinate system to obtain a complete stitched image.

[0122] ​Step S20545, multi-band fusion is performed on the spliced image to obtain a target scanning image.

[0123] When performing multi-band fusion, the spliced image can be decomposed into superposition of images of different frequencies, and different weights are used for fusion at different frequencies. Specifically, a wide wavelength weighted signal can be used in the low frequency part, and a narrow weighted signal can be used in the high frequency part, and finally the images of different frequencies are weighted and superimposed to obtain the target scanning image.

[0124] Step S206, text recognition is performed on the target scanning image to obtain a text recognition result.

[0125] In the embodiments of the present application, the Optical Character Recognition (OCR) technology can be used to perform text recognition on the target scanning image to obtain the text recognition result.

[0126] Optionally, in order to avoid scanning an incomplete sentence in the early scanning process, semantic completeness analysis can also be performed on the first sentence in the text recognition result to determine whether the first sentence is a complete sentence. If the first sentence is not a complete sentence, the first sentence is deleted from the text recognition result; if the first sentence is a complete sentence, the original text recognition result is maintained.

[0127] After obtaining the text recognition result, the text recognition result can also be displayed on the interactive interface of the scanning pen, so that the user can view the complete scanning content in time.

[0128] In summary, the embodiments of the present application detect a preset scanning preparation action; if the scanning preparation action is detected, a first scanning image set is collected through the camera device of the scanning pen; a preset scanning execution action is detected; if the scanning execution action is detected, a second scanning image set is collected through the camera device; the first scanning image set and the second scanning image set are spliced to obtain a target scanning image; text recognition is performed on the target scanning image to obtain a text recognition result. In the embodiments of the present application, image scanning can be performed in advance after the scanning preparation action is detected, and the images scanned in advance and the images scanned normally (i.e. the images scanned after the scanning execution action is detected) are spliced and recognized, so that the missing scanning content caused by delay and too fast scanning speed can be effectively compensated, and complete scanning content can be obtained.

[0129] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0130] A text recognition method, Figure 5 An embodiment structure diagram of a text recognition device is shown.

[0131] The text recognition device can be applied to a scanning pen and can specifically include:

[0132] The first detection module 501 is configured to detect a preset scanning preparation action.

[0133] The first scanning module 502 is configured to, if the scanning preparation action is detected, collect a first scanning image set by using a camera of the scanning pen.

[0134] The second detection module 503 is configured to detect a preset scanning execution action.

[0135] The second scanning module 504 is configured to, if the scanning execution action is detected, collect a second scanning image set by using the camera.

[0136] The image splicing module 505 is configured to splice the first scanning image set and the second scanning image set to obtain a target scanning image.

[0137] The text recognition module 506 is configured to perform text recognition on the target scanning image to obtain a text recognition result.

[0138] In a specific implementation manner of the embodiment of the present application, the first detection module can be specifically configured to detect the scanning preparation action by using an action state sensor of the scanning pen, or detect a trigger signal corresponding to the scanning preparation action.

[0139] In a specific implementation manner of the embodiment of the present application, the first scanning module can be specifically configured to measure a distance between the scanning pen and a scanning object by using a distance sensor of the scanning pen, adjust a focal length of the camera according to the distance, and collect a real-time scanning image of the scanning object by using the camera with the adjusted focal length, and add the real-time scanning image into the first scanning image set.

[0140] In a specific implementation manner of the embodiment of the present application, the image splicing module can include:

[0141] The image selection sub-module is configured to select a preset number of last collected scanning images from the first scanning image set as a third scanning image set, and remove a scanning image with a brightness less than a preset brightness threshold from the third scanning image set to obtain a fourth scanning image set.

[0142] The image stitching sub-module is configured to stitch the fourth set of scanning images and the second set of scanning images to obtain the target scanning image.

[0143] In an implementation of the embodiment of the present application, the image stitching sub-module can include:

[0144] The feature point extraction unit is configured to extract image feature points of each frame of scanning image in the fourth set of scanning images and the second set of scanning images.

[0145] The feature matching unit is configured to perform pairwise matching on the each frame of scanning image according to the image feature points of the each frame of scanning image to obtain matching confidence of each image pair.

[0146] The matching screening unit is configured to screen each frame of matching scanning image from the each frame of scanning image according to the matching confidence of each image pair.

[0147] The image stitching unit is configured to stitch the each frame of matching scanning image to obtain the target scanning image.

[0148] In an implementation of the embodiment of the present application, the image stitching unit can include:

[0149] The rotation matrix calculation sub-unit is configured to calculate a rotation matrix of the each frame of matching scanning image respectively.

[0150] The image transformation sub-unit is configured to perform image transformation on the each frame of scanning image according to the rotation matrix of the each frame of matching scanning image to obtain each frame of transformed image.

[0151] The waveform correction sub-unit is configured to perform waveform correction on the each frame of transformed image to obtain each frame of corrected image.

[0152] The image stitching sub-unit is configured to stitch the each frame of corrected image to obtain a stitched image.

[0153] The image fusion sub-unit is configured to perform multi-band fusion on the stitched image to obtain the target scanning image.

[0154] In an implementation of the embodiment of the present application, the text recognition device can further include:

[0155] The semantic completeness analysis module is configured to perform semantic completeness analysis on a first sentence in the text recognition result to determine whether the first sentence is a complete sentence; and if the first sentence is not a complete sentence, delete the first sentence from the text recognition result.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the apparatuses, modules and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0157] In the foregoing embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can refer to the related description of other embodiments.

[0158] Figure 6 A schematic block diagram of a scanning pen is shown, and only parts related to the embodiments of the present application are shown for the convenience of description.

[0159] As Figure 6 shown, the scanning pen 6 of this embodiment includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. The processor 60 implements the steps in each of the text recognition method embodiments described above when executing the computer program 62, for example Figure 2 steps S201 to S206 shown. Alternatively, the processor 60 implements the functions of each module / unit in each of the apparatus embodiments described above when executing the computer program 62, for example Figure 5 the functions of the modules 501 to 506 shown.

[0160] For example, the computer program 62 can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 62 in the scanning pen 6.

[0161] Those skilled in the art can understand, Figure 6 is only an example of the scanning pen 6 and does not constitute a limitation on the scanning pen 6, and can include more or fewer components than shown, or combine certain components, or different components, for example, the scanning pen 6 can also include an input / output device, a network access device, a bus, etc.

[0162] The processor 60 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0163] The memory 61 can be an internal storage unit of the scanning pen 6, such as a hard disk or a memory of the scanning pen 6. The memory 61 can also be an external storage device of the scanning pen 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 61 can also include both the internal storage unit and the external storage device of the scanning pen 6. The memory 61 is used to store the computer program and other programs and data required by the scanning pen 6. The memory 61 can also be used to temporarily store data that has been output or is to be output.

[0164] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0165] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0166] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0167] In the embodiments provided in the present application, it should be understood that the disclosed device / scanning pen and method can be implemented in other ways. For example, the device / scanning pen embodiments described above are merely schematic, for example, the division of the modules or units is merely a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0168] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0169] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0170] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be implemented by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the contents included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0171] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A text recognition method, applied in a scanning pen, characterized in that, include: The preset scan preparation actions are checked; If the scanning preparation action is detected, a first set of scan images is acquired through the camera device of the scanning pen; The preset scanning actions are detected; If the scanning action is detected, a second set of scanned images is acquired through the camera device; The first scanned image set and the second scanned image set are stitched together to obtain the target scanned image; Text recognition is performed on the target scan image to obtain the text recognition result.

2. The text recognition method according to claim 1, characterized in that, The detection of the preset scan preparation actions includes: The scanning preparation action is detected by the motion status sensor of the scanning pen; Alternatively, the trigger signal corresponding to the scan preparation action can be detected.

3. The text recognition method according to claim 1, characterized in that, The acquisition of the first set of scanned images by the camera device of the scanning pen includes: The distance between the scanning pen and the object to be scanned is measured by the distance sensor of the scanning pen; The focal length of the camera device is adjusted according to the distance; The camera device, with its focus adjusted, acquires real-time scan images of the object to be scanned and adds these real-time scan images to the first set of scan images.

4. The text recognition method according to claim 1, characterized in that, The step of stitching together the first scanned image set and the second scanned image set to obtain the target scanned image includes: Select a preset number of scanned images from the first scanned image set as the third scanned image set; The fourth set of scanned images is obtained by removing scanned images whose brightness is less than a preset brightness threshold from the third set of scanned images. The target scan image is obtained by stitching together the fourth scan image set and the second scan image set.

5. The text recognition method according to claim 4, characterized in that, The step of stitching the fourth scanned image set and the second scanned image set together to obtain the target scanned image includes: Extract image feature points from each frame of the scanned images in the fourth scanned image set and the second scanned image set; Based on the image feature points of each scanned image, the scanned images of each frame are matched in pairs to obtain the matching confidence of each image pair; Based on the matching confidence of each image pair, the matching scan images of each frame are selected from the scan images of each frame; The target scan image is obtained by stitching together the matched scan images from each frame.

6. The text recognition method according to claim 5, characterized in that, The step of stitching together the matched scanned images of each frame to obtain the target scanned image includes: Calculate the rotation matrix of each frame of the matched scan image; The image transformation of each frame of the scanned image is performed based on the rotation matrix of each frame of the matched scanned image to obtain the transformed image of each frame; The waveform of each frame of the transformed image is corrected to obtain the corrected image of each frame. The corrected images of each frame are stitched together to obtain a stitched image; The stitched image is fused using multi-band fusion to obtain the target scan image.

7. The text recognition method according to any one of claims 1 to 6, characterized in that, After performing text recognition on the target scanned image and obtaining the text recognition result, the process further includes: Perform semantic completeness analysis on the first sentence in the text recognition result to determine whether the first sentence is a complete sentence; If the first statement is not a complete statement, then the first statement is deleted from the text recognition result.

8. A text recognition device, used in a scanning pen, characterized in that, include: The first detection module is used to detect the preset scanning preparation actions; The first scanning module is used to acquire a first set of scan images through the camera device of the scanning pen if the scanning preparation action is detected. The second detection module is used to detect preset scanning execution actions; The second scanning module is used to acquire a second set of scan images through the camera device if the scanning execution action is detected. The image stitching module is used to stitch the first scanned image set and the second scanned image set together to obtain the target scanned image; The text recognition module is used to perform text recognition on the target scanned image and obtain the text recognition result.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the text recognition method as described in any one of claims 1 to 7.

10. A scanning pen, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the text recognition method as described in any one of claims 1 to 7.

Images