Interactive Method, Device, Computer Equipment and Storage Medium for Text Scanning

Abstract

The present application relates to an interaction method, device, computer device, and storage medium for text scanning. The method relates to text recognition technology of artificial intelligence, and the method includes: in response to a trigger operation for text scanning, displaying a scanning screen including at least one text object; in the scanning screen, displaying continuous drawing of a stroke along the edge of the text object; when the edge is blocked, displaying the end of the stroke drawing of the text object and the drawn stroke; when the edge is not blocked, displaying continuous drawing of the stroke along the edge of the text object until a closed stroke is formed, ending the drawing and displaying the closed stroke; in response to a trigger operation for text recognition, displaying the text recognition result corresponding to the text object. Using this method can improve the accuracy and efficiency of text scanning and text recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and particularly to an interaction method, device, computer device, and storage medium for text scanning. Background Art

[0002] In recent years, users usually import the data of offline documents or images into computer devices for storage, which can not only avoid the loss of documents or data but also facilitate users to view them. For example, in an office scenario, users can manually enter the information of paper business cards provided by customers or colleagues into intelligent devices. Also, for example, users obtain the translated content after translating the text in the picture by manual entry. Obviously, the above manual entry methods are time-consuming and laborious.

[0003] With the rapid development of Internet technology and artificial intelligence, text recognition technology has emerged, which can help users automatically scan and recognize the text information in text objects, such as character recognition, text translation, text scanning, etc. Scanning the text object, as the first step of text recognition, determines the accuracy of text recognition to a certain extent.

[0004] In daily life, currently when scanning a text object, if the text object is blocked, it may cause text recognition to fail, and users will try text scanning multiple times, resulting in low efficiency and accuracy of text recognition. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide an interaction method, device, computer device, and storage medium for text scanning that can improve the efficiency and accuracy of text recognition.

[0006] An interaction method for text scanning includes:

[0007] In response to a trigger operation for text scanning, display a scanning screen including at least one text object;

[0008] In the scanning screen, display continuous drawing of a stroke along the edge of the text object;

[0009] When the edge is blocked, display the end of the stroke drawing for the text object and the drawn stroke;

[0010] When the edge is not blocked, display continuous drawing of a stroke along the edge of the text object until a closed stroke is formed, then end the drawing and display the closed stroke;

[0011] In response to a trigger operation for text recognition, display the text recognition result corresponding to the text object.

[0012] An interaction method for text scanning, the method includes:

[0013] In response to a trigger operation for text scanning, display a scanning screen including a plurality of text objects;

[0014] For each text object in the scanning screen, in the scanning screen, display each stroke continuously drawn along the edge of each text object;

[0015] For each text object in the scanning screen, when the edge of the text object is occluded, display the end of the stroke drawing for the text object and the drawn strokes, and when the edge is not occluded, display continuing to continuously draw the stroke along the edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke;

[0016] In response to a trigger operation for text recognition, display the text recognition result corresponding to at least one text object among the plurality of text objects.

[0017] An interactive device for text scanning, the device includes:

[0018] A scanning screen display module, configured to, in response to a trigger operation for text scanning, display a scanning screen including at least one text object;

[0019] A stroke display module, configured to, in the scanning screen, display continuously drawing a stroke along the edge of the text object;

[0020] The stroke display module is further configured to, when the edge is occluded, display the end of the stroke drawing for the text object and the drawn strokes; when the edge is not occluded, display continuing to continuously draw the stroke along the edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke;

[0021] A text recognition result display module, configured to, in response to a trigger operation for text recognition, display the text recognition result corresponding to the text object.

[0022] In one embodiment, the stroke display module is further configured to, in the scanning screen, start from any pixel point on the unoccluded edge of the text object, display continuously drawing a stroke along the pixel points on the unoccluded edge of the text object, and end the drawing until all the pixel points on the unoccluded edge are traversed, and display the drawn stroke along the unoccluded edge.

[0023] In one embodiment, the stroke display module is further configured to, in the scanning screen, display continuously drawing a stroke along the edge of the text object in a first preset color, and when the edge is occluded, end the stroke drawing for the text object and blink and display the drawn stroke.

[0024] In one embodiment, the stroke display module is further configured to, in the scanned image, when continuously drawing a stroke along the edge of the text object, make the stroke follow the text object in real time as the text object is offset in real time in the scanned image.

[0025] In one embodiment, the stroke display module is further configured to, in the scanned image, start continuously drawing a stroke from any pixel point on the complete edge of the text object; when the edge is not blocked, continue to display continuously drawing a stroke along the pixel points on the complete edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed.

[0026] In one embodiment, for a text object that is completely displayed in the scanned image, the stroke display module is further configured to, in the scanned image, display a stroke continuously drawn along the clockwise or counterclockwise direction of the complete edge of the text object in a second preset color.

[0027] In one embodiment, the stroke display module is further configured to, in the scanned image, display a stroke continuously drawn along the clockwise or counterclockwise direction of the complete edge of the text object in a second preset color; when the edge is not blocked, continue to display continuously drawing a stroke along the pixel points on the complete edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed.

[0028] In one embodiment, the device further includes:

[0029] A prompt information display module, configured to, when the edge is blocked, after ending the drawing of the stroke for the text object and displaying the drawn stroke, display a prompt information indicating that the scanning of the text object fails;

[0030] The prompt information display module is further configured to, when the edge is not blocked, after displaying continuing to continuously draw a stroke along the edge of the text object until a closed stroke is formed and then displaying the closed stroke, display a prompt information indicating that the scanning of the text object is successful.

[0031] In one embodiment, the text recognition result display module is further configured to, in response to a trigger operation for text translation, display the currently acquired image of the scanned image; in the acquired image, display the translation content corresponding to the text content in the text object.

[0032] In one embodiment, the device further includes a prompt information display module, configured to, when a graphic code is included in the text object in the scanned image, display a prompt information indicating that the scanning of the graphic code is successful in the scanned image;

[0033] The text recognition result display module is further configured to display an access interface corresponding to an embedded link of the graphic code in the text object.

[0034] In one embodiment, the text object is a card or certificate, and the scanned image display module is further configured to display a card or certificate information entry interface; in response to a card or certificate scanning trigger operation in the card or certificate information entry interface, enter a card or certificate scanning interface from the card or certificate information entry interface; in the card or certificate scanning interface, display a scanned image including at least one card or certificate.

[0035] In one embodiment, the text recognition result display module is further configured to, in response to a text recognition trigger operation in the card or certificate scanning interface, return from the card or certificate scanning interface to the card or certificate information entry interface; in the card or certificate information entry interface, display the text recognition entry information corresponding to the card or certificate.

[0036] In one embodiment, the device further includes a rectangle detection module, including:

[0037] A preprocessing unit, configured to preprocess an image collected from the scanned image to obtain a preprocessed image;

[0038] An edge pixel recognition unit, configured to determine edge pixel points in the preprocessed image;

[0039] A rectangle recognition unit, configured to, after classifying the edge pixel points, obtain at least one set of edge points and recognize the shape corresponding to each set of edge points; when the recognized shape is a closed rectangle, determine that the edge of the text object corresponding to the set of edge points is not blocked in the scanned image, and when the recognized shape is not a closed rectangle, determine that the edge of the text object corresponding to the set of edge points is blocked in the scanned image.

[0040] In one embodiment, the preprocessing unit is further configured to perform denoising processing and contrast enhancement processing on the collected image to obtain a preprocessed image;

[0041] The edge pixel recognition unit is further configured to calculate the gradient and direction corresponding to each pixel point in the preprocessed image, and determine edge pixel points according to the gradient and direction.

[0042] An interactive device for text scanning, the device includes:

[0043] A scanned image display module, configured to display a scanned image including a plurality of text objects in response to a text scanning trigger operation;

[0044] A stroke display module, configured to, for each text object in the scanned image, display, in the scanned image, each stroke that is continuously drawn along the edge of each text object;

[0045] The stroke display module is further configured to, for each text object in the scanned image, when the edge of the text object is occluded, display the end of the stroke drawing for the text object and the drawn strokes, and when the edge is not occluded, display continuing to continuously draw the stroke along the edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke;

[0046] A text recognition result display module, configured to, in response to a trigger operation for text recognition, display the text recognition result corresponding to at least one text object among the multiple text objects.

[0047] A computer device, comprising a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0048] In response to a trigger operation for text scanning, display a scanned image including at least one text object;

[0049] In the scanned image, display continuously drawing a stroke along the edge of the text object;

[0050] When the edge is occluded, display the end of the stroke drawing for the text object and the drawn strokes;

[0051] When the edge is not occluded, display continuing to continuously draw the stroke along the edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke;

[0052] In response to a trigger operation for text recognition, display the text recognition result corresponding to the text object.

[0053] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0054] In response to a trigger operation for text scanning, display a scanned image including at least one text object;

[0055] In the scanned image, display continuously drawing a stroke along the edge of the text object;

[0056] When the edge is occluded, display the end of the stroke drawing for the text object and the drawn strokes;

[0057] When the edge is not blocked, continue to draw a continuous stroke along the edge of the text object until a closed stroke is formed, then end the drawing and display the closed stroke;

[0058] In response to a trigger operation for text recognition, display the text recognition result corresponding to the text object.

[0059] A computer program, the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the above text scanning interaction method.

[0060] The above text scanning interaction method, device, computer device and storage medium, as the first step of text recognition, during the process of scanning a text object, dynamically and continuously draw a stroke along the edge of the text object in the scanning screen, and display the edge of the text object to the user in the form of a stroke animation, which can give the user correct and targeted feedback information during the scanning process, display the state of the current text object in the scanning screen, and guide the user to make adjustments to the text object in the scanning screen, thereby improving the efficiency and accuracy of text scanning and text recognition.

[0061] Specifically, for a text object in the scanning screen, during the process of drawing a stroke along its edge, when the edge is blocked, end the drawing and display the drawn stroke. This drawing process and the drawn stroke can guide the user to adjust the blocked text object so that it is completely displayed in the scanning screen, reducing the situation of failure or low accuracy after multiple attempts to perform text recognition on the text object; when the edge is not blocked, continuing to draw a stroke along its edge can convey to the user a signal that the text object has been successfully scanned and the next step of text recognition can be performed. Thus, in response to a trigger operation for text recognition, display the text recognition result corresponding to the text object. The entire scanning process avoids the user's multiple attempts of ineffective text scanning, improving the accuracy and efficiency of text scanning and text recognition. Description of the Drawings

[0062] Figure 1 It is an application environment diagram of the text scanning interaction method in an embodiment;

[0063] Figure 2 It is a flowchart of the text scanning interaction method in an embodiment;

[0064] Figure 3 It is a schematic diagram of a scanning screen in an embodiment;

[0065] Figure 4Schematic diagram of a stroke drawn for a text object shown as partially occluded in an embodiment;

[0066] Figure 5 Schematic diagram of a stroke drawn for a text object shown as fully displayed in an embodiment;

[0067] Figure 6 Schematic diagram of strokes drawn for multiple text objects shown in a scanning screen in an embodiment;

[0068] Figure 7 Schematic diagram of a stroke drawn for a text object shown as partially occluded in another embodiment;

[0069] Figure 8 Schematic diagram of a stroke drawn for a text object shown as fully displayed in another embodiment;

[0070] Figure 9 Schematic diagram of the change of a scanning screen in an embodiment;

[0071] Figure 10 Schematic diagram of the change of a scanning screen in another embodiment;

[0072] Figure 11 Schematic diagram of the user interaction interface where the scanning screen is located in an embodiment;

[0073] Figure 12 Schematic diagram of the interface for starting card scanning in an embodiment;

[0074] Figure 13 Schematic diagram of the card information entry interface and the card scanning interface in an embodiment;

[0075] Figure 14 Sequence diagram of the interaction method for text scanning in an embodiment;

[0076] Figure 15 Structural block diagram of the interaction device for text scanning in an embodiment;

[0077] Figure 16 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0078] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0079] The interaction method for text scanning provided by the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. In one embodiment, in response to a trigger operation for text scanning, the terminal 102 displays a scanning screen including at least one text object; in the scanning screen, continuous drawing strokes are displayed along the edge of the text object; when the edge is blocked, the end of drawing the strokes of the text object and the drawn strokes are displayed; when the edge is not blocked, continuous drawing strokes are displayed along the edge of the text object until a closed stroke is formed and the drawing ends and the closed stroke is displayed; in response to a trigger operation for text recognition, the text recognition result corresponding to the text object is displayed; in response to a trigger operation for text recognition, the text recognition result corresponding to the text object is displayed. Among them, the edge of the text object is detected by the server 104 or can also be detected by the terminal 102. The terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0080] In one embodiment, an online document application is run on the terminal 102, and the online document application executes the above-mentioned text scanning interaction method to implement the function of text recognition. In one embodiment, a social application, such as an instant messaging application, is run on the terminal 102, and the social application run on the terminal 102 can execute the above-mentioned text scanning interaction method.

[0081] The execution subject of the text scanning interaction method provided by the embodiments of the present application can be the text scanning interaction device provided by the embodiments of the present application, or a computer device integrated with the text scanning interaction device, where the text scanning interaction device can be implemented in a hardware or software manner. The computer device can be Figure 1 the terminal 102 or the server 104 shown in

[0082] The text scanning interaction method provided by the embodiments of the present application relates to artificial intelligence (AI) technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0083] Artificial intelligence technology is an interdisciplinary subject that involves a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0084] The interactive method for text scanning provided by the embodiments of this application relates to the computer vision technology (Computer Vision, CV) of artificial intelligence. Computer vision is a science that studies how to enable machines to "see". Further speaking, it refers to using cameras and computers to replace human eyes to identify and measure targets, etc., which is machine vision, and further performing graphic processing to make the computer process into images that are more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to establish artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc., and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0085] Text recognition is one of the branches in the research field of computer vision, commonly known as optical character recognition, with the full English name of OCR. OCR uses optical technology and computer technology to read the text printed or written on paper and convert it into a format that can be accepted by a computer and understood by people. OCR technology is a key technology for realizing high-speed text entry.

[0086] The interactive method for text scanning provided by the embodiments of this application relates to the text recognition technology in the field of computer vision technology. For example, during the text scanning process, the terminal 102 can acquire the captured image, perform text recognition on the text object in the captured image to obtain the text content, and display the text content. The terminal 102 can also further translate the obtained text content to obtain the translation content and then display the translation content. The terminal 102 can also upload the captured image to the server 104. After the server 104 performs text recognition on the captured image to obtain the text content, it returns the text content to the terminal 102 for display. After the server 104 obtains the translation content corresponding to the text content, it returns the translation content to the terminal 102 for display.

[0087] The text recognition generally includes the following steps: input image, image preprocessing, image layout processing, image segmentation, feature extraction, model training, and post-recognition processing. In recent years, the efficiency and accuracy of the latter several steps have been improved in various ways, but the importance of image input has been ignored. As the first step of text recognition, image input is the prerequisite and key for subsequent processing steps. In the application scenario where users perform text recognition through intelligent terminals, the input image is usually obtained by scanning text objects. However, in the common scenario, during the scanning process, the text objects in the scanning screen are usually partially blocked. For example, the camera is too close to the text object, or there is an overlapping relationship between multiple text objects, resulting in occlusion of the text object, making it impossible to perform accurate text recognition. Users have to try text scanning and text recognition again and again, resulting in low efficiency of text recognition.

[0088] Therefore, the embodiment of the present application provides an interactive method for text scanning. As the first step of text recognition, during the process of scanning a text object, a stroke is dynamically and continuously drawn along the edge of the text object in the scanning screen, and the edge of the text object is displayed to the user in the form of a stroke animation, which can give the user correct and targeted feedback information during the scanning process, display the state of the current text object in the scanning screen, and guide the user to adjust the text object in the scanning screen, thereby improving the efficiency and accuracy of text scanning and text recognition.

[0089] In one embodiment, as Figure 2 shown, an interactive method for text scanning is provided. Taking the method applied to the Figure 1 terminal 102 as an example for description, it includes the following steps:

[0090] Step 202, in response to a trigger operation for text scanning, display a scanning screen including at least one text object.

[0091] Among them, text scanning is the process of scanning a text object for obtaining an acquisition image to be subjected to text recognition. The trigger operation for text scanning is a trigger operation to start text scanning, and the trigger operation for text scanning can be a single-click operation, a double-click operation, or a sliding operation triggered by the user.

[0092] The text object is an entity object to be subjected to text recognition. In order to perform stroke drawing on the text object, the text object can be an entity object with a regular shape including text content, including but not limited to a rectangular text object, a circular text object, or an object with other shapes capable of performing stroke drawing on it, such as a document, a certificate, a business card, etc. Of course, the regular shape here is not used to limit the entity object itself. The regular shape means that the two-dimensional plane style presented by the entity object in the scanned image is a regular shape. For example, when it is necessary to recognize the text content on the cover of a book, the user scans the cover of the book, and the four sides of the book cover are straight lines that can be drawn with strokes in the scanned image.

[0093] The scanned image is a real-time image including at least one text object. The scanned image can include one text object, or can include two or more text objects. There may be an overlapping or covering relationship between the multiple text objects. In one embodiment, the terminal can respond to a trigger operation for text scanning, start the camera for image acquisition, and display a scanned image including at least one text object according to the real-time acquisition image of the camera. As Figure 3 shown, it is a schematic diagram of a scanned image in an embodiment. Referring to Figure 3 , in this scanned image 300, it includes a text object 302 that is fully displayed and a text object 304 that is partially blocked.

[0094] Step 204, in the scanned image, display continuous stroke drawing along the edge of the text object.

[0095] In order to be able to give the user correct and targeted feedback information during the scanning process, by dynamically and continuously drawing strokes along the edge of each text object in the scanned image, and presenting the edge of the text object to the user in the form of a stroke animation, it can show the state of the current text object in the scanned image and guide the user to make adjustments to the text object in the scanned image.

[0096] Step 206, when the edge is blocked, display the end of stroke drawing on the text object and the drawn strokes.

[0097] In the embodiment of the present application, that the edge of the text object is blocked means that a part of the edge of the text object is blocked, which is called partial occlusion. Specifically, when continuously drawing strokes along the edge of the text object, when drawing to the blocked edge of the text object, the stroke drawing on the text object is ended and the drawn strokes are displayed. That is, when the text object in the scanned image is partially blocked, in the scanned image, display the continuous strokes drawn along at least a part of the unblocked edge of the text object.

[0098] Among them, the edges of the text object that are partially occluded are locally occluded, and only the local edges of the text object, that is, the unoccluded edges, can be displayed in the scanned image. In this case, if the text object is directly recognized, there may be a situation of recognition failure or low recognition accuracy. For this reason, by dynamically and continuously drawing a stroke along the edge of the text object that is partially occluded in the scanned image, and presenting the edge of the text object that is partially occluded to the user in the form of a stroke animation, it is possible to give the user correct and targeted feedback information during the scanning process, display the state of the text object that is currently partially occluded in the scanned image, guide the user to make adjustments to the text object that is partially occluded in the scanned image, thereby improving the efficiency and accuracy of text scanning and text recognition.

[0099] The stroke is a kind of dynamic and continuous prompt information, which can also be called feedback information. It is a dynamic feedback information that is drawn and displayed along the edge of the text object. The stroke can guide the user to make adjustments to the text object that is partially occluded during the scanning process, so that the adjusted text object can be completely collected, avoiding the user's multiple attempts at ineffective text recognition, thereby improving the accuracy and efficiency of text recognition.

[0100] In one embodiment, after the terminal starts text scanning, it obtains the captured image in real time. The captured image includes a text object that is partially occluded. The terminal can recognize the unoccluded edges of the text object and display a stroke animation drawn along the unoccluded edges. Optionally, for the text object that is partially occluded, after locally drawing a stroke along its unoccluded edges, the completed local stroke can continue to be displayed in the scanned image. Of course, after the drawing is completed, the completed local stroke can also no longer be displayed in the scanned image.

[0101] In one embodiment, the terminal can display a stroke drawn along all the unoccluded edges of the text object that is partially occluded and display the completed stroke. In another embodiment, the terminal can display a stroke drawn along a part of the unoccluded edges of the text object that is partially occluded.

[0102] It should be noted that the stroke drawn along the edge of the text object is not limited to being drawn exactly along the edge of the text object. The edge of the text object can be a reference information during the drawing process, which can roughly reflect the position of the text object in the scanned image. Therefore, a slight deviation or error between the drawn stroke and the edge of the text object also falls within the protection scope of the text scanning interaction method provided by the embodiments of the present application.

[0103] As Figure 4 shown, it is a schematic diagram of drawing a stroke for a text object that is shown as being partially occluded in the scanned image in one embodiment. Referring to Figure 4, in this scanning screen 40, it includes a text object 402 that is partially occluded. The text object 402 is partially occluded by an occluder. For the text object 402 that is partially occluded, starting from point A on the unoccluded edge, it is gradually drawn to point B, then to point C, and finally to point D, completing the drawing of the stroke for the partially occluded text object.

[0104] Step 208, when the edge is not occluded, continue to display continuous drawing of the stroke along the edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke.

[0105] In the embodiment of the present application, that the edge of the text object is not occluded means that the edge of the text object is completely shown in the scanning screen. Specifically, during the process of continuously drawing the stroke along the edge of the text object, when there is no occluded situation for the edge of the text object during the drawing process, continue to continuously draw the stroke along the edge of the text object until the drawn stroke forms a closed shape, that is, until a closed stroke is formed to end the drawing. That is, when the text object in the scanning screen is completely shown, in the scanning screen, display the stroke continuously drawn along the complete edge of the text object.

[0106] Among them, the edge of the text object that is completely shown will also be completely shown. The complete edge of the text object is shown in the scanning screen. In this case, the stroke drawn along its complete edge can convey a signal to the user that the text object has been successfully scanned, guiding the user to perform the next text recognition, and thus being able to respond to the trigger operation of text recognition and display the text recognition result corresponding to the text object, improving the accuracy and efficiency of text scanning and text recognition.

[0107] In one embodiment, after the terminal starts text scanning, it obtains the captured image in real time. The captured image includes a text object that is completely shown. The terminal can recognize the complete edge of the text object and display the stroke animation drawn along the complete edge. Optionally, after the stroke drawn along the complete edge of the text object that is completely shown ends, the completed closed stroke can continue to be continuously displayed in the scanning screen. Of course, after the drawing ends, the completed closed stroke can also no longer be shown in this scanning screen.

[0108] As Figure 5 shown, it is a schematic diagram of drawing a stroke for a text object shown as completely shown in the scanning screen in one embodiment. Referring to Figure 5 , in this scanning screen 50, it includes a text object 502 that is completely shown. For the text object 502 that is completely shown, starting from point A on the complete edge, it is gradually drawn to point B, then to point C, and finally to point A, completing the drawing of the stroke for the text object that is completely shown.

[0109] As shown Figure 6 in the figure, it is a schematic diagram of drawing strokes for multiple text objects displayed in a scanning screen in an embodiment. Referring to Figure 6 , in the scanning screen 60, it includes a first text object 602 that is completely displayed and a second text object 604 that is partially blocked by the first text object 602. For the first text object 602, a stroke 602a that is gradually and continuously drawn along its complete edge is displayed. For the second text object 604, a stroke 604a that is gradually and continuously drawn along its partial edge is displayed. Moreover, the stroke 602a and the stroke 604a are drawn simultaneously.

[0110] Step 210, in response to a trigger operation for text recognition, display the text recognition result corresponding to the text object.

[0111] As mentioned above, text scanning is used to obtain an input image to be subjected to text recognition. After the strokes of the text object are displayed in the scanning screen, the scanning of the text object is completed, and the input image is obtained, and the next step of text recognition can be performed. The trigger operation for text recognition is a trigger operation to start text recognition. The trigger operation for text recognition can be a click operation, a double-click operation, or a swipe operation triggered by the user.

[0112] In one embodiment, the terminal can set a text recognition trigger control in the user interaction interface where the scanning screen is located. When the terminal obtains a trigger operation for the text recognition trigger control, it triggers text recognition for at least one text object in the scanning screen, obtains the text recognition result, and displays the text recognition result. For example, the terminal can obtain the text content obtained by performing character recognition on the text object and display the text content in the scanning screen. For example, after the terminal obtains the text content obtained by performing character recognition on the text object, it can obtain the translation content corresponding to the text content and display the translation content in the scanning screen. For another example, the terminal can obtain the text object image cut out from the captured image of the scanning screen and store the text object image in a preset file format, such as the PDF format or the table format.

[0113] In one embodiment, the terminal can directly display the text recognition result in the scanning screen. In some other embodiments, the terminal can also display the scanning interface of the scanning screen to enter another user interaction interface, and display the text recognition result in the user interaction interface.

[0114] In one embodiment, the terminal can display the text recognition results corresponding to each text object in the scanning screen. In some other embodiments, the terminal can also only display the text recognition results corresponding to the text objects that are completely displayed in the scanning screen.

[0115] The above-mentioned interactive method for text scanning, as the first step of text recognition, during the process of scanning a text object, continuously and dynamically draw a stroke along the edge of the text object in the scanning screen, and display the edge of the text object to the user in the form of a stroke animation, which can give the user correct and targeted feedback information during the scanning process, display the state of the current text object in the scanning screen, and guide the user to adjust the text object in the scanning screen, thereby improving the efficiency and accuracy of text scanning and text recognition. Specifically, for the text object in the scanning screen, the stroke drawn along its edge can guide the user to adjust the occluded text object so that it is completely displayed in the scanning screen, reducing the situation of failure or low accuracy after multiple attempts to perform text recognition on this text object; when the edge is not occluded, the stroke continued to be drawn along its edge can convey to the user a signal that the text object has been successfully scanned and the next step of text recognition can be performed, so that in response to the trigger operation of text recognition, the text recognition result corresponding to the text object can be displayed. The entire scanning process avoids the user's multiple ineffective text scanning attempts and improves the accuracy and efficiency of text scanning and text recognition.

[0116] In one embodiment, in the scanning screen, continuously draw a stroke along the edge of the text object. When the edge is occluded, display the end of the stroke drawing for the text object and the drawn stroke, including: in the scanning screen, start from any pixel point on the unoccluded edge of the text object, and display continuously draw a stroke along the pixel points on the unoccluded edge of the text object until the drawing ends when all the pixel points on the unoccluded edge are traversed, and display the drawn stroke along the unoccluded edge.

[0117] Among them, any pixel point on the unoccluded edge of the text object can be a pixel point close to the occluded edge or a pixel point with an appropriate distance from the occluded edge. It can be understood that the strokes obtained by using different pixel points on the unoccluded edge as the starting points for drawing may be different.

[0118] Specifically, for the pixel point close to the occluded edge, the stroke continuously drawn along the unoccluded edge of the text object from this pixel point is the stroke drawn along the entire unoccluded edge of the text object. As shown in part (a) of Figure 7 In the scanning screen 70, it includes a text object 702 that is partially occluded. The text object 702 is partially occluded by an occluder. For the partially occluded text object 702, its unoccluded edge 704 includes a pixel point A1 close to the occluded edge. If starting from point A1 on the unoccluded edge 704 and drawing along the clockwise direction, gradually draw to point B1, then to point C1, and finally to point D1, the stroke for the entire unoccluded edge 704 of the partially occluded text object is completed.

[0119] For a pixel point at an appropriate distance from the occluded edge, the stroke continuously drawn along the unoccluded edge of the text object starting from this pixel point is a stroke drawn along a part of the unoccluded edge of the text object. As shown in part (b) of Figure 7 In the scanning screen 70, as shown in part (b) of , there is a text object 702 that is partially occluded. The text object 702 is partially occluded by an occluder. For the partially occluded text object 702, the unoccluded edge 704 includes a pixel point A2 that is far from the occluded edge. If starting from point A2 on the unoccluded edge 704 and drawing along the clockwise direction, gradually drawing to point B2, and finally drawing to point C2, the stroke drawn for a part of the unoccluded edge 704 of the partially occluded text object is completed.

[0120] Optionally, the terminal can start from any pixel point on the unoccluded edge of the text object in the scanning screen and display a stroke continuously drawn along the clockwise or counterclockwise direction of the unoccluded edge of the text object. It can be understood that for the situations shown in parts (a) and (b) of Figure 7 If the stroke is drawn along the counterclockwise direction, there will be too few strokes, which will affect the feedback effect of the drawn stroke on the user. Therefore, if the clockwise direction is used for drawing, after detecting the pixel on the unoccluded edge of the text object, the terminal can, according to the coordinates of the pixel point in the image, identify the pixel point on the unoccluded edge that is located at the lower left corner of the image and use the pixel point at the lower left corner as the starting point for drawing the stroke. Similarly, if the counterclockwise direction is used for drawing, after detecting the pixel on the unoccluded edge of the text object, the terminal can, according to the coordinates of the pixel point in the image, identify the pixel point on the unoccluded edge that is located at the lower right corner of the image and use the pixel point at the lower right corner as the starting point for drawing the stroke.

[0121] In this embodiment, by continuously drawing a stroke starting from any pixel point on the unoccluded edge, the effect of displaying a dynamic stroke can be achieved, so that the user can obtain better and more effective prompt information through the dynamic stroke during the text scanning process.

[0122] In one embodiment, in the scanning screen, a stroke is continuously drawn along the edge of the text object; when the edge is occluded, the drawing of the stroke for the text object and the drawn stroke are displayed. This includes: in the scanning screen, a stroke is continuously drawn along the edge of the text object in a first preset color, and when the edge is occluded, the drawing of the stroke for the text object is ended and the drawn stroke is displayed in a flashing manner.

[0123] Among them, the first preset color can be the default color, such as blue, yellow, or red. The first preset color can also be a color with a large contrast to the background area other than the text object in the scanned image, so that the stroke drawn in the first preset color can better prompt the user.

[0124] In addition, after the stroke of the text object that is partially occluded is drawn, all the pixel points on the stroke can flash together to further prompt the user that the text object is partially occluded, which may lead to the failure of scanning the text object. Optionally, the terminal can flash the drawn stroke in the first preset color after the stroke of the partially occluded text object is drawn. Optionally, the terminal can flash the drawn stroke in a second preset color different from the first preset color after the stroke of the partially occluded text object is drawn.

[0125] Optionally, the terminal can continuously flash the drawn stroke. Optionally, after flashing the drawn partial stroke, the terminal can continue to statically display the completed partial stroke in the scanned image. Optionally, after flashing the drawn partial stroke, the terminal can stop displaying the completed partial stroke in the scanned image.

[0126] In this embodiment, by displaying the drawn stroke in a preset color and flashing the stroke after completion, it can better prompt the user that the text object is partially occluded and may be scanned unsuccessfully, thereby guiding the user to make adjustments to the text object.

[0127] In one embodiment, step 204 includes: when the text object in the scanned image is partially occluded, when displaying the stroke continuously drawn along at least a part of the unoccluded edge of the text object in the scanned image, the stroke follows the text object in real time as the text object offsets in real time in the scanned image.

[0128] In this embodiment, since the scanned image is composed of the images captured by the camera in real time, as the camera shakes or moves, the position of the text object partially occluded in the scanned image will change accordingly. And the process of drawing the stroke is a dynamic process. When the position of the text object in the scanned image changes, during the process of drawing the stroke, the currently drawn partial stroke or the completed stroke will also change or move as the text object offsets in real time, achieving the effect of the stroke following the text object.

[0129] Optionally, during the process of drawing a stroke on the terminal, the drawn stroke follows the text object in real time as the text object shifts. After the stroke drawing on the terminal is completed, the completed stroke follows the text object in real time as the text object shifts. It can be understood that the follow-up of the stroke to the text object starts during the drawing process. During the process of following the text object after the drawing is completed, the drawn stroke may decrease, may increase rapidly, or may also shift.

[0130] Specifically, the terminal can detect in real time the shift of the text object that is partially occluded in the scanned image. When the shift is within the preset range, the currently drawn partial stroke or the completed drawn stroke moves with the shift of the text object. When the camera moves too much relative to the text object or shakes too much, exceeding the preset range, the shift of the text object in the scanned image is too large, which can trigger redrawing the stroke for the text object, and display the redrawn stroke for the shifted text object in the scanned image. Or, when exceeding the preset range, it can trigger directly displaying a new stroke along the local edge of the shifted text object.

[0131] For example, when the user makes an adjustment to the text object that is partially occluded, so that the text object is completely displayed in the scanned image, it can trigger redrawing the stroke for the text object and display the process of redrawing the stroke in the scanned image.

[0132] In this embodiment, by displaying in the scanned image a stroke that follows the text object in real time as the text object shifts, it can better prompt the user that the text object being partially occluded may cause the scanning to fail, thereby guiding the user to make adjustments to the text object.

[0133] In one embodiment, in the scanned image, continuously drawing a stroke along the edge of the text object includes: starting continuous stroke drawing from any pixel point on the complete edge of the text object in the scanned image; when the edge is not occluded, displaying continuing to continuously draw the stroke along the edge of the text object until a closed stroke is formed and then ending the drawing and displaying the closed stroke, including: when the edge is not occluded, displaying continuing to continuously draw the stroke along the pixel points on the complete edge of the text object until a closed stroke is formed and then ending the drawing and displaying the closed stroke.

[0134] Among them, any pixel point on the complete edge of the text object can be any pixel point. It can be understood that for a text object that is completely displayed in the scanned image, starting the drawing from different pixel points on the complete edge of the text object can obtain a closed stroke of the complete edge.

[0135] Such as Figure 8As shown, it is a schematic diagram of drawing a closed stroke for a text object that is fully displayed in an embodiment. Refer to Figure 8 In part (a), in the scanning screen 80, it includes a text object 802 that is fully displayed. It can start from point A1 on the complete edge 804 and draw in a clockwise direction, gradually draw to point B1, then to point C1, and finally to point A1 to complete the closed stroke drawn for the complete edge 804 of the text object that is fully displayed. Refer to Figure 8 In part (b), it can start from point A2 on the complete edge 804 and draw in a clockwise direction, gradually draw to point B2, then to point C2, and finally to point A2 to complete the closed stroke drawn for the complete edge 804 of the text object that is fully displayed.

[0136] In this embodiment, for the text object that is fully displayed, the closed stroke drawn along its complete edge can convey a signal to the user that the text object has been successfully scanned and text recognition can be performed in the next step.

[0137] In an embodiment, in the scanning screen, continuously drawing a stroke along the edge of the text object is displayed, including: in the scanning screen, continuously drawing a stroke along the clockwise or counterclockwise direction of the complete edge of the text object in a second preset color; when the edge is not blocked, continuously drawing a stroke along the edge of the text object until a closed stroke is formed and then ending the drawing and displaying the closed stroke, including: when the edge is not blocked, continuously drawing a stroke along the pixel points on the complete edge of the text object until a closed stroke is formed and then ending the drawing and displaying the closed stroke.

[0138] Optionally, the first terminal can display the stroke drawn along the partial edge of the text object that is partially blocked in the scanning screen in a first preset color, and display the stroke continuously drawn along the complete edge of the text object that is fully displayed in the scanning screen in the first preset color or a second preset color different from the first preset color to distinguish between the two.

[0139] Optionally, when the text object is a rectangular text object, starting from any pixel point on the complete edge of the rectangular text object, display a stroke continuously drawn along the complete rectangular edge of the rectangular text object until a closed rectangular stroke is formed and then ending the drawing.

[0140] Optionally, the terminal can continue to statically display the drawn closed stroke in the scanning screen, or the terminal can stop displaying the drawn closed stroke in the scanning screen after displaying the drawn closed stroke.

[0141] In this embodiment, for the text objects fully displayed in the scanned image, by drawing a stroke with a preset color and displaying the stroke after completion, a better effect of prompting the user can be achieved.

[0142] In one embodiment, the above-mentioned interactive method for text scanning further includes: when the edge is blocked, after ending the drawing of the stroke for the text object and displaying the drawn stroke, displaying a prompt message indicating that the scanning of the text object has failed.

[0143] Since there may be a situation where some text content in the text object partially blocked cannot be recognized, therefore, to improve the efficiency of text recognition, the terminal can, after completing the local drawing of the stroke along the edge, display a prompt message indicating that the scanning of the text object has failed in the scanned image, so as to inform the user not to perform text recognition on the text object, thereby guiding the user to make adjustments to the text object. The prompt message can be in the form of text or an icon.

[0144] Optionally, for a text object partially blocked, after displaying the completed stroke, the completed stroke can be statically and continuously displayed in the scanned image, and at the same time, a prompt message indicating that the scanning of the text object has failed can be displayed, or the completed stroke can no longer be displayed in the scanned image, and only the prompt message indicating that the scanning of the text object has failed can be displayed. The prompt message can be displayed close to the text object.

[0145] As Figure 9 shown, it is a schematic diagram of the change of the scanned image in one embodiment. Referring to Figure 9 part (a), in the scanned image 90, it includes a text object 902 partially blocked. Referring to Figure 9 part (b), a stroke 904 drawn along the unblocked edge of the text object 902 is displayed. Referring to Figure 9 part (c), in the scanned image, the stroke 904 along the text object 902 is displayed, and at the same time, a prompt message 906 indicating that the scanning of the text object has failed is displayed.

[0146] Optionally, the terminal can, in response to the trigger operation of text recognition, only perform text recognition on the text objects that are successfully scanned in the scanned image, display the text recognition results corresponding to the text objects that are successfully scanned, and not perform text recognition on the text objects that fail to be scanned.

[0147] In this embodiment, by further displaying a prompt message indicating the failure of scanning for the text object partially blocked in the scanned image, the user can be further guided to make adjustments to the text object partially blocked.

[0148] In one embodiment, the above-mentioned interactive method for text scanning further includes: when the edge is not blocked, display continuous drawing along the edge of the text object until a closed stroke is formed, then end the drawing and display the closed stroke, and then display a prompt message indicating that the text object has been successfully scanned.

[0149] Since the text content in the text object that is completely displayed can be fully collected, accurate text content can be obtained by performing text recognition on the text object. Therefore, after the closed stroke is drawn, the terminal can display a prompt message indicating that the text object has been successfully scanned in the scanning screen to inform the user to further perform text recognition on the text object, thereby improving the efficiency of text recognition. The prompt message can be in the form of text or an icon.

[0150] Optionally, for a text object that is partially blocked, after the drawn stroke is displayed, the drawn stroke can be statically and continuously displayed in the scanning screen, and at the same time, a prompt message indicating that the scanning of the text object has failed can be displayed. Alternatively, the drawn stroke can no longer be displayed in the scanning screen, and only the prompt message indicating that the scanning of the text object has failed can be displayed. The prompt message can be displayed near the text object.

[0151] Such as Figure 10 shown, is a schematic diagram of the change of the scanning screen in one embodiment. Referring to Figure 10 part (a), in the scanning screen 100, it includes a text object 1002 that is completely displayed. Referring to Figure 10 part (b), a stroke 1004 drawn along the complete edge of the text object 1002 is displayed. Referring to Figure 10 part (c), the stroke 1004 along the text object 1002 is displayed in the scanning screen, and at the same time, a prompt message 1006 indicating that the text object has been successfully scanned is displayed.

[0152] Optionally, the terminal can respond to the trigger operation of text recognition, and only perform text recognition on the text objects that have been successfully scanned in the scanning screen, display the text recognition results corresponding to the text objects that have been successfully scanned, and do not perform text recognition on the text objects that have failed to be scanned.

[0153] In this embodiment, by further displaying a prompt message indicating successful scanning of the text object that is completely displayed in the scanning screen, the user can be further guided to perform text recognition on the text object that is completely displayed.

[0154] In one embodiment, step 210 includes: in response to the trigger operation of text translation, display the currently acquired image of the scanning screen; in the acquired image, display the translation content corresponding to the text content in the text object.

[0155] Specifically, text recognition may include text translation. Text translation means that after identifying the text content of a text object, the text content is further translated to obtain the corresponding translated content. Among them, the triggering operation of text translation is the triggering operation to start text translation. When the user believes that the text object has been successfully scanned through the stroke displayed on the terminal, the text translation operation can be triggered in this user interface. When the user believes that the text object may have failed to be translated through the stroke displayed on the terminal, after adjusting the text object in the scanned image, the text translation operation can be triggered again. The terminal can respond to the text translation trigger operation in the scanned image, display the captured acquisition image of the current scanned image, and after obtaining the translated content of the text content in the acquisition graphic, display the translated content in the acquisition image.

[0156] Optionally, the terminal can set a text translation trigger control in the user interaction interface where the scanned image is located. The terminal responds to the triggering operation of the user on this control in the user interaction interface, displays the captured acquisition image of the current scanned image, and displays the translated content of the text content in the text object in this acquisition image.

[0157] Optionally, the displayed translated content can cover and replace the original text content in the acquisition image. Optionally, the displayed translated content can also be displayed in contrast with the original text content in this acquisition image. Optionally, the terminal can send the captured acquisition image to the server, identify the text content in the acquisition image through the server, obtain the corresponding translated content, receive the translated content returned by the server, and then display the translated content in the acquisition image.

[0158] As Figure 11 shown, it is a schematic diagram of the user interaction interface where the scanned image is located in an embodiment. Referring to Figure 11 part (a), in the user interaction interface where the scanned image 110 is located, it also includes a text object 1102 that is fully displayed, a text translation trigger control 1104, and a complete stroke 1106 of the drawn text object 1102. The terminal responds to the triggering operation 1108 on this control, and displays Figure 11 part (b), that is, the captured acquisition image of the current scanned image, and displays the translated content of the text content in the text object in the acquisition image. This translated content replaces and covers the original text content.

[0159] In this embodiment, by displaying the drawn stroke in the scanned image, it is possible to guide the adjustment of the text object displayed in the scanned image, thereby improving the efficiency and accuracy of text scanning and text translation.

[0160] In one embodiment, the above text scanning interaction method further includes: when a graphic code is included in a text object in the scanning screen, displaying a prompt message indicating successful scanning of the graphic code in the scanning screen; step 210 includes: displaying an access interface corresponding to the embedded link of the graphic code in the text object.

[0161] Among them, the graphic code can be a bar code, a two-dimensional code or a mini-program code. Different from text recognition which recognizes the text content in a text object, a graphic code is a black-and-white geometric pattern distributed regularly in a two-dimensional plane in a plane, which can be used to record data and information. After scanning the graphic code, you can directly enter the access page corresponding to the graphic code. In some scenarios, there will be graphic codes on text objects. For example, there will be a graphic code of the enterprise where the employee is located printed on a business card. For example, there will be a graphic code for querying bill information printed on a paper bill.

[0162] Specifically, when a completely displayed graphic code is included in a text object in the scanning screen, the terminal can, when collecting the graphic code, display a prompt message indicating successful scanning of the graphic code in the scanning screen, and according to the user's confirmation operation, enter from the user interaction interface where the scanning screen is located into the access page corresponding to the embedded link of the graphic code, or directly enter the access page corresponding to the embedded link of the graphic code after successful scanning. The prompt message can be a text-based prompt message or an icon-based prompt message.

[0163] In this embodiment, if a graphic code is collected during text scanning, the access page corresponding to the embedded link of the graphic code can be directly displayed, which is convenient for user operation.

[0164] In one embodiment, as Figure 12 shown, when the text object is a card or certificate, step 202 includes:

[0165] Step 1202, displaying a card / certificate information entry interface.

[0166] Among them, the card or certificate can be a certificate or card including text content, such as a business card, an ID card or a handwritten note, etc. The card / certificate information entry interface is a user interaction interface in the text recognition function provided on the terminal. Optionally, this text recognition function can be provided by a social application on the terminal, or can also be provided by an online document application on the terminal.

[0167] Step 1204, in response to a card / certificate scanning trigger operation in the card / certificate information entry interface, entering the card / certificate scanning interface from the card / certificate information entry interface.

[0168] Among them, the card and certificate scanning trigger operation is the trigger operation to start card and certificate scanning. The terminal can respond to this card and certificate scanning trigger operation and jump from or switch to the card and certificate scanning interface from the card and certificate information entry interface.

[0169] Step 1206, in the card and certificate scanning interface, display a scanning screen including at least one card and certificate.

[0170] Specifically, after the terminal starts the camera, it obtains the captured image including the card and certificate in real time, and displays the captured image in real time in the card and certificate scanning interface, that is, displays the scanning screen of this card and certificate.

[0171] Such as Figure 13 shown, is a schematic diagram of the card and certificate information entry interface and the card and certificate scanning interface in an embodiment. Refer to Figure 13 , in the card and certificate information entry interface 130, it includes a card and certificate scanning control 1302. The terminal can respond to the trigger operation 1304 of the user for this card and certificate scanning control 1302, enter the card and certificate scanning interface 131, and display the scanning screen in this card and certificate scanning interface 131.

[0172] In a specific application scenario, the user can open an online document application, enter the card and certificate information entry interface of the online document application, trigger the card and certificate scanning trigger operation in this card and certificate information entry interface. The terminal starts the camera according to this trigger operation, and enters the card and certificate scanning interface from this card and certificate scanning information entry interface, and displays the scanning screen including text objects captured in real time by the camera in this card and certificate scanning interface.

[0173] In this embodiment, by setting the card and certificate information entry interface and triggering text scanning in the card and certificate information entry interface, the functions of text scanning and text recognition can be directly realized based on the card and certificate information entry interface, without additional operations, which is convenient for users to operate.

[0174] In an embodiment, step 210 includes: responding to the text recognition trigger operation in the card and certificate scanning interface, returning to the card and certificate information entry interface from the card and certificate scanning interface; in the card and certificate information entry interface, display the text recognition entry information corresponding to the card and certificate.

[0175] Among them, the card and certificate information entry interface may further include a card and certificate filling form. The user can manually enter the information in the card and certificate into this card and certificate filling form, or automatically fill the recognized text recognition entry information into this card and certificate filling form through text. For example, when the card and certificate is a business card, the items of the card and certificate filling form may include name, position, company, address, mobile phone, etc.

[0176] Optionally, the terminal can set a text recognition trigger control in the card scanning interface. When the user believes that the card has been successfully scanned based on the stroke of the card displayed on the terminal, a text recognition trigger operation for the card can be triggered in this card scanning interface. When the user believes that the card recognition may fail based on the stroke displayed on the terminal, after adjusting the card in the scanning screen, the text recognition trigger operation for the card can be triggered again.

[0177] Optionally, the terminal can send the captured acquisition image to the server, identify the text recognition entry information corresponding to the card in the acquisition image through the server, receive the text recognition entry information returned by the server, and then automatically fill the text recognition entry information into the card filling form in the card scanning interface.

[0178] Refer to Figure 13 , in the card scanning interface 131, there is a text recognition trigger control 1312. The terminal responds to the trigger operation 1314 for this control, obtains the text recognition entry information obtained by performing text recognition on the text object in this scanning screen, and automatically fills the text recognition entry information into the card information entry interface 130.

[0179] In this embodiment, the stroke of the card displayed by the terminal guides the user to perform the next text recognition, obtains the text recognition entry information, and automatically displays the obtained text recognition entry information, which can improve the accuracy and efficiency of text recognition for the card.

[0180] In one embodiment, the above text scanning interaction method further includes: preprocessing the image collected from the scanning screen to obtain a preprocessed image; determining the edge pixel points in the preprocessed image; after classifying the edge pixel points, obtaining at least one edge point set; identifying the shape corresponding to each edge point set; when the identified shape is a closed shape, determining that the text object corresponding to the edge point set is completely displayed in the scanning screen, and when the identified shape is not a closed shape, determining that the text object corresponding to the edge point set is partially blocked in the scanning screen.

[0181] In one embodiment, when the identified shape is a closed rectangle, it is determined that the text object corresponding to the edge point set is completely displayed in the scanning screen, and when the identified shape is not a closed rectangle, it is determined that the text object corresponding to the edge point set is partially blocked in the scanning screen.

[0182] Considering that most text objects in daily life are rectangles, in this embodiment, by determining whether the edge pixels of the text object can form a closed rectangle, it is determined whether the text object is completely displayed or partially blocked. This recognition step can be executed by the terminal or by the server.

[0183] Specifically, first, it is necessary to preprocess the currently captured acquisition image in the scanned image, including denoising and contrast enhancement processing of the acquired image to obtain a preprocessed image. For example, the medianBlur function, that is, the median filter, can be used to smooth the acquired image. Then, the edge detection algorithm is used to identify the edge pixel points in the image. The terminal can use the Canny edge detection algorithm to calculate the gradient and direction corresponding to each pixel point in the preprocessed image, and determine the edge pixel points based on the gradient and direction. Then, the terminal sets the edge pixel points as a data set. After classifying this data set, at least one edge point set is obtained. For example, when there are two text objects in the acquired image, the edge pixel points in this data set are classified into two categories to obtain two edge point sets. When there is one text object in the acquired image, the edge pixel points in this data set will be classified into one category to obtain one edge point set. Then, combined with the position information of each pixel point, the shape formed by the edge pixel points in each edge point set is identified. For example, the terminal can use the contour detection function findcontour to obtain the contour corresponding to the edge point set, and determine whether a closed shape can be formed, so as to determine whether the text object corresponding to this edge point set is completely displayed or partially occluded. Further, it can also be determined whether it is a closed rectangle by using the polygon fitting curve function approxPloyDP for this contour.

[0184] As Figure 14 shown, it is a timing diagram of the interactive method for text scanning in an embodiment. Referring to Figure 14 , after starting text scanning, the user acquires multiple text objects N1 and N2 through the camera of the terminal and sends the acquired image to the server. After the server identifies the edge pixel points of the multiple text objects, it is determined that the edge pixel points of N1 form a closed rectangle. The terminal displays a dynamic stroke drawn along the edge of the text object N1 and displays a prompt message indicating that the text object N1 has been scanned successfully. The server determines that the edge pixel points of the text object N2 do not form a closed rectangle. The terminal displays a dynamic stroke drawn along the unoccluded edge of the text object N2 and displays a prompt message indicating that N2 has failed to be scanned. After the user actively adjusts the text object N2 and makes the text object N2 completely displayed in the acquired image, the terminal displays a dynamic stroke drawn along the complete edge of the text object N2 and displays a prompt message indicating that N2 has been scanned successfully, thus completing the scanning of multiple text objects.

[0185] In one embodiment, an interactive method for text scanning is provided, including:

[0186] In response to a trigger operation for text scanning, a scanning screen including a plurality of text objects is displayed; for each text object in the scanning screen, in the scanning screen, each stroke that is continuously drawn along the edge of each text object is displayed; for each text object in the scanning screen, when the edge of the text object is occluded, the end of the stroke drawing for the text object and the drawn strokes are displayed, and when the edge is not occluded, the continuous drawing of the stroke along the edge of the text object is displayed until the drawing is ended when a closed stroke is formed and the closed stroke is displayed; in response to a trigger operation for text recognition, the text recognition result corresponding to at least one text object among the plurality of text objects is displayed.

[0187] It should be noted that for the explanations and descriptions of the nouns and concepts involved in this embodiment, reference can be made to the relevant content mentioned above, and details will not be elaborated here. In this embodiment, the scanning screen includes a plurality of text objects, and the plurality of text objects refer to two or more text objects. For each scanning object, the continuous drawing of strokes along the edge of each scanning object is displayed simultaneously. When a certain text object in the scanning screen is not occluded, the stroke drawing is ended and the drawn strokes are displayed. When a certain text object in the scanning screen is not occluded, the continuous drawing of the stroke along the edge of the text object is displayed until the drawing is ended when a closed stroke is formed and the closed stroke is displayed. During the process of scanning a plurality of text objects, each stroke is dynamically and continuously drawn along the edge of each text object in the scanning screen, and the edge of each text object is presented to the user in the form of a stroke animation, which can provide the user with correct and targeted feedback information during the scanning process, display the current state of each text object in the scanning screen, and guide the user to make adjustments to each text object in the scanning screen, thereby improving the efficiency and accuracy of text scanning and text recognition.

[0188] In a specific application scenario, the interactive method for text scanning includes the following steps:

[0189] The user starts text scanning and enters the text scanning interface; in the text scanning interface, a scanning screen including at least one text object is displayed; for the captured image in the scanning screen, the captured image is preprocessed to obtain a preprocessed image, then the edge pixel points in the preprocessed image are identified, and the edge pixel points are classified. For example, they are divided into two categories, namely the edge point set X1 composed of the edge pixel points belonging to the completely displayed text object N1 and the edge point set X2 composed of the edge pixel points of the partially occluded text object N2. For the pixel points in X1, if it is determined that a closed rectangle can be formed, any one of the pixel points is selected, and each edge pixel point in the edge point set is displayed in blue in the clockwise direction, so as to form a dynamic stroke drawn along the contour. When the drawing is completed, that is, when the rectangular closed stroke is formed, a prompt message indicating successful scanning is displayed inside the closed stroke, so as to guide the user to perform the triggering operation of the next text recognition. For the pixel points in X2, if it is determined that a closed rectangle cannot be formed, any one of the pixel points is selected, and each edge pixel point in the edge point set is displayed in blue in the clockwise direction, so as to form a dynamic stroke drawn along the contour. When the drawing is completed, that is, when the last edge pixel point in X2 is drawn, an unclosed stroke is formed. At this time, the stroke composed of all the pixel points on the stroke can be displayed in yellow and flicker together, and a prompt message indicating that the scanning of this text object fails is displayed, so as to guide the user to make adjustments to this text object. Finally, the terminal responds to the triggering operation of text recognition, performs text recognition on the successfully scanned text object N1, obtains a text recognition result, and displays the text recognition result of the text object N1.

[0190] It should be understood that although the steps in the above flowchart are displayed sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0191] In one embodiment, as Figure 15 shown, an interactive device 1500 for text scanning is provided. This device can be a software module, a hardware module, or a combination of both to become a part of a computer device. Specifically, the device includes: a scanning screen display module 1502, a stroke display module 1504, and a text recognition result display module 1506, where:

[0192] A scanning screen display module 1502, configured to display a scanning screen including at least one text object in response to a triggering operation of text scanning;

[0193] A stroke display module 1504, configured to display continuous drawing strokes along the edges of text objects in the scanning screen;

[0194] The stroke display module 1504 is further configured to, when the edge is occluded, display the end of drawing strokes for the text object and the drawn strokes; when the edge is not occluded, display continuous drawing strokes along the edges of the text object until a closed stroke is formed and then end the drawing and display the closed stroke;

[0195] A text recognition result display module 1506, configured to display the text recognition result corresponding to the text object in response to a triggering operation of text recognition.

[0196] In one embodiment, the stroke display module 1504 is further configured to start from any pixel point on the unoccluded edge of the text object in the scanning screen and display continuous drawing strokes along the pixel points on the unoccluded edge of the text object until all pixel points on the unoccluded edge are traversed and then end the drawing and display the drawn strokes along the unoccluded edge.

[0197] In one embodiment, the stroke display module 1504 is further configured to display continuous drawing strokes along the edges of the text object in the scanning screen in a first preset color, and when the edge is occluded, end drawing strokes for the text object and display the drawn strokes in a flashing manner.

[0198] In one embodiment, the stroke display module 1504 is further configured to, when displaying continuous drawing strokes along the edges of the text object in the scanning screen, make the strokes follow the text object in real time as the text object is offset in real time in the scanning screen.

[0199] In one embodiment, the stroke display module 1504 is further configured to start continuous drawing strokes from any pixel point on the complete edge of the text object in the scanning screen; when the edge is not occluded, display continuous drawing strokes along the pixel points on the complete edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke.

[0200] In one embodiment, the stroke display module 1504 is further configured to display continuous drawing strokes along the clockwise or counterclockwise direction of the complete edge of the text object in the scanning screen in a second preset color; when the edge is not occluded, display continuous drawing strokes along the pixel points on the complete edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke.

[0201] In one embodiment, the text scanning interaction device 1500 further includes:

[0202] A prompt message display module, configured to display a prompt message indicating that the text object scanning fails after ending the stroke drawing of the text object and displaying the drawn stroke when the edge is blocked;

[0203] The prompt message display module is further configured to display a prompt message indicating that the text object scanning is successful after continuously drawing strokes along the edge of the text object until a closed stroke is formed and displaying the closed stroke when the edge is not blocked.

[0204] In one embodiment, the text recognition result display module 1506 is further configured to display the currently captured image of the scanning screen in response to a trigger operation for text translation; in the captured image, display the translation content corresponding to the text content in the text object.

[0205] In one embodiment, the text scanning interaction device 1500 further includes a prompt message display module, configured to display a prompt message indicating that the barcode scanning is successful in the scanning screen when the text object in the scanning screen includes a barcode;

[0206] The text recognition result display module 1506 is further configured to display the access interface corresponding to the embedded link of the barcode in the text object.

[0207] In one embodiment, when the text object is a card or certificate, the scanning screen display module 1504 is further configured to display a card or certificate information entry interface; in response to a card or certificate scanning trigger operation in the card or certificate information entry interface, enter the card or certificate scanning interface from the card or certificate information entry interface; in the card or certificate scanning interface, display a scanning screen including at least one card or certificate.

[0208] In one embodiment, the text recognition result display module 1506 is further configured to return from the card or certificate scanning interface to the card or certificate information entry interface in response to a text recognition trigger operation in the card or certificate scanning interface; in the card or certificate information entry interface, display the text recognition entry information corresponding to the card or certificate.

[0209] In one embodiment, the text scanning interaction device 1500 further includes a rectangle detection module, including:

[0210] A preprocessing unit, configured to preprocess the image captured from the scanning screen to obtain a preprocessed image;

[0211] An edge pixel recognition unit, configured to determine the edge pixel points in the preprocessed image;

[0212] A rectangle recognition unit, which is used to classify edge pixel points and obtain at least one edge point set to recognize the shape corresponding to each edge point set; when the recognized shape is a closed rectangle, it is determined that the edge of the text object corresponding to the edge point set is not blocked in the scanned image, and when the recognized shape is not a closed rectangle, it is determined that the edge of the text object corresponding to the edge point set is blocked in the scanned image.

[0213] In one embodiment, the preprocessing unit is further configured to perform denoising processing and contrast enhancement processing on the acquired image to obtain a preprocessed image;

[0214] The edge pixel recognition unit is further configured to calculate the gradient and direction corresponding to each pixel point in the preprocessed image, and determine edge pixel points according to the gradient and direction.

[0215] A text scanning interaction device, comprising:

[0216] A scanned image display module, which is used to respond to a trigger operation of text scanning and display a scanned image including a plurality of text objects;

[0217] A stroke display module, which is used to, for each text object in the scanned image, display each stroke continuously drawn along the edge of each text object in the scanned image;

[0218] The stroke display module is further configured to, for each text object in the scanned image, when the edge of the text object is blocked, display the end of the stroke drawing of the text object and the drawn stroke, and when the edge is not blocked, display continuing to continuously draw a stroke along the edge of the text object until a closed stroke is formed and then end the drawing and display the closed stroke;

[0219] A text recognition result display module, which is used to respond to a trigger operation of text recognition and display the text recognition result corresponding to at least one text object among the plurality of text objects.

[0220] The above text scanning interaction device, as the first step of text recognition, dynamically and continuously draws a stroke along the edge of the text object in the scanning screen during the process of scanning the text object, and displays the edge of the text object to the user in the form of a stroke animation. It can give the user correct and targeted feedback information during the scanning process, display the state of the current text object in the scanning screen, and guide the user to adjust the text object in the scanning screen, thereby improving the efficiency and accuracy of text scanning and text recognition. Specifically, for a text object that is partially occluded in the scanning screen, the stroke drawn along its partial edge can guide the user to adjust the text object that is partially occluded so that it is completely displayed in the scanning screen, reducing the situation of failure or low accuracy after multiple attempts to perform text recognition on the partially occluded text object; for a text object that is completely displayed, the stroke drawn along its complete edge can convey to the user a signal that the text object has been successfully scanned and the next step of text recognition can be performed. Thus, in response to the trigger operation of text recognition, the text recognition result corresponding to the text object can be displayed, and the entire scanning process avoids the user's multiple ineffective text scanning attempts, improving the accuracy and efficiency of text scanning and text recognition.

[0221] For the specific limitations of the text scanning interaction device, reference can be made to the limitations of the text scanning interaction method in the above text, which will not be elaborated here. Each module in the above text scanning interaction device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above modules.

[0222] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 16As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it realizes an interactive method for text scanning. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball, or touchpad set on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0223] Those skilled in the art can understand that Figure 16 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0224] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps in the above method embodiments are realized.

[0225] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are realized.

[0226] In one embodiment, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the above method embodiments.

[0227] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0228] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0229] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An interactive method for text scanning, comprising: In response to a trigger operation for text scanning, displaying a scanning screen including at least one text object; In the scanning screen, displaying continuous drawing of a stroke along the edge of the text object; When the edge is occluded, displaying the end of drawing the stroke for the text object and the drawn stroke; When the edge is not occluded, displaying continuous drawing of a stroke along the edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed; In response to a trigger operation for text recognition, displaying the text recognition result corresponding to the text object.

2. The method according to claim 1, characterized in that, The step of, in the scanning screen, displaying continuous drawing of a stroke along the edge of the text object; when the edge is occluded, displaying the end of drawing the stroke for the text object and the drawn stroke, includes: In the scanning screen, starting from any pixel point on the unoccluded edge of the text object, displaying continuous drawing of a stroke along the pixel points on the unoccluded edge of the text object until all the pixel points on the unoccluded edge are traversed, at which point the drawing ends and the drawn stroke along the unoccluded edge is displayed.

3. The method according to claim 1, characterized in that The step of, in the scanning screen, displaying continuous drawing of a stroke along the edge of the text object; when the edge is occluded, displaying the end of drawing the stroke for the text object and the drawn stroke, includes: In the scanning screen, displaying continuous drawing of a stroke along the edge of the text object in a first preset color, and when the edge is occluded, ending the drawing of the stroke for the text object and flashing the drawn stroke.

4. The method according to claim 1, characterized in that The step of, in the scanning screen, displaying continuous drawing of a stroke along the edge of the text object, includes: In the scanning screen, when displaying continuous drawing of a stroke along the edge of the text object, the stroke follows the text object in real time as the text object is offset in real time in the scanning screen.

5. The method according to claim 1, wherein The step of, in the scanning screen, displaying continuous drawing of a stroke along the edge of the text object, includes: In the scanning screen, starting continuous drawing of a stroke from any pixel point on the complete edge of the text object; The step of, when the edge is not occluded, displaying continuous drawing of a stroke along the edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed, includes: When the edge is not occluded, displaying continuous drawing of a stroke along the pixel points on the complete edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed.

6. The method according to claim 1, characterized in that The step of, in the scanning screen, displaying continuous drawing of a stroke along the edge of the text object, includes: In the scanning screen, displaying continuous drawing of a stroke along the complete edge of the text object in a clockwise or counterclockwise direction in a second preset color; The step of, when the edge is not occluded, displaying continuous drawing of a stroke along the edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed, includes: When the edge is not blocked, continue to draw a continuous stroke along the pixel points on the complete edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed.

7. The method according to claim 1, characterized in that, The method further includes: When the edge is blocked, after ending the stroke drawing of the text object and displaying the drawn stroke, display a prompt message indicating that the scanning of the text object has failed; When the edge is not blocked, after continuing to draw a continuous stroke along the edge of the text object until a closed stroke is formed, at which point the drawing ends and the closed stroke is displayed, display a prompt message indicating that the scanning of the text object has been successful.

8. The method according to claim 1, characterized in that, The displaying of the text recognition result corresponding to the text object in response to the trigger operation for text recognition includes: In response to the trigger operation for text translation, display the currently acquired image of the scanning screen; In the acquired image, display the translation content corresponding to the text content in the text object.

9. The method according to claim 1, characterized in that, The method further includes: When the text object in the scanning screen includes a graphic code, in the scanning screen, display a prompt message indicating that the scanning of the graphic code has been successful; The displaying of the text recognition result corresponding to the text object in response to the trigger operation for text recognition includes: Display the access interface corresponding to the embedded link of the graphic code in the text object.

10. The method according to claim 1, wherein The text object is a card or certificate. The displaying of the scanning screen including at least one text object in response to the trigger operation for text scanning includes: Display a card or certificate information entry interface; In response to the card or certificate scanning trigger operation in the card or certificate information entry interface, enter the card or certificate scanning interface from the card or certificate information entry interface; In the card or certificate scanning interface, display the scanning screen including at least one card or certificate.

11. The method according to claim 10, wherein The displaying of the text recognition result corresponding to the text object in response to the trigger operation for text recognition includes: In response to the text recognition trigger operation in the card or certificate scanning interface, return from the card or certificate scanning interface to the card or certificate information entry interface; In the card or certificate information entry interface, display the text recognition entry information corresponding to the card or certificate.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: Preprocess the image acquired from the scanning screen to obtain a preprocessed image; Determine the edge pixel points in the preprocessed image; After classifying the edge pixel points, obtain at least one set of edge points; Identify the shape corresponding to each set of edge points; When the recognized shape is a closed rectangle, determine that the edge of the text object corresponding to the set of edge points is not blocked in the scanning screen. When the recognized shape is not a closed rectangle, determine that the edge of the text object corresponding to the set of edge points is blocked in the scanning screen.

13. The method according to claim 12, wherein, The preprocessing of the currently acquired image in the scanning screen to obtain a preprocessed image includes: Perform denoising processing and contrast enhancement processing on the acquired image to obtain a preprocessed image; The determining of the edge pixel points in the preprocessed image includes: Calculate the gradient and direction corresponding to each pixel point in the preprocessed image, and determine the edge pixel points based on the gradient and direction.

14. An interactive method for text scanning, characterized in that, The method includes: In response to a trigger operation for text scanning, a scanning screen including a plurality of text objects is displayed; For each text object in the scanning screen, in the scanning screen, each stroke continuously drawn along the edge of each text object is displayed; For each text object in the scanning screen, when the edge of the text object is occluded, the drawing of the stroke for the text object and the drawn stroke are displayed as ended, and when the edge is not occluded, the continuous drawing of the stroke along the edge of the text object is displayed, and the drawing ends and the closed stroke is displayed when a closed stroke is formed; In response to a trigger operation for text recognition, the text recognition result corresponding to at least one text object among the plurality of text objects is displayed.

15. The method according to claim 14, wherein The step of, for each text object in the scanning screen, displaying in the scanning screen each stroke continuously drawn along the edge of each text object includes: For each text object in the scanning screen, in the scanning screen, each stroke continuously drawn along the edge of each text object in a first preset color is displayed; The step of, when the edge of the text object is occluded, ending the drawing of the stroke for the text object and displaying the drawn stroke includes: When the edge of the text object is occluded, ending the drawing of the stroke for the text object and flashing the drawn stroke in a second preset color.

Images