Document processing method and electronic device

Abstract

The present application provides a document processing method and an electronic device. The method includes: displaying a first interface for displaying a video during recording; in response to detecting a first document in the first interface, displaying a first control in the first interface; in response to a first operation on the first control, determining a recording focal length; according to the recording focal length, detecting an image within the document border of the first document to obtain a picture set of the first document; after the video recording ends, generating a second document according to the picture set of the first document. Thus, in the present application, when a document is detected during video recording, the first control can be displayed to facilitate providing the user with the function of obtaining the first document. Responding to the first control to obtain the first document can save the user's operation time and ensure the quality of the pictures of the first document obtained. After the video recording ends, generating a second document according to the picture set of the first document can ensure the efficiency of obtaining the second document.

Description

Technical Field

[0001] This application belongs to the technical field of terminals, and particularly relates to a document processing method and an electronic device. Background Art

[0002] In scenarios such as speeches, offline or online teaching, and meetings, users often use electronic devices to record documents, such as electronic documents or paper documents.

[0003] Currently, users can use an electronic device to record or take pictures of the content of a document. After the recording is completed, the user also needs to use the electronic device to take screenshots and splice the recorded video to obtain the document. Or, after taking pictures, the user also needs to use the electronic device to splice the taken pictures to obtain the document.

[0004] However, in the above implementation process, the obtained document has low quality, and requires cumbersome or time-consuming operations by the user, which will affect the user experience. Summary of the Invention

[0005] This application provides a document processing method and an electronic device. During video recording, when a document is detected, a control for triggering the acquisition of the document can be displayed, and the document can be acquired in response to the control, improving the quality of the obtained document and saving the user's operation time.

[0006] In a first aspect, this application provides a document processing method, including: displaying a first interface, where the first interface is used to display the video during recording; in response to detecting a first document in the first interface, displaying a first control in the first interface; in response to a first operation on the first control, determining a recording focal length, where the recording focal length is used to adjust the size of the viewfinder frame of the first interface to match the size of the document border of the first document in the first interface; according to the recording focal length, detecting the images within the document border of the first document to obtain a picture set of the first document, where the picture set of the first document includes one or more pictures; after the video recording ends, generating a second document according to the picture set of the first document.

[0007] In the above method, during video recording, when the first document is detected, the first control for triggering the electronic device to acquire the first document can be displayed in the first interface, so that the electronic device can remind the user through the first control that the electronic device has the function of being able to acquire the first document, reflecting the diversified services of the electronic device and providing the possibility of acquiring the first document during video recording.

[0008] In addition, the recording focal length can be determined so that the size of the viewfinder of the first interface matches the size of the document border of the first document. Thus, an image of the first document with high clarity can be obtained, and the pictures in the picture set of the first document obtained based on this image are pictures with high clarity. After the video recording is completed, a second document can be generated according to the pictures in the picture set of the first document. This saves the user's operation time and improves the quality of the generated second document.

[0009] Combined with the first aspect, in some implementation manners of the first aspect, the above method further includes: determining a recording direction, where the recording direction is used to represent the direction of the viewfinder of the first interface; detecting the image within the document border of the first document according to the recording focal length, including: detecting the image within the document border of the first document according to the recording focal length and the recording direction.

[0010] Among them, the step of determining the recording direction can enable the electronic device to more accurately adjust the recording focal length, and thus, an image of the first document with high clarity can be obtained.

[0011] Combined with the first aspect, in some implementation manners of the first aspect, determining the recording focal length and the recording direction includes: determining the current direction of the viewfinder of the first interface and the direction of the document border of the first document; when the current direction of the viewfinder of the first interface is inconsistent with the direction of the document border of the first document, displaying a second control on the first interface; after receiving a second operation on the second control, adjusting the current direction of the viewfinder of the first interface to obtain the recording direction; determining the recording focal length according to the recording direction and the ratio between the document border of the first document and the viewfinder of the first interface corresponding to the adjusted direction; when no second operation on the second control is received after a preset duration, or when the current direction of the viewfinder of the first interface is consistent with the direction of the document border of the first document, determining the current direction of the viewfinder of the first interface as the recording direction; determining the recording focal length according to the recording direction and the ratio between the document border of the first document and the viewfinder of the first interface corresponding to the current direction.

[0012] Among them, by determining the recording focal length and the recording direction, the viewfinder of the first interface can have a high degree of fitness with the document border of the first document. Thus, the electronic device can obtain an image of the first document with high clarity, providing data preparation for obtaining pictures of the first document with high quality.

[0013] In combination with the first aspect, in certain implementations of the first aspect, images within the document border of the first document are detected to obtain a set of pictures of the first document, including: determining the document border of the first document in the first interface according to the recording focal length and the recording direction; obtaining the i-th image within the document border of the first document, where i is a positive integer taking values greater than or equal to 1; determining the content change amount between the i-th image and the (i - 1)-th image; when the content change amount between the i-th image and the (i - 1)-th image is greater than or equal to a preset change amount, adjusting the tilt angle of the i-th image; saving the picture corresponding to the adjusted i-th image in the picture set of the first document; updating i to i + 1, and continuing to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends; when the content change amount between the i-th image and the (i - 1)-th image is less than the preset change amount, updating i to i + 1, and continuing to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends.

[0014] In the above method, by means of OCR technology and / or layout analysis technology, it is possible to prevent duplicate saving of pictures corresponding to the same image of the first document.

[0015] In addition, adjusting the tilt angle of the i-th image can prepare data for saving pictures corresponding to the i-th image with high quality.

[0016] In combination with the first aspect, in certain implementations of the first aspect, determining the content change amount between the i-th image and the (i - 1)-th image includes:

[0017] Performing optical character recognition and / or layout analysis on the i-th image and the (i - 1)-th image to obtain the content detection situation of the i-th image and the (i - 1)-th image;

[0018] Determining the content change amount between the i-th image and the (i - 1)-th image according to the content detection situation of the i-th image and the (i - 1)-th image.

[0019] In the above method, by performing optical character recognition and / or layout analysis, it is possible to accurately identify the page-turning situation of the first document, and thus accurately avoid duplicate saving of pictures corresponding to the same image of the first document.

[0020] In combination with the first aspect, in certain implementations of the first aspect, images within the document border of the first document are detected to obtain a set of pictures of the first document, including determining the document border of the first document in the first interface according to the recording focal length and the recording direction; performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document, where i is a positive integer taking values greater than or equal to 1, and the qualities of the i-th images in the multiple optical flow trackings are different; when the number of successful optical flow trackings is greater than or equal to a preset number, adjusting the tilt angle of the i-th image corresponding to when the number of successful optical flow trackings is greater than or equal to the preset number; saving the picture corresponding to the adjusted i-th image in the set of pictures of the first document; when the ratio between the number of tracked feature points of the i-th image and the number of tracked feature points of the (i - 1)-th image is less than a preset threshold, updating i to i + 1, and continuing to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends; when the ratio between the number of tracked feature points of the i-th image and the number of tracked feature points of the (i - 1)-th image is greater than or equal to the preset threshold, continuing to execute the step of performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document until the number of successful optical flow trackings is greater than or equal to the preset number; when the number of successful optical flow trackings is less than the preset number, continuing to execute the step of performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document until the number of successful optical flow trackings is greater than or equal to the preset number; when the optical flow tracking is unsuccessful, continuing to execute the step of determining the document border of the first document in the first interface until the video recording ends.

[0021] The above method determines whether a high-quality image is obtained by means of whether the optical flow tracking reaches a preset number, thereby being able to prepare data for saving the picture corresponding to the high-quality image of the first document.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the above method specifically includes: performing the k-th optical flow tracking on the feature points of the i-th image within the document border of the first document, where k is a positive integer taking values greater than or equal to 1; determining whether the k-th optical flow tracking is successful; when the k-th optical flow tracking is successful, determining the number of successful optical flow trackings before the k-th optical flow tracking plus one as the number of successful optical flow trackings; when the number of successful optical flow trackings is greater than or equal to a preset number, adjusting the tilt angle of the i-th image in the k-th optical flow tracking; when the number of successful optical flow trackings is less than the preset number, updating k to k + 1, and continuing to execute the step of performing the k-th optical flow tracking on the feature points of the i-th image within the document border of the first document until the number of successful optical flow trackings is greater than or equal to the preset number; when the k-th optical flow tracking is unsuccessful, continuing to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends.

[0023] Through the above method, it is possible to prevent the pictures corresponding to the images of the first document with poor quality from being saved according to whether each optical flow tracking in multiple optical flow trackings is successful.

[0024] In combination with the first aspect, in some implementations of the first aspect, after saving the picture corresponding to the adjusted i-th image in the picture set of the first document, the method further includes: when the i-th image is the (i - 1)-th image, determining whether there is occlusion in the i-th image and / or the (i - 1)-th image; when there is occlusion in the i-th image and / or the (i - 1)-th image, saving the picture corresponding to the image with a smaller occlusion amount and deleting the picture corresponding to the image with a larger occlusion amount in the picture set of the first document according to the occlusion amount between the i-th image and the (i - 1)-th image; when there is no occlusion in both the i-th image and the (i - 1)-th image, saving the picture corresponding to the image with higher quality and deleting the picture corresponding to the image with poor quality in the picture set of the first document according to the quality of the i-th image and the (i - 1)-th image.

[0025] Through the above method, it is possible to delete the pictures corresponding to the images with a large occlusion amount or poor quality by the occlusion amount or quality, so as to retain the pictures corresponding to the images with a small occlusion amount or high quality. Furthermore, it is also possible to further avoid repeatedly saving the pictures corresponding to the same image of the first document.

[0026] In combination with the first aspect, in some implementations of the first aspect, adjusting the inclination angle of any one image includes: determining the positions of each vertex of the document border of the first document corresponding to the image in the first interface; determining the distances from each vertex of the document border of the first document corresponding to the image from the first interface to the presentation interface of the first document; adjusting the inclination angle of the image according to the positions of each vertex in the first interface and the distances from each vertex from the first interface to the presentation interface of the first document.

[0027] Through the above method, the inclination angle of the image is adjusted to ensure that the document border is parallel to the viewfinder, so that the text, patterns, and / or tables presented on the image are restored from the deformed state to the normal state. Thus, it is possible to provide data preparation for saving the pictures corresponding to the images with high quality.

[0028] In combination with the first aspect, in some implementations of the first aspect, the above method further includes: in response to a third operation on the first control, stopping the detection of the image within the document border of the first document.

[0029] Wherein, the first control is not only used to trigger the electronic device to obtain the first document, but also used to trigger the electronic device to stop obtaining the first document. Thus, the functions of triggering the electronic device to obtain the first document and triggering the electronic device to stop obtaining the first document can be realized through the first control.

[0030] In combination with the first aspect, in certain implementations of the first aspect, the above method further includes: displaying a third control in the first interface; starting video recording in response to a fourth operation on the third control; or stopping video recording in response to a fifth operation on the third control.

[0031] Wherein, the third control is used to trigger the electronic device to start video recording, and the third control is also used to trigger the electronic device to stop video recording. Thus, the functions of triggering the electronic device to start video recording and triggering the electronic device to stop video recording can be realized through the third control.

[0032] In combination with the first aspect, in certain implementations of the first aspect, generating a second document according to the picture set of the first document includes: displaying a second interface, where the second interface is used to display each picture in the picture set of the first document; determining a selected picture in the picture set of the first document in response to a sixth operation on the second interface; displaying a fourth control in the second interface; and generating a second document according to the selected picture in response to a seventh operation on the fourth control.

[0033] Wherein, the fourth control is used to trigger the electronic device to generate a second document. After the video recording ends, the electronic device can select pictures in the picture set of the first document and generate a second document in response to an operation on the fourth control, which simplifies the user's operation, improves the efficiency of obtaining the second document, and improves the usage experience of the electronic device.

[0034] In addition, the order of the pictures in the generated second document can be the order of the pictures selected in the picture set of the first document, or the chronological order of obtaining the pictures in the picture set of the first document.

[0035] In combination with the first aspect, in certain implementations of the first aspect, before generating a second document according to the selected picture in response to a seventh operation on the fourth control, the above method further includes: displaying a fifth control in the second interface; and performing at least one of image enhancement, image shadow removal, or image grayscale conversion on the selected picture in response to an eighth operation on the fifth control.

[0036] Wherein, the fifth control is used to trigger the electronic device to perform at least one of image enhancement, image shadow removal, or image grayscale conversion on the selected picture. Thus, the electronic device can adjust the selected picture according to the user's needs, reflecting the diversified services of the electronic device.

[0037] In a second aspect, the present application provides a document processing device, which includes one or more modules for executing the document processing method in the first aspect and any possible design of the first aspect.

[0038] In a third aspect, the present application provides an electronic device, including a processor; when the processor executes computer code or instructions in a memory, the electronic device is caused to execute the document processing method in the first aspect and any possible design of the first aspect.

[0039] In a fourth aspect, the present application provides an electronic device, including a processor and a memory; the memory is coupled to the processor, and the memory is used to store computer program code, the computer program code includes computer code or instructions, and when the processor executes the computer code or instructions, the electronic device is caused to execute the document processing method in the first aspect and any possible design of the first aspect.

[0040] In a fifth aspect, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen, and a sensor; the chip system includes: one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected through lines; the interface circuits are used to receive signals from the memory and send signals to the processors, and the signals include computer code or instructions stored in the memory; when the processors execute the computer code or instructions, the electronic device executes the document processing method in the first aspect and any possible design of the first aspect.

[0041] Among them, the chip system may include one chip or multiple chips; when the chip system includes multiple chips, the present application does not limit parameters such as the type and quantity of the chips.

[0042] In a sixth aspect, the present application provides a computer-readable storage medium, in which code or instructions are stored, and when the code or instructions run on an electronic device, the electronic device is caused to execute the document processing method in the first aspect and any possible design of the first aspect.

[0043] In a seventh aspect, the present application provides a computer program product, and when the computer program product runs on a computer, the computer is caused to execute the document processing method in the first aspect and any possible design of the first aspect.

[0044] It can be understood that the beneficial effects of the above second aspect to seventh aspect can be referred to the relevant descriptions in the above first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 Schematic diagram of a scenario of a document processing method provided by an embodiment of the present application;

[0047] Figure 2 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0048] Figures 3A - 3R Schematic diagram of a human-computer interaction interface provided by an embodiment of the present application;

[0049] Figure 4 Schematic flowchart of a document processing method provided by an embodiment of the present application;

[0050] Figure 5 Schematic flowchart of a document processing method provided by an embodiment of the present application;

[0051] Figure 6 Schematic flowchart of a document processing method provided by an embodiment of the present application;

[0052] Figure 7 Schematic flowchart of a document processing method provided by an embodiment of the present application;

[0053] Figure 8 Schematic flowchart of a document processing method provided by an embodiment of the present application;

[0054] Figure 9 Schematic flowchart of a document processing method provided by an embodiment of the present application. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B; herein, "and / or" is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0056] Hereinafter, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include one or more of such features.

[0057] The present application can provide a document processing method, a document processing device, an electronic device, a chip system, a computer-readable storage medium, and a computer program product. During video recording, when it is detected that a document is displayed on the electronic device, a control can be displayed. This control is used to trigger the electronic device to obtain the document, so as to remind the user through this control that the electronic device has the function of being able to obtain the document. After receiving an operation on the control, the recording focal length is adjusted, and the recording focal length can be determined, so that the clarity of the image of the document presented on the electronic device can be ensured through this recording focal length. Furthermore, the image within the document border of the document is detected, and one or more pictures of the document can be obtained. After the video recording ends, a document corresponding to the document can be automatically generated according to the one or more pictures of the document.

[0058] Thus, a document with high-quality content is obtained, the user's operation is simplified, and the user's experience is improved.

[0059] Among them, the document processing method provided by the present application can be applied to an electronic device. The electronic device can be an electronic device with a display screen hardware and corresponding software support.

[0060] For example, the electronic device can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a home device, etc. The present application does not impose any restrictions on the specific type of the electronic device.

[0061] Among them, the document processing method of the present application is applicable to the recording of documents in the form of paper documents or electronic documents, etc. Among them, the format of the document can be a portable document format (PDF), a powerpoint (PPT), an editable text file (word), or a long picture, etc. The document corresponding to the document can be the same as or different from the format of the document.

[0062] For the sake of convenience of description, in the present application, the case where an electronic document is displayed on a display device is taken as an example for illustration. Among them, the display device can be an electronic device with a display screen hardware and corresponding software support. For example, the display device can be a projector, a television, a mobile phone, a tablet computer, a notebook computer, a netbook, etc.

[0063] Please refer to Figure 1 , Figure 1 which shows a schematic diagram of a scenario of a document processing method provided by an embodiment of the present application.

[0064] AsFigure 1 As shown, the display device 10 can display a presentation interface of Document 1, and Document 1 can be displayed horizontally or vertically in the presentation interface of Document 1.

[0065] When Document 1 is displayed horizontally in the presentation interface of Document 1, the long side of Document 1 is in the horizontal direction, and the short side of Document 1 is in the vertical direction perpendicular to the horizontal direction. At this time, the presentation direction of Document 1 is the horizontal screen direction.

[0066] When Document 1 is displayed vertically in the presentation interface of Document 1, the short side of Document 1 is in the horizontal direction, and the long side of Document 1 is in the vertical direction perpendicular to the horizontal direction. At this time, the presentation direction of Document 1 is the vertical screen direction.

[0067] It should be understood that the presentation direction of Document 1 may include but is not limited to the specific implementation manners described above.

[0068] Continuing to combine Figure 1 , the electronic device 20 can record a video of Document 1 displayed on the display device 10 along the recording direction of the electronic device 20, and can display the image of the recorded video in a display interface. Thus, the electronic device 20 can execute the document processing method of the present application to obtain the content of Document 1.

[0069] When the user holds or places the electronic device 20 in the horizontal screen direction, the long side of the electronic device 20 is in the horizontal direction, and the short side of the electronic device 20 is in the vertical direction perpendicular to the horizontal direction.

[0070] Based on the foregoing description, the recording direction of the electronic device 20 can generally be regarded as the horizontal screen direction. In addition, the electronic device 20 can also adjust the recording direction of the electronic device 20 to the vertical screen direction.

[0071] When the user holds or places the electronic device 20 in the vertical screen direction, the short side of the electronic device 20 is in the horizontal direction, and the long side of the electronic device 20 is in the vertical direction perpendicular to the horizontal direction.

[0072] Based on the foregoing description, the recording direction of the electronic device 20 can generally be regarded as the vertical screen direction. In addition, the electronic device 20 can also adjust the recording direction of the electronic device 20 to the horizontal screen direction.

[0073] It should be understood that in addition to the implementation manners described in the above two cases, in the above two cases, the recording direction of the electronic device 20 can also be reversed. It can be seen that the recording direction of the electronic device 20 has nothing to do with the horizontal or vertical screen direction in which the electronic device 20 is held or placed.

[0074] The display interface of the electronic device 20 can include a viewfinder frame, and the viewfinder frame is used to display the image of the video recorded by the camera of the electronic device 20.

[0075] Among them, the direction of the viewfinder can generally be regarded as the recording direction of the electronic device 20. In other words, when the recording direction of the electronic device 20 is the landscape screen direction, the direction of the viewfinder is the landscape screen direction. When the recording direction of the electronic device 20 is the portrait screen direction, the direction of the viewfinder is the portrait screen direction.

[0076] It can be seen that the direction of the viewfinder generally remains consistent with the recording direction of the electronic device 20, that is, the direction of the viewfinder can generally represent the recording direction of the electronic device 20.

[0077] In addition, the electronic device 20 can also adjust the direction of the viewfinder according to needs. In other words, the direction of the viewfinder can also be inconsistent with the recording direction of the electronic device 20. When Document 1 appears in the display interface of the electronic device 20, the display interface of the electronic device can also include the document border of Document 1, and the document border of Document 1 is used to display the image of Document 1 in the image of the recorded video. In other words, when Document 1 is displayed in the display interface, the document border of Document 1 is included in the viewfinder.

[0078] Among them, the direction of the document border of Document 1 can generally be regarded as the presentation direction of Document 1. In other words, when the presentation direction of Document 1 is the landscape screen direction, the direction of the document border of Document 1 is the landscape screen direction. When the presentation direction of Document 1 is the portrait screen direction, the direction of the document border of Document 1 is the portrait screen direction.

[0079] It can be seen that the direction of the document border of Document 1 generally remains consistent with the presentation direction of Document 1, that is, the direction of the document border of Document 1 can generally represent the presentation direction of Document 1.

[0080] In addition, the electronic device 20 can also adjust the direction of the document border of Document 1 according to needs. In other words, the direction of the document border of Document 1 can also be inconsistent with the presentation direction of Document 1.

[0081] In addition, Document 1 can include multiple images / pages. The display device 10 can play from one image of Document 1 to another image / page of Document 1. Based on this, the foregoing process is called that there is page turning in Document 1.

[0082] For example, when Document 1 is a PPT-format document and the PPT-format document has multiple pages, the process of Document 1 turning from one page to another indicates that there is page turning in Document 1.

[0083] In addition, the lens of the electronic device 20 and / or the presentation interface of Document 1 can be blocked by an occluder such as a human body or other objects, so that the image of Document 1 in the display interface of the electronic device is blocked. Based on this, the foregoing process is called that there is occlusion in Document 1.

[0084] It should be understood that the above is an example illustration of the scenario and does not impose any limitation on the scenario of the present application.

[0085] Please refer to Figure 2 , Figure 2 which shows a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0086] As Figure 2 shown, taking the electronic device 20 as a mobile phone as an example, the electronic device 20 may include a processor 210, a memory 220, a communication module 230, a display screen 240, a camera 250, etc.

[0087] Among them, the processor 210 may include one or more processing units, and the memory 220 is used to store program codes and data. In the present application, the processor 210 can execute the computer execution instructions stored in the memory 220 to control and manage the actions of the electronic device 20.

[0088] The communication module 230 can be used for communication between various internal modules of the electronic device 20, or communication between the electronic device 20 and other external electronic devices, etc. Exemplarily, if the electronic device 20 communicates with other electronic devices in a wired connection manner, the communication module 230 may include an interface, such as a USB interface. The USB interface can be an interface that conforms to the USB standard specification, specifically a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge the electronic device 20, and can also be used to transfer data between the electronic device 20 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as augmented reality (AR) devices, etc.

[0089] Alternatively, the communication module 230 may include audio devices, radio frequency circuits, Bluetooth chips, wireless fidelity (Wi-Fi) chips, near-field communication (NFC) modules, etc., and can realize the interaction between the electronic device 20 and other electronic devices in a variety of different ways.

[0090] Among them, the display screen 240 can display images or videos in the human-computer interaction interface, etc.

[0091] The camera 250 can be used to capture images or record videos.

[0092] Optionally, the electronic device 20 may further include peripheral devices 260, such as a mouse, a keyboard, a speaker, a microphone, etc.

[0093] It should be understood that, except for the various components or modules listed in Figure 2 , the structure of the electronic device 20 is not specifically limited in this application.

[0094] In addition, the electronic device 20 may further include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0095] Based on the foregoing description, in combination with Figures 3A - 3R , the document processing method of this application implemented by the electronic device will be introduced in detail. For the sake of convenience of explanation, Figures 3A - 3R , taking the electronic device as a mobile phone, the recording direction of the mobile phone is the portrait direction, and the presentation direction of the document 1 is the landscape direction as an example for illustration.

[0096] Please refer to Figures 3A - 3R , Figures 3A - 3R which shows a schematic diagram of a human-computer interaction interface provided by an embodiment of this application.

[0097] The mobile phone can display an interface 21 as shown in Figure 3A . The interface 21 is used to display the icons of various applications (APPs) installed on the mobile phone. For example, the interface 21 may include: a control 2001, a control 2002, and a control 2003. Among them, the control 2001 is used to display the icon of the camera APP. The control 2002 is used to display the icon of the album APP. The control 2003 is used to display the icon of the file management APP.

[0098] After receiving an operation (such as a click, double-click, or long-press operation, etc.) triggered by the user on the control 2001 as shown in Figure 3A , the mobile phone can change from the interface 21 as shown in Figure 3A to display an interface 22 as shown in Figure 3B . The interface 22 is used to display the picture of the recorded video.

[0099] Among them, the interface 22 may include: a viewfinder 201, a preview frame 202, a control 203, and a control 204. The viewfinder 201 is used to display the video during the recording process. The preview frame 202 is used to preview the pictures stored in the mobile phone. The control 203 is used to display the recording focal length. The control 204 is used to trigger the mobile phone to start or stop video recording. Among them, the control 203 can be displayed inside the viewfinder 201, and the preview frame 202 and the control 204 can be displayed outside the viewfinder 201.

[0100] After receiving an operation triggered by the user on the control 204 as shown in Figure 3B , the mobile phone starts video recording. Thus, the mobile phone can change from displaying the interface 22 as shown in Figure 3B to display asFigure 3C The interface 22 shown Figure 3C In it, the recorded focal length displayed by the control 203 is 1x. The direction of the viewfinder frame 201 can be set to be consistent with the recording direction of the mobile phone, that is, the direction of the viewfinder frame 201 is the portrait screen direction.

[0101] Document 1 has not started playing, and the viewfinder frame 201 does not display Document 1. After Document 1 starts playing, Document 1 is displayed in the interface 22, and the viewfinder frame 201 changes from never displaying Document 1 to displaying Document 1. Thus, the mobile phone can change from displaying the interface 22 as Figure 3C shown to displaying the interface 22 as Figure 3D shown.

[0102] Figure 3D In it, the interface 22 may further include: a control 205, and the control 205 is used to trigger the mobile phone to start obtaining Document 1.

[0103] In addition, after receiving an operation in which the user triggers the control 204 as Figure 3B shown, the mobile phone starts video recording, and the mobile phone determines that Document 1 is displayed in the viewfinder frame 201. Thus, the mobile phone can change from displaying the interface 22 as Figure 3B shown to displaying the interface 22 as Figure 3D shown.

[0104] After receiving an operation in which the user triggers the control 205 as Figure 3D shown, the mobile phone starts obtaining Document 1. Thus, the mobile phone can change from displaying the interface 22 as Figure 3D shown to displaying the interface 22 as Figure 3E shown.

[0105] Figure 3E In it, the interface 22 may further include: a document border 206, and the document border 206 is used to display Image 1 of Document 1. The direction of the document border 206 can be set to be consistent with the presentation direction of Document 1, that is, the direction of the document border 206 is the landscape screen direction.

[0106] In addition, the interface 22 as Figure 3E shown may further include a prompt message (such as "detecting document..."), and the prompt message is used to prompt the user that the mobile phone is obtaining Document 1.

[0107] It can be seen that the direction of the viewfinder frame 201 is the portrait screen direction, and the direction of the document border 206 is the landscape screen direction. The mobile phone can determine that the direction of the viewfinder frame 201 is inconsistent with the direction of the document border 206. Based on this, Figure 3EIn [the above], the interface 22 may further include a control 207, and the control 207 is used to adjust the orientation of the viewfinder 201. Thus, through the control 207, an entry for whether to adjust the orientation of the viewfinder 201 can be provided to the user, which is conducive to adaptively adjusting the orientation of the viewfinder 201 according to the user's needs.

[0108] After receiving an operation in which the user triggers the Figure 3E shown control 207, the mobile phone adjusts the orientation of the viewfinder 201 from the portrait orientation to the landscape orientation, so that the orientation of the viewfinder 201 is consistent with the orientation of the document border 206. Thus, the mobile phone can change from displaying the Figure 3E shown interface 22 to displaying the Figure 3F shown interface 22.

[0109] In addition, in the interface 22, the control 207 is further used to cancel the adjustment of the orientation of the viewfinder 201, so that the orientation of the viewfinder 201 is restored from the adjusted landscape orientation to the unadjusted portrait orientation.

[0110] It can be seen that the orientation of the viewfinder 201 becomes the landscape orientation, the orientation of the document border 206 is the landscape orientation, and the orientation of the viewfinder 201 is consistent with the orientation of the document border 206. Based on this, the mobile phone can adjust the recording focal length according to the area ratio between the document border 206 and the viewfinder 201, so that the image 1 located within the document border 206 becomes clearer. Thus, the mobile phone can change from displaying the Figure 3F shown interface 22 to displaying the Figure 3G shown interface 22. Figure 3G In [the above], the recording focal length displayed in the control 203 changes from 1x to 3x.

[0111] Among them, A, B, C, and D are used to represent the four vertices of the document border 206.

[0112] In addition, the mobile phone can also adjust the orientation of the viewfinder 201 and the recording focal length simultaneously. Thus, after receiving an operation in which the user triggers the Figure 3E shown control 207, the orientation of the viewfinder 201 and the recording focal length displayed in the control 203 change simultaneously, and the mobile phone can change from displaying the Figure 3E shown interface 22 to displaying the Figure 3G shown interface 22.

[0113] Or, after a duration of 1 has elapsed without receiving an operation in which the user Figure 3EAfter the operation triggered by the control 207 shown, the orientation of the viewfinder 201 remains in the portrait orientation, while the orientation of the document border 206 is in the landscape orientation. The orientation of the viewfinder 201 is inconsistent with that of the document border 206. Based on this, the mobile phone can adjust the recording focal length according to the area ratio between the document border 206 and the viewfinder 201, so that the image 1 located within the document border 206 becomes clear. For example, the mobile phone can change the recording focal length from 1x to 2.5x.

[0114] Among them, the present application does not limit the specific value of the duration 1.

[0115] It should be understood that in addition to the above situation, when the presentation orientation of the document 1 remains in the landscape orientation and the recording orientation of the mobile phone changes to the landscape orientation, the orientation of the viewfinder 201 can be consistent with the recording orientation of the mobile phone, that is, the orientation of the viewfinder 201 is in the landscape orientation. It can be seen that the orientation of the viewfinder 201 is consistent with that of the document border 206. Based on this, the mobile phone can adjust the recording focal length according to the area ratio between the document border 206 and the viewfinder 201, so that the image 1 located within the document border 206 becomes clear. For example, the mobile phone can change the recording focal length from 1x to 2.5x.

[0116] In addition, in addition to the method of adjusting the orientation of the viewfinder 201, the mobile phone can also adjust the orientation of the document border 206 so that the orientation of the viewfinder 201 is consistent with that of the document border 206.

[0117] For the sake of convenience of explanation, an example is given with the recording focal length being 3x and the orientation of the viewfinder 201 being in the landscape orientation.

[0118] The mobile phone detects the image 1 located within the document border 206 and can save the image 1 located within the document border 206. Thus, the mobile phone can change from displaying the interface 22 as shown in Figure 3G to displaying the interface 22 as shown in Figure 3H In the interface 22 shown. Figure 3H In this case, the preview frame 202 changes from a blank display to a thumbnail of the picture 1 corresponding to the image 1.

[0119] During the saving process of the image 1, the mobile phone can correct the image 1 based on the four vertices A, B, C, and D and save the corrected image 1.

[0120] As the document 1 is turned page by page, the document 1 in the interface 22 can change from the image 1 to the image 2. Thus, the mobile phone can detect and save the image 2. The mobile phone can change from displaying the interface 22 as shown in Figure 3H to displaying the interface 22 as shown in Figure 3I In the interface 22 shown. Figure 3I In this case, the preview frame 202 changes from the thumbnail of the picture 1 to the thumbnail of the picture 2 corresponding to the image 2.

[0121] As the document 1 is paged, the document 1 in the interface 22 can change from image 2 to image 3. Thus, the mobile phone can detect and save image 3. The mobile phone can change from displaying the interface 22 as shown in Figure 3I to displaying the interface 22 as shown in Figure 3J . Figure 3J In it, the preview box 202 changes from the thumbnail of picture 2 to the thumbnail of picture 3 corresponding to image 3.

[0122] After receiving the operation of the user triggering the control 205 as shown in Figure 3J , the mobile phone can stop obtaining the document 1. Thus, the mobile phone can change from displaying the interface 22 as shown in Figure 3J to displaying the interface 22 as shown in Figure 3K .

[0123] Figure 3K In it, the mobile phone is still recording video, and the mobile phone has stopped obtaining the document 1. Therefore, as the document 1 is paged, the document 1 in the interface 22 can change from image 3 to image 4, and the mobile phone will not obtain and save image 4, and the preview box 202 still displays the thumbnail of picture 3.

[0124] After receiving the operation of the user triggering the control 204 as shown in Figure 3K , the mobile phone can stop video recording. Thus, the mobile phone can change from displaying the interface 22 as shown in Figure 3K to displaying the interface 22 as shown in Figure 3L . Figure 3L In it, the interface 22 no longer displays the video recording screen.

[0125] In some embodiments, after stopping video recording, the mobile phone can automatically generate a document according to the saved pictures 1, 2, and 3.

[0126] In other embodiments, after stopping video recording, the mobile phone can, in response to the operation of the user, generate a document according to the saved pictures 1, 2, and 3.

[0127] For example, after receiving the operation of the user triggering the control 2002 as shown in Figure 3A , the mobile phone can change from the interface 21 as shown in Figure 3A to displaying the interface 23 as shown in Figure 3M .

[0128] Alternatively, after receiving the operation of the user triggering the preview box 202 in the interface 22, the mobile phone can change from the interface 21 as shown in Figure 3L to displaying the interface 23 as shown in Figure 3M .

[0129] Alternatively, after receiving the operation of the user on Figure 3AAfter the operation triggered by the control 2003 shown, the mobile phone can change from the interface 21 as shown in Figure 3A to the interface 23 as shown in Figure 3M .

[0130] Figure 3M In , the interface 23 is used to display the saved Picture 1, Picture 2, and Picture 3. The interface 23 may include: control 2301, control 2302, control 2303, control 208, and control 209. Among them, control 2301 is used to display Picture 1. Control 2302 is used to display Picture 2. Control 2303 is used to display Picture 3. Control 208 is used to edit pictures. Control 209 is used to generate a document from the pictures.

[0131] After receiving the operation triggered by the user on the Figure 3N control 2301, control 2302, and control 2303 shown, the mobile phone can change from the interface 23 as shown in Figure 3M to the interface 23 as shown in Figure 3N . In the interface 23, the selected states of the pictures are shown in control 2301, control 2302, and control 2303. At this time, all three pictures are selected.

[0132] After receiving the operation triggered by the user on the Figure 3N control 208 shown, the mobile phone can change from the interface 23 as shown in Figure 3N to the interface 23 as shown in Figure 3O . Figure 3O In , the interface 23 may further include: enhancement control 2081, shadow removal control 2082, grayscale control 2083, and select all control 2084. Among them, the enhancement control 2081 is used to brighten and / or sharpen the selected picture. The shadow removal control 2082 is used to remove the shadow from the selected picture. The grayscale control 2083 is used to grayscale the selected picture. The select all control 2084 is used to perform image enhancement, image shadow removal, and image grayscaling on the selected pictures.

[0133] After receiving the operation triggered by the user on the Figure 3O select all control 2084 shown, the mobile phone can perform image enhancement, image shadow removal, and image grayscaling on the selected pictures. Thus, the mobile phone can change from the interface 23 as shown in Figure 3O to the interface 23 as shown in Figure 3N . Figure 3N In , the interface 23 can display the three edited pictures.

[0134] After receiving the operation triggered by the user on the Figure 3N control 209 shown, the mobile phone can change from the interface 23 as shown in Figure 3NThe interface 23 shown becomes the display as Figure 3P the interface 23 shown. Figure 3P In, the interface 23 may further include: a PDF control 2091, a PPT control 2092, a word control 2093, and a long picture control 2094. Among them, the PDF control 2091 is used to generate a PDF - format document from the selected pictures. The PPT control 2092 is used to generate a PPT - format document from the selected pictures. The word control 2093 is used to generate a word - format document from the selected pictures. The long picture control 2094 is used to generate a long picture - format document from the selected pictures.

[0135] After receiving an operation by the user to trigger Figure 3P the PDF control 2091 shown, the mobile phone can generate a PDF - format document 2 from the three selected pictures. Thus, the mobile phone can save the document 2 in the file management APP.

[0136] After receiving an operation by the user to trigger Figure 3A the control 2003 shown, the mobile phone can change from the interface 21 shown as Figure 3A to the display as Figure 3Q the interface 24 shown. Figure 3Q In, the interface 24 is used to display the document 2. The interface 24 may include: a control 2401. The control 2401 is used to trigger the display of folder 1, and folder 1 may include: picture 1, picture 2, picture 3, and document 2; or, document 2.

[0137] After receiving an operation by the user to trigger Figure 3Q the control 2401 shown, the mobile phone can change from the interface 24 shown as Figure 3Q to the display as Figure 3R the interface 24 shown. Figure 3R In the interface 24 may further include: document 2.

[0138] In summary, during the video recording process, the mobile phone can obtain the image of document 1 and save the image of document 1. Thus, the mobile phone generates document 2 based on the saved image of document 1, enabling the user to quickly obtain the content of document 1.

[0139] Based on the above Figures 3A - 3R description shown, when the electronic device implements the document processing method, it can adopt various methods.

[0140] Next, in combination with Figures 4 - 5 , the specific implementation process of the document processing method of this application will be introduced in detail.

[0141] Figure 4 is a schematic flowchart of an example document processing method provided by this application.

[0142] As Figure 4 shown, the method can be divided into three different stages, namely 1.1 Determination Stage, 1.2 Duplication Avoidance Stage, and 1.3 Saving Stage.

[0143] The 1.1 Determination Stage can be understood as the stage of determining the clarity and position of the image of Document 1.

[0144] The 1.2 Duplication Avoidance Stage can be understood as the stage of avoiding duplicate saving of pictures corresponding to the same image of Document 1.

[0145] The 1.3 Saving Stage can be understood as the stage of saving the pictures corresponding to the images of Document 1.

[0146] As Figure 4 shown, the document processing method of the present application may include:

[0147] S11. Start video recording.

[0148] The electronic device can display the video during recording in a display interface. The present application does not limit the specific implementation manner of the display interface.

[0149] In some embodiments, the above display interface may include a video recording control, and the video recording control is used to trigger the electronic device to start video recording. Among them, the present application does not limit parameters such as the display position, display style, and display size of the video recording control.

[0150] Thus, after receiving an operation on the video recording control, the electronic device can start video recording.

[0151] 【1.1 Determination Stage】

[0152] S12. When Document 1 is detected, display a document scanning control.

[0153] During the recording of the video, as Document 1 is presented on the display interface and the image of Document 1 is displayed on the above display interface, at this time, the electronic device can detect Document 1.

[0154] Among them, when Document 1 is a paper document, the presentation interface of Document 1 is the page of the paper document. When Document 1 is an electronic document, the presentation interface of Document 1 is the playback interface of the display device for playing the electronic document.

[0155] Thus, when the electronic device detects Document 1, it can display a document scanning control in the above display interface. The present application does not specifically limit parameters such as the display position, display style, and display size of the document scanning control.

[0156] In some embodiments, the document scanning control is used to trigger the electronic device to start obtaining Document 1.

[0157] Thus, the electronic device provides the function of the electronic device obtaining Document 1 to the user through the document scanning control.

[0158] S13. When receiving an operation of the document scanning control, determine the recording focal length and the recording direction.

[0159] After the electronic device receives an operation on the document scanning control, it can determine that the user wants to obtain Document 1. Thus, the electronic device can start obtaining Document 1. After starting to obtain Document 1, the electronic device can determine the recording focal length to ensure obtaining a high-clarity Document 1.

[0160] Among them, the recording focal length is the distance from the lens of the electronic device to the presentation interface of Document 1. When Document 1 is a paper document, the recording focal length is the distance from the lens of the electronic device to the page of the paper document. When Document 1 is an electronic document, the recording focal length is the distance from the lens of the electronic device to the playback interface of the electronic document.

[0161] Among them, when the electronic device is relatively far from the presentation interface of Document 1, the smaller the recording focal length, the less clear the image of Document 1 displayed in the presentation interface. The larger the recording focal length, the clearer the image of Document 1 displayed in the presentation interface.

[0162] Thus, the electronic device adjusts the recording focal length according to the preset focal range to determine the recording focal length, so that the size of the viewfinder frame in the display interface is adapted to the size of the document border of Document 1 in the display interface, facilitating obtaining a high-clarity Document 1.

[0163] Among them, the viewfinder frame in the display interface is used to display the image of the video recorded by the electronic device. The document border of Document 1 is used to display the image of Document 1 in the image of the recorded video.

[0164] Among them, the above size can include at least one of area, length, width, and perimeter.

[0165] Among them, the preset focal range is the focal length change range of the lens of the electronic device.

[0166] In addition, after starting to obtain Document 1, the electronic device can also determine the recording direction, enabling the electronic device to more accurately adjust the recording focal length.

[0167] Among them, the recording direction is used to indicate that the direction of the viewfinder frame in the display interface is the portrait orientation or the landscape orientation.

[0168] Thus, when the user wants to adjust the orientation of the viewfinder or the document border of Document 1, the electronic device adjusts the orientation of the viewfinder or the document border of Document 1, and can determine the recording orientation, so that the size of the viewfinder in the display interface fits better with the size of the document border of Document 1 in the display interface.

[0169] S14. Determine the document border of Document 1 according to the recording focal length and the recording orientation.

[0170] The electronic device can determine the document border of Document 1 in the above display interface according to the recording focal length and the recording orientation, so as to determine the position of the image of Document 1 in the above display interface.

[0171] Thus, the electronic device can obtain the image of Document 1 located within the document border according to the recording focal length and the recording orientation, so that the position of the obtained image of Document 1 is accurate and the clarity is high.

[0172] In summary, when the electronic device detects Document 1, it displays a document scanning control, provides the user with the function of the electronic device to obtain Document 1, makes it possible to obtain Document 1 during video recording, improves the user's operation experience, and ensures the diversified services of the electronic device.

[0173] By determining the recording focal length and the recording orientation, the electronic device makes the image of Document 1 located within the document border clearer, ensures that the electronic device obtains an image of Document 1 with high quality, and provides conditions for the case of obtaining Document 1 at a long distance.

[0174] 【1.2 Avoiding repetition stage】

[0175] S15. Obtain the image i located within the document border.

[0176] When the electronic device determines the document border of Document 1, it can determine the position of the image i of Document 1 in the above display interface. Thus, the electronic device can obtain the image i according to the position of the image i of Document 1.

[0177] Among them, the image i is used to represent a frame image of Document 1 located within the document border. i is a positive integer that takes all values greater than or equal to 1.

[0178] S16. Perform optical character recognition and / or layout analysis on the image i.

[0179] After the electronic device obtains the image i located within the document border, it can obtain the number and position of characters and the number and position of tables in the image i through optical character recognition (OCR) technology.

[0180] And / or, after the electronic device obtains the image i located within the document border, it can use layout analysis technology to obtain the bending degree of the patterns, paragraphs, and text lines in the image i.

[0181] S17. Determine whether the character change and / or layout change between the image i and the image i-1 is greater than or equal to a threshold 1.

[0182] The image i-1 is used to represent a frame of the document 1 located within the document border before the image i.

[0183] The electronic device can obtain the character change between the image i and the image i-1 based on the number and position of characters, and the number and position of tables in the image i, as well as the number and position of characters, and the number and position of tables in the image i-1. Among them, the character change between the image i and the image i-1 is used to represent the character change situation between the image i and the image i-1.

[0184] And / or, the electronic device can obtain the layout change between the image i and the image i-1 based on the bending degree of the patterns, paragraphs, and text lines in the image i, as well as the bending degree of the patterns, paragraphs, and text lines in the image i-1. Among them, the layout change between the image i and the image i-1 is used to represent the layout change situation between the image i and the image i-1.

[0185] When i = 1, the image 1 is the first frame of the document 1 obtained by the electronic device. Thus, the electronic device does not need to execute S17 and can directly execute S18. Or, the electronic device can determine that a frame of the image before the image 1 is empty, then the electronic device can determine that the character change and / or layout change between the image 1 and the empty image is greater than or equal to the threshold 1, and can execute S18.

[0186] When i ≥ 2, the electronic device can determine whether the character change and / or layout change between the image i and the image i-1 is greater than or equal to the threshold 1.

[0187] Among them, the threshold 1 is preset, and the threshold 1 can be used to prevent the same image of the document 1 located within the document border from all entering the 1.3 saving stage, and to prepare data to prevent duplicate saving.

[0188] When the character change and / or layout change between the image i and the image i-1 is greater than or equal to the threshold 1, the electronic device can determine that there is occlusion between any one of the image i-1 and the image i, that any one of the image i-1 and the image i is unclear due to factors such as poor lighting, or that the document 1 has been flipped from the image i-1 to the image i. Thus, the electronic device can execute S18, avoiding duplicate saving of the same image of the document 1.

[0189] When the character change and / or layout change between image i and image i-1 is less than threshold 1, the electronic device can determine that there is no occlusion or blurring in both image i-1 and image i, and Document 1 has not been flipped from image i-1 to image i. Thus, the electronic device can execute S14 again, that is, determine the document border of Document 1 according to the recording focal length and recording direction, so as to facilitate the acquisition and saving of the image after image i.

[0190] Among them, when the electronic device determines whether the character change and layout change between image i and image i-1 are greater than or equal to threshold 1, the execution order of judging the character change and layout change is not in sequence, and can be executed sequentially or simultaneously.

[0191] In summary, with the help of OCR technology and / or layout analysis technology, the electronic device can prevent the same image of Document 1 from entering the 1.3 saving stage. Thus, the electronic device makes data preparations to prevent duplicate saving of pictures corresponding to the same image of Document 1.

[0192] 【1.3 Saving Stage】

[0193] S18. Adjust the tilt angle of image i.

[0194] It should be understood that during the recording of the video, there may be a situation where the distances between the electronic device and each position of the presentation interface of Document 1 are not completely the same. Such a situation may cause the image i located within the document border to be tilted, resulting in the text, patterns, and / or tables presented on image i being deformed.

[0195] When the electronic device determines that the character change and / or layout change between image i and image i-1 is greater than or equal to threshold 1, it adjusts the tilt angle of image i so that the text, patterns, and / or tables presented on image i are restored from the deformed state to the normal state, making data preparations for saving a high-quality picture corresponding to image i.

[0196] Among them, the tilt angle represents the degree of tilt of image i located within the document border.

[0197] S19. Save the picture corresponding to the adjusted image i.

[0198] The electronic device can save the picture corresponding to the adjusted image i in the electronic device and / or a storage device communicating with the electronic device by means of interception, etc. Among them, the storage method and specific type of the storage device in this application are not limited.

[0199] S20. When image i is image i-1, delete the pictures corresponding to the images of poor quality.

[0200] Among them, S20 is an optional step.

[0201] It should be understood that when there is occlusion in any one of the images in image i-1 and image i, or when any one of the images in image i-1 and image i is unclear due to factors such as poor lighting, both the image i-1 and image i of Document 1 can enter the 1.3 saving stage. At this time, there is a situation where pictures corresponding to the same image of Document 1 are saved repeatedly.

[0202] When i = 1, image 1 is the first frame image of Document 1 obtained by the electronic device. Thus, the electronic device does not need to execute S20 and can directly execute S21. Alternatively, if the electronic device determines that a frame of image before image 1 is empty, the electronic device can determine that image 1 and the empty image are not the same image and can execute S21.

[0203] When i ≥ 2, the electronic device can determine whether image i is image i-1.

[0204] In some embodiments, when the similarity value between image i and image i-1 is higher than the similarity threshold, the electronic device can determine that image i is image i-1.

[0205] Thus, when image i is image i-1, the electronic device can delete the pictures corresponding to the images with poor quality to save the pictures corresponding to the images with high quality, avoiding repeated saving.

[0206] S21: Update i to i + 1.

[0207] After the electronic device executes S20, the process of obtaining and saving image i is completed. Thus, the electronic device can continue to obtain and save the frame of image after image i, that is, the electronic device can execute S21 to update i to i + 1.

[0208] Before stopping video recording, the electronic device can repeatedly execute S14 - S21 until video recording stops or obtaining Document 1 stops during video recording.

[0209] In summary, the electronic device adjusts the tilt angle of image i to restore image i from the deformed state to the normal state, and saves the pictures corresponding to image i restored to the normal state, facilitating the acquisition of pictures with high quality.

[0210] When image i is image i-1, the electronic device deletes the pictures corresponding to the images with poor quality. Through this step, it is possible to further avoid repeated saving of pictures corresponding to the same image of Document 1, and the electronic device can save the pictures corresponding to the images with high quality.

[0211] S22: Stop video recording.

[0212] This application does not limit the specific implementation manner of stopping video recording.

[0213] In some embodiments, the video recording control in S11 is further used to trigger the electronic device to stop video recording. When the electronic device receives the operation of the video recording control, it can stop video recording. Thus, the electronic device can no longer continue to obtain Document 1.

[0214] In other embodiments, the display interface can also display an end video recording control, and the end video recording control is used to trigger the electronic device to stop video recording. When the electronic device receives the operation of the end video recording control, it can stop video recording.

[0215] In addition, the electronic device can also stop obtaining Document 1 before stopping video recording, that is, stop obtaining Document 1 during the video recording process. When stopping obtaining Document 1 during the video recording process, the electronic device is still recording the video but has stopped obtaining Document 1. Therefore, as Document 1 is flipped, the electronic device will not obtain and save the next frame image of Document 1.

[0216] Among them, the present application does not limit the specific implementation manner of stopping obtaining Document 1.

[0217] In some embodiments, the document scanning control in S12 is further used to trigger the electronic device to stop obtaining Document 1. When the electronic device receives the operation of the document scanning control, it can stop obtaining Document 1.

[0218] In other embodiments, the display interface can also display a cancel document scanning control, and the cancel document scanning control is used to trigger the electronic device to stop obtaining Document 1. When the electronic device receives the operation of the cancel document scanning control, it can stop obtaining Document 1.

[0219] It can be seen that the electronic device can stop obtaining Document 1 in various ways, reflecting the diverse services of the electronic device.

[0220] In summary, during the video recording process of the electronic device, when Document 1 is detected, the document scanning control is displayed to facilitate providing the function of obtaining Document 1 to the user. Thus, the electronic device can obtain Document 1 in response to the operation of the document scanning control, simplifying the user's operation and improving the user experience. The electronic device uses OCR technology and / or layout analysis to avoid the same image of Document 1 from entering the saving stage. In addition, when the electronic device has saved the picture corresponding to the same image of Document 1, it can delete the picture corresponding to the image with poor quality, avoiding the repeated saving of the picture corresponding to the same image of Document 1 again and enabling the electronic device to save high-quality pictures.

[0221] Based on Figure 4 the embodiments shown, the electronic device can more accurately identify the page-turning situation of Document 1 by means of optical character recognition and / or layout analysis, and can better avoid repeated saving of pictures corresponding to the same image. It can be seen thatFigure 4 The illustrated embodiment is applicable to a scenario where an electronic device can accurately avoid duplicate saving of the same image in Document 1.

[0222] Figure 5 It is a schematic flowchart of a document processing method provided by this application.

[0223] As Figure 5 shown, this method can be divided into three different stages, namely 2.1 Determination Stage, 2.2 Quality Assurance Stage, and 2.3 Saving Stage.

[0224] The 2.1 Determination Stage can be understood as the stage of determining the clarity and position of the image of Document 1.

[0225] The 2.2 Quality Assurance Stage can be understood as the stage of ensuring the quality of the image of Document 1.

[0226] The 2.3 Saving Stage can be understood as the stage of saving the picture corresponding to the image of Document 1.

[0227] As Figure 5 shown, the document processing method of this application may include:

[0228] S31. Start video recording.

[0229] 【2.1 Determination Stage】

[0230] S32. When Document 1 is detected, display a document scanning control.

[0231] S33. When an operation on the document scanning control is received, determine the recording focal length and recording direction.

[0232] S34. According to the recording focal length and recording direction, determine the document border of Document 1.

[0233] Among them, S31, S32, S33, and S34 are respectively similar in implementation manner to S11, S12, S13, and S14 in the Figure 4 illustrated embodiment, and will not be elaborated here.

[0234] 【2.2 Quality Assurance Stage】

[0235] S35. Set the optical flow tracking count to 0.

[0236] Before the electronic device performs optical flow tracking on the feature points of the image i located within the document border, the optical flow tracking count can be set to 0 to facilitate accurate counting of the optical flow tracking count during the optical flow tracking process.

[0237] S36. Perform optical flow tracking on the feature points of the image i located within the document border.

[0238] S37. Determine whether the current optical flow tracking is successful.

[0239] S38. Increment the optical flow tracking count by 1.

[0240] S39. Determine whether the optical flow tracking count is greater than or equal to threshold 2.

[0241] Considering that the optical flow tracking is relatively fast, the electronic device can perform multiple optical flow trackings on the feature points of image i through the optical flow tracking algorithm. Among them, the feature points of image i for each optical flow tracking may be the same or different, so that the quality of image i for each optical flow tracking may be the same or different.

[0242] Based on the above description, the electronic device can regard any one of the multiple optical flow trackings as the current optical flow tracking. Thus, the electronic device can obtain the feature points of image i for the current optical flow tracking, and determine whether the current optical flow tracking is successful according to whether the number of feature points of image i for the current optical flow tracking is greater than or equal to the feature point threshold.

[0243] Among them, the feature point threshold is preset in advance. With the help of the feature point threshold, it is possible to prevent images with poor quality from entering the 2.3 saving stage.

[0244] For example, during multiple optical flow trackings, when determining whether the current optical flow tracking is successful, the electronic device can use 80% of the number of feature points of image i in the previous optical flow tracking as the feature point threshold for determining whether the current optical flow tracking is successful.

[0245] During multiple optical flow trackings, for m1, the actual measurement is 25; the threshold is 25 * 80% = 20, for m2, the actual measurement is 20, the optical flow tracking is successful, for m2, the actual measurement is 5, the optical flow tracking is unsuccessful.

[0246] If so, the electronic device can determine that the current optical flow tracking is successful. Thus, the electronic device can execute S38, that is, increment the optical flow tracking count by 1.

[0247] If not, the electronic device can determine that the quality of image i for the current optical flow tracking is poor. Thus, the electronic device can execute S34 to make the electronic device perform optical flow tracking on image i again.

[0248] For example, the number of feature points of image i for the current optical flow tracking is 25, and the feature point threshold is 20. When the number of feature points of the image for the current optical flow tracking is greater than or equal to 20, such as 24, the electronic device can determine that the current optical flow tracking is successful; when it is less than the feature point threshold, such as 5, indicating that 80% of the feature points have been lost, the electronic device can determine that the current optical flow tracking is unsuccessful.

[0249] In S38, when the electronic device determines that the current optical flow tracking is successful, it can increment by 1 the number of optical flow tracking times before the current optical flow tracking, and determine it as the optical flow tracking times.

[0250] During the process of the electronic device performing multiple optical flow trackings, it can determine whether the accumulated number of optical flow tracking times in S38 is greater than or equal to threshold 2.

[0251] Among them, threshold 2 can be set according to actual requirements. For example, threshold 2 can be set to 5 times.

[0252] When the number of optical flow tracking times is greater than or equal to threshold 2, the electronic device can continuously and stably perform optical flow tracking on image i, so that the electronic device can perform optical flow tracking on a high-quality image i. Thus, the electronic device can enter the 2.3 saving stage.

[0253] When the number of optical flow tracking times is less than threshold 2, the electronic device has not continuously and stably performed optical flow tracking on image i. Thus, the electronic device can execute S36 and continue to perform optical flow tracking on the feature points of image i located within the document border.

[0254] For example, when document 1 is a PPT format document and threshold 2 is 5 times, after the electronic device successfully performs the first optical flow tracking on the image of the first page of this PPT format document located within the document border, it can increment the optical flow tracking times by one and continue tracking. After the electronic device successfully performs the second optical flow tracking on the image of the first page of this PPT format document, it increments the optical flow tracking times by one again and continues tracking. Until the number of optical flow tracking times reaches 5 times, the electronic device can determine that it has tracked an image of high quality on the first page of the PPT.

[0255] In summary, the electronic device can use the optical flow tracking technology to ensure that the high-quality image i of document 1 enters the 2.3 saving stage. Thus, the electronic device can prepare the data for saving the pictures corresponding to the high-quality images.

[0256] 【2.3 Saving Stage】

[0257] S40: Clear the optical flow tracking times to 0.

[0258] After the electronic device obtains a high-quality image i by using the optical flow tracking technology, it can also clear the optical flow tracking times to 0, which is convenient for the electronic device to accurately count the optical flow tracking times of the images located within the document border after image i.

[0259] S41: Adjust the tilt angle of image i corresponding to when the optical flow tracking times is greater than or equal to threshold 2.

[0260] Among them, S40 and S41 are not in a sequential order in terms of timing, and can be executed sequentially or simultaneously.

[0261] S42. Save the picture corresponding to the adjusted image i.

[0262] When the number of optical flow tracking times of the electronic device is greater than or equal to the threshold 2, the tilt angle of the image i corresponding to the optical flow tracking times greater than or equal to the threshold 2 can be adjusted. Thus, the electronic device saves the picture corresponding to the image i corresponding to the optical flow tracking times greater than or equal to the threshold 2.

[0263] Among them, the image i corresponding to the optical flow tracking times greater than or equal to the threshold 2 can be understood as any image i in the optical flow tracking of the image i greater than or equal to the threshold 2. For example, the image i of the last optical flow tracking, or the image i with the highest quality in all optical flow trackings.

[0264] Among them, in addition to the above description, S41 and S42 are similar to the implementation manners of S18 and S19 in the Figure 4 illustrated embodiment, and will not be elaborated herein in this application.

[0265] S43. When the image i is the image i - 1, delete the picture corresponding to the image with poor quality.

[0266] Among them, S43 is an optional step.

[0267] When i = 1, the image 1 is the first frame image of the document 1 obtained by the electronic device. Thus, the electronic device does not need to execute S43 and can directly execute S44. Or, if the electronic device determines that a frame image before the image 1 is empty, the electronic device can determine that the image 1 and the empty image are not the same image and can execute S44.

[0268] When i ≥ 2, the electronic device can determine whether the image i is the image i - 1.

[0269] Among them, in addition to the above description, S43 is similar to the implementation manner of S20 in the Figure 4 illustrated embodiment, and will not be elaborated herein in this application.

[0270] S44. Determine whether to trigger re - detection.

[0271] It should be understood that during the video recording process of the electronic device, due to factors such as the movement of the electronic device, occlusion of the image i, or poor lighting, the image i obtained each time may be unclear, resulting in different qualities of the image i obtained by the electronic device each time.

[0272] When i = 1, the electronic device triggers re - detection and executes S45.

[0273] When i ≥ 2, the electronic device determines whether to trigger re - detection by whether the image i is the image i - 1.

[0274] In some embodiments, when the image i is not the same as the image i-1, the ratio between the number of feature points of the image i and the number of feature points of the image i-1 is less than a preset threshold, and the electronic device may trigger re-detection. When the image i is the same as the image i-1, the ratio between the number of feature points of the image i and the number of feature points of the image i-1 is greater than or equal to the preset threshold, and the electronic device may not trigger re-detection and execute S36 again.

[0275] Among them, the number of feature points of the above-mentioned image i can be understood as the number of feature points of the image i in any one of multiple optical flow trackings performed on the image i. For example, the number of feature points of the image i in the first optical flow tracking, or the number of feature points of the image i in the last optical flow tracking.

[0276] Similarly, the number of feature points of the above-mentioned image i-1 can be understood as the number of feature points of the image i-1 in any one of multiple optical flow trackings performed on the image i-1. For example, the number of feature points of the image i-1 in the first optical flow tracking, or the number of feature points of the image i-1 in the last optical flow tracking.

[0277] If it is less, the electronic device can determine that the document 1 has been flipped from the image i-1 to the image i or the electronic device has moved so that the image i has moved out of the viewfinder. Thus, the electronic device can trigger re-detection and execute S45.

[0278] If it is greater than or equal to, the electronic device can determine that the document 1 has not been flipped from the image i-1 to the image i and the electronic device has not moved so that the image i has moved out of the viewfinder. Thus, the electronic device can execute S36 again, perform multiple optical flow trackings on the feature points of the image i located within the document border, and facilitate obtaining a high-quality image i again.

[0279] Assume that the document 1 is a PPT-format document and the preset threshold is 0.7.

[0280] When the number of feature points of the 5th frame image is 20 and the number of feature points of the 6th frame image is 5, the ratio 0.25 between the number of feature points of the 6th frame image and the number of feature points of the 5th frame image is less than 0.7, then the electronic device can determine that the 5th frame image and the 6th frame image are images of different pages of the PPT-format document. Thus, the electronic device can trigger re-detection, execute S45, and execute S34-S44 again until the video recording ends or the acquisition of the document 1 stops during the video recording.

[0281] When the number of feature points in the fifth-frame image is 25 and the number of feature points in the sixth-frame image is 20, the ratio 0.8 between the number of feature points in the sixth-frame image and the number of feature points in the fifth-frame image is greater than 0.7. Then, the electronic device can determine that both the fifth-frame image and the sixth-frame image are images of the same page of the PPT-format document. Thus, the electronic device may not trigger re-detection and execute S36 - S44 until the video recording is stopped or the acquisition of Document 1 is stopped during the video recording.

[0282] In addition, in S43, when re-detection is not triggered, the electronic device can determine that the image entering the 2.3 saving stage is the same image of Document 1. Thus, the electronic device can delete the pictures corresponding to the images with poor quality.

[0283] After re-detection is triggered, when the electronic device executes S34 - S43 for the first time, the electronic device can determine that the image currently entering the 2.3 saving stage is not the same image of Document 1 as the image entering the 2.3 saving stage last time. After executing S42, the electronic device does not need to execute S43 and can directly execute S44.

[0284] S45. Update i to i + 1.

[0285] When the electronic device triggers re-detection, the process of acquiring and saving image i is completed. Thus, the electronic device can continue to acquire and save the images after image i, that is, the electronic device can execute S45 and execute S34 - S44 again.

[0286] Thus, the electronic device can, by means of the condition of whether re-detection is triggered, ensure that when Document 1 flips from image i - 1 to image i or the electronic device moves such that the image i of Document 1 moves out of the viewfinder, the electronic device can trigger re-detection, facilitating the acquisition and saving of the images after image i.

[0287] S46. Stop video recording.

[0288] Among them, S46 is similar to the implementation manner of S22 in the Figure 4 illustrated embodiment and will not be elaborated here.

[0289] In summary, during the video recording process of the electronic device, when Document 1 is detected, a document scanning control is displayed to facilitate providing the user with the function of obtaining Document 1. Thus, the electronic device can obtain Document 1 in response to the operation of the document scanning control, simplifying the user's operation and improving the user experience. The electronic device performs multiple optical flow tracking operations on the same image i of Document 1 through optical flow tracking, enabling the high-quality image i to enter the saving stage. Additionally, when the electronic device has already saved the picture corresponding to the same image i of Document 1, it can delete the picture corresponding to the poor-quality image, avoiding duplicate saving of the pictures corresponding to the same image i of Document 1 and enabling the electronic device to save high-quality pictures.

[0290] Based on Figure 5 the embodiments shown, the electronic device can obtain high-quality images through optical flow tracking, facilitating the saving of the pictures corresponding to the high-quality images. It can be seen that Figure 5 the embodiments shown are applicable to scenarios where the electronic device is prone to movement, making it difficult to stably record videos and difficult to obtain high-quality images of Document 1.

[0291] Based on the above Figure 4 or Figure 5 description of the embodiments shown, when determining the recording focal length and recording direction, the electronic device can adopt the following method.

[0292] Next, in combination with Figure 6 , the specific implementation process of the recording focal length and recording direction of this application will be introduced in detail.

[0293] Please refer to Figure 6 , Figure 6 which shows a schematic flowchart of determining the recording focal length and recording direction provided by an embodiment of this application.

[0294] As Figure 6 shown, the determination of the recording focal length and recording direction provided by this application may include:

[0295] S51. Detect the document border of Document 1.

[0296] It should be understood that after the electronic device receives the operation on the document scanning control, it is possible that due to the movement of the electronic device, the image of Document 1 moves out of the display interface, or the presentation of Document 1 ends on the presentation interface of Document 1. Thus, the electronic device can determine whether the image of Document 1 is displayed on the display interface by detecting the document border of Document 1.

[0297] Among them, the electronic device can detect the document border of Document 1 through a border detection algorithm.

[0298] The present application can store the border detection algorithm in the electronic device and / or a storage device communicating with the electronic device, facilitating the electronic device to call the border detection algorithm to detect the document border of Document 1.

[0299] S52. Determine whether a document border is detected.

[0300] When a document border is detected, the electronic device can determine that an image of Document 1 is displayed in the display interface, and the electronic device can execute S53.

[0301] When no document border is detected, the electronic device can determine that an image of Document 1 is not displayed in the display interface, and the electronic device can execute S51 again until the document border of Document 1 is detected.

[0302] S53. Determine whether the direction of the document border is consistent with the current direction of the viewfinder.

[0303] The electronic device can determine whether the direction of the document border is consistent with the current direction of the viewfinder.

[0304] Among them, the direction of the document border can be set to be consistent with the presentation direction of Document 1. When the presentation direction of Document 1 is the landscape orientation, the direction of the document border is the landscape orientation. When the presentation direction of Document 1 is the portrait orientation, the direction of the document border is the portrait orientation.

[0305] Among them, the current direction of the viewfinder can be set to be consistent with the recording direction of the electronic device. When the recording direction of the electronic device is the landscape orientation, the current direction of the viewfinder is the landscape orientation. When the recording direction of the electronic device is the portrait orientation, the current direction of the viewfinder is the portrait orientation.

[0306] When the direction of the document border is the landscape orientation and the current direction of the viewfinder is also the landscape orientation, or when the direction of the document border is the portrait orientation and the current direction of the viewfinder is also the portrait orientation, the electronic device can determine that the direction of the document border is consistent with the current direction of the viewfinder.

[0307] When the direction of the document border is the landscape orientation and the current direction of the viewfinder is the portrait orientation, or when the direction of the document border is the portrait orientation and the current direction of the viewfinder is the landscape orientation, the electronic device can determine that the direction of the document border is inconsistent with the current direction of the viewfinder.

[0308] When the direction of the document border is consistent with the current direction of the viewfinder, the electronic device can determine that the current direction of the viewfinder and the direction of the document border do not need to be adjusted. Thus, the electronic device can execute S54.

[0309] When the direction of the document border is inconsistent with the current direction of the viewfinder, the electronic device can determine that the current direction of the viewfinder or the direction of the document border needs to be adjusted. Thus, the electronic device can execute S57.

[0310] S54. Determine whether the area ratio between the document border and the viewfinder in the current direction is within Range 1.

[0311] The electronic device can determine the area of the document border by multiplying the length and width of the document border. The electronic device can determine the area of the viewfinder in the current direction by multiplying the length and width of the viewfinder in the current direction. Based on the area of the document border and the area of the viewfinder in the current direction, the electronic device can obtain the area ratio between the document border and the viewfinder in the current direction.

[0312] Thus, the electronic device can determine whether the area ratio between the document border and the viewfinder in the current direction is within Range 1.

[0313] Among them, Range 1 is preset according to actual needs. For example, Range 1 can be set to 20% - 100%.

[0314] For example, when Range 1 is 20% - 100%, the length of the document border is 5 cm, the width is 3 cm, the length of the viewfinder is 10 cm, and the width is 6 cm, the area of the document border is 15 cm 2 , the area of the viewfinder is 60 cm 2 , and the area ratio between the document border and the viewfinder is 15 / 60 = 25%. The electronic device can determine that the area ratio between the document border and the viewfinder is within Range 1.

[0315] When the area ratio between the document border and the viewfinder in the current direction is within Range 1, the electronic device can determine that the current recording focal length does not need to be adjusted. Thus, the electronic device can execute S55.

[0316] When the area ratio between the document border and the viewfinder in the current direction is not within Range 1, the electronic device can determine that the current recording focal length needs to be adjusted. Thus, the electronic device can execute S56.

[0317] S55. Determine the current recording focal length as the recording focal length, and determine the current direction of the viewfinder as the recording direction.

[0318] When the electronic device determines that the direction of the document border is consistent with the current direction of the viewfinder, and the area ratio between the document border and the viewfinder is within Range 1, it can determine the current recording focal length as the recording focal length and determine the current direction of the viewfinder as the recording direction. Thus, the electronic device can obtain a viewfinder that is adapted to both the size and direction of the document border.

[0319] For example, when the orientation of the document border is landscape and the current orientation of the viewfinder is also landscape, and the current recording focal length is 1x, the electronic device can determine that the orientation of the document border is consistent with the current orientation of the viewfinder, and the area ratio between the document border and the viewfinder is within range 1. The electronic device can determine that the recording focal length is 1x and determine the current orientation of the viewfinder as the recording orientation.

[0320] S56. Determine the recording focal length according to the area ratio between the document border and the viewfinder in the current orientation, and determine the current orientation of the viewfinder as the recording orientation.

[0321] When the electronic device determines that the orientation of the document border is consistent with the current orientation of the viewfinder, and the area ratio between the document border and the viewfinder in the current orientation is not within range 1, the electronic device can adjust the recording focal length according to the area ratio between the document border and the viewfinder in the current orientation, and determine the current orientation of the viewfinder as the recording orientation. Thus, the electronic device can obtain a viewfinder that is adapted to both the size and orientation of the document border.

[0322] For example, when the orientation of the document border is landscape, the current orientation of the viewfinder is also landscape, the current recording focal length is 1x, and the area ratio between the document border and the viewfinder is not within range 1, the electronic device can adjust the current recording focal length from 1x to 3x according to the preset focal length range, so that the area ratio between the document border and the viewfinder is within range 1, and determine the current orientation of the viewfinder as the recording orientation.

[0323] S57. Display a recommended landscape control or a recommended portrait control.

[0324] When the orientation of the document border is inconsistent with the current orientation of the viewfinder, the electronic device can display a recommended landscape control or a recommended portrait control on the display interface of the electronic device.

[0325] This application does not specifically limit parameters such as the display position, display style, and display size of the recommended landscape control or the recommended portrait control.

[0326] Among them, the recommended landscape control is used to trigger the viewfinder to adjust from the portrait orientation to the landscape orientation. The recommended portrait control is used to trigger the viewfinder to adjust from the landscape orientation to the portrait orientation.

[0327] Thus, the electronic device provides the function of adjusting the orientation of the viewfinder to the user through the recommended landscape control or the recommended portrait control.

[0328] In some embodiments, the orientation of the document border is landscape, the current orientation of the viewfinder is portrait, and the orientation of the document border is inconsistent with the current orientation of the viewfinder. At this time, the display interface of the electronic device can display the recommended landscape control.

[0329] In some other embodiments, the orientation of the document border is in the portrait orientation, and the current orientation of the viewfinder is in the landscape orientation. Since the orientation of the document border is inconsistent with the current orientation of the viewfinder, the display interface of the electronic device may display a recommended portrait control.

[0330] S58. Determine whether an operation on the recommended landscape control or the recommended portrait control is received.

[0331] The electronic device can determine whether an operation of a type such as click, double - click, long - press, etc. performed by the user on the recommended landscape control or the recommended portrait control is received.

[0332] When the electronic device does not receive an operation on the recommended landscape control or the recommended portrait control within a preset duration, it can be determined that the user does not want to adjust the orientation of the viewfinder. Thus, the electronic device can execute S59.

[0333] When the electronic device receives an operation on the recommended landscape control or the recommended portrait control, it can be determined that the user wants to adjust the orientation of the viewfinder. Thus, the electronic device can execute S60.

[0334] S59. Determine the recording focal length according to the area ratio between the document border and the viewfinder in the current orientation, and determine the current orientation of the viewfinder as the recording orientation.

[0335] When the electronic device does not receive an operation on the recommended landscape control or the recommended portrait control within a preset duration, it can directly determine the recording focal length according to the area ratio between the document border and the viewfinder in the current orientation, and determine the current orientation of the viewfinder as the recording orientation. Thus, the electronic device can obtain a viewfinder that is adapted to the size of the document border and corresponds to the orientation that the user wants to obtain.

[0336] S60. Adjust the current orientation of the viewfinder to the landscape orientation or the portrait orientation.

[0337] Among them, after the electronic device receives an operation on the recommended landscape control or the recommended portrait control, it can adjust the current orientation of the viewfinder to the landscape orientation or the portrait orientation.

[0338] In some embodiments, the orientation of the document border is in the landscape orientation, and the current orientation of the viewfinder is in the portrait orientation. The display interface of the electronic device may display a recommended landscape control. After the electronic device receives an operation performed by the user on the recommended landscape control, it can adjust the viewfinder from the portrait orientation to the landscape orientation. Thus, the electronic device can obtain a viewfinder that is adapted to the orientation of the document border.

[0339] In some other embodiments, the orientation of the document border is in the portrait orientation, the current orientation of the viewfinder is in the landscape orientation, the display interface of the electronic device can display a recommended portrait control, and after the electronic device receives an operation performed by the user on the recommended portrait control, the viewfinder can be adjusted from the landscape orientation to the portrait orientation. Thus, the electronic device can obtain a viewfinder adapted to the orientation of the document border.

[0340] Thus, by adjusting the orientation of the viewfinder, the electronic device makes the orientation of the viewfinder consistent with the orientation of the document border, ensuring that the electronic device can more accurately adjust the recording focal length.

[0341] S61. Determine the recording focal length according to the area ratio between the document border and the viewfinder after the adjustment direction, and determine the adjusted direction of the viewfinder as the recording direction.

[0342] The electronic device can adjust the orientation of the viewfinder to determine the viewfinder after the adjustment direction, so that the orientation of the document border is consistent with the orientation of the viewfinder. The electronic device can determine the recording focal length according to the area ratio between the document border and the viewfinder after the adjustment direction. Thus, the electronic device can obtain a viewfinder adapted to both the size and orientation of the document border.

[0343] It should be understood that in addition to the above method, the electronic device can also adjust the orientation of the document border so that the orientation of the document border is consistent with the orientation of the viewfinder.

[0344] Assume that the current recording focal length is 1x, the range 1 is 20% - 100%, the orientation of the document border is in the landscape orientation, and the current orientation of the viewfinder is in the portrait orientation.

[0345] When the electronic device does not receive an operation on the recommended landscape control within a preset duration, the electronic device can determine the current orientation of the viewfinder as the recording direction, and can adjust the current recording focal length from 1x to 2.5x according to the recording direction. At this time, the area ratio between the document border and the viewfinder in the current orientation can reach 60%, enabling the electronic device to obtain a high - clarity image of Document 1.

[0346] When the electronic device receives an operation on the recommended landscape control, the electronic device can determine the adjusted direction of the viewfinder as the recording direction, and can adjust the current recording focal length from 1x to 3x according to the adjusted recording direction. At this time, the area ratio between the document border and the viewfinder in the adjusted direction can reach 80%, enabling the electronic device to obtain a high - clarity image of Document 1.

[0347] In summary, the electronic device can obtain a viewfinder adapted to the size of the document border and corresponding to the orientation desired by the user by adjusting the recording focal length and / or the recording direction, ensuring high clarity of the image located within the document border.

[0348] Based on the above Figure 4 or Figure 5 Based on the description of the above embodiments, when the electronic device adjusts the tilt angle of any one image, the following method can be adopted.

[0349] Next, in combination with Figure 7 , the specific implementation process of adjusting the tilt angle of any one image in this application will be introduced in detail.

[0350] Please refer to Figure 7 , Figure 7 which shows a schematic flowchart of adjusting the tilt angle of any one image provided by an embodiment of this application.

[0351] As Figure 7 shown, the adjustment of the tilt angle of any one image provided by this application may include:

[0352] S71. Determine the positions of each vertex of the document border of Document 1 corresponding to the image.

[0353] Among them, the positions of each vertex of the document border of Document 1 may be Figures 3G - 3J the positions of the four vertices A, B, C, and D of the document border 206 shown.

[0354] In addition, the positions of the four vertices A, B, C, and D of the document border can be represented by coordinates.

[0355] In some embodiments, the electronic device may establish a coordinate system with one of the vertices of the document border of Document 1 as the origin, with the long side direction of the document border as the x-axis direction and the short side direction as the y-axis direction. Thus, the electronic device can determine the respective x coordinates and y coordinates of the four vertices A, B, C, and D of the document border.

[0356] Thus, the electronic device obtains the respective x coordinates and y coordinates of the four vertices A, B, C, and D, preparing the data for adjusting the tilt angle of the image according to the respective x coordinates and y coordinates of the four vertices A, B, C, and D of the document border.

[0357] S72. Determine the distance between each vertex of the document border of Document 1 corresponding to the image and the presentation interface of Document 1.

[0358] When the electronic device adjusts the tilt angle of the image, it can determine the distance between each vertex of the document border of Document 1 corresponding to the image and the presentation interface of Document 1 through a depth estimation algorithm, facilitating more accurate adjustment of the tilt angle of the image in combination with the positions in S71.

[0359] The present application can store the depth estimation algorithm in the electronic device and / or a storage device communicating with the electronic device, facilitating the electronic device to call the depth estimation algorithm to determine the distance from each vertex of the document border of the document 1 corresponding to the image to the presentation interface of the document 1.

[0360] In some embodiments, the electronic device can establish a coordinate system with one vertex of the document border of the document 1 as the origin, with the long side direction of the document border as the x-axis direction, the short side direction as the y-axis direction, and the direction perpendicular to both the x-axis direction and the y-axis direction as the z-axis direction. Thus, the electronic device can determine the z coordinates of the four vertices A, B, C, and D of the document border according to the above distances.

[0361] Thus, the electronic device obtains the x coordinates, y coordinates, and z coordinates of the four vertices A, B, C, and D respectively, preparing data for adjusting the tilt angle of the image according to the x coordinates, y coordinates, and z coordinates of the four vertices A, B, C, and D of the document border.

[0362] Among them, S71 and S72 can be executed simultaneously or sequentially.

[0363] S73. Adjust the tilt angle of the image according to the position of each vertex and the distance from each vertex to the presentation interface of the first document.

[0364] According to the descriptions of S72 and S73 above, the electronic device can obtain the x coordinates, y coordinates, and z coordinates of the four vertices A, B, C, and D of the document border. Thus, the electronic device can perform perspective correction adjustment on the tilt angle of the image through the x coordinates, y coordinates, and z coordinates of the four vertices A, B, C, and D of the document border.

[0365] In some embodiments, the electronic device adjusts the tilt angle of the image through the x coordinates and y coordinates of the four vertices A, B, C, and D respectively, so that the image is restored from the deformed state to the normal state, preparing data for the electronic device to be able to save the picture corresponding to the high-quality image.

[0366] In some other embodiments, the electronic device adjusts the tilt angle of the image through the x coordinates, y coordinates, and z coordinates of the four vertices A, B, C, and D respectively, so that the image is better restored from the deformed state to the normal state, preparing data for the electronic device to be able to save the picture corresponding to the higher-quality image.

[0367] Among them, after perspective correction adjustment, the x coordinates of vertex A and vertex C of the document border are the same, the x coordinates of vertex B and vertex D of the document border are the same, the y coordinates of vertex A and vertex B of the document border are the same, the y coordinates of vertex C and vertex D of the document border are the same, and the z coordinates of the four vertices A, B, C, and D of the document border are the same.

[0368] In summary, the electronic device adjusts the tilt angle of the image through the above positions and distances to ensure that the document border is parallel to the viewfinder, so that the text, pattern, and / or table presented on the image are restored from the deformed state to the normal state. Thus, data preparation can be provided for the picture corresponding to the high-quality image.

[0369] Based on the above Figure 4 or Figure 5 According to the description of the embodiments shown, the electronic device can adopt various methods to delete the picture corresponding to the poor-quality image when the image i is the image i-1.

[0370] Next, in combination with Figure 8 , the specific implementation process of deleting the picture corresponding to the poor-quality image when the image i is the image i-1 in the present application will be introduced in detail.

[0371] Please refer to Figure 8 , Figure 8 which shows a schematic flowchart of deleting the picture corresponding to the poor-quality image when the image i is the image i-1 provided by an embodiment of the present application.

[0372] As Figure 8 shown, the method of deleting the picture corresponding to the poor-quality image when the image i is the image i-1 provided by the present application may include:

[0373] S81. Determine whether the image i is the image i-1.

[0374] Among them, the implementation manner of S71 is similar to that of S20 in the Figure 4 embodiment shown, and will not be elaborated herein in the present application.

[0375] In addition, the implementation manner of S71 is similar to that of S43 in the Figure 5 embodiment shown, and will not be elaborated herein in the present application.

[0376] S82. Determine whether there is occlusion in the image i and / or the image i-1.

[0377] The electronic device can adopt various methods to determine whether there is occlusion in the image i and / or the image i-1.

[0378] For example, the above method can be whether the ratio between the length of each current side of the document border and the perimeter of the document border is greater than a preset length threshold. Alternatively, the above method can be whether the area ratio between the area of the occluder and the area of the document border is greater than a preset area threshold.

[0379] It should be understood that when the image of Document 1 is occluded, the document border is also occluded, and at least one side of the document border becomes shorter. Then, the electronic device can calculate the ratio between the length of the shortened side of the document border and the perimeter of the document border, and when this ratio is greater than the preset length threshold, the electronic device can determine that there is an occlusion in the image.

[0380] Among them, the occluder is usually the head of a human body. When the electronic device detects a human face through a face detection algorithm, it can calculate the ratio between the predicted area of the human face and the area of the document border, and when this ratio is greater than the preset area threshold, the electronic device can determine that there is an occlusion in the image.

[0381] This application can store the face detection algorithm in the electronic device and / or a storage device communicating with the electronic device, facilitating the electronic device to call the face detection algorithm to perform face detection on the image of Document 1.

[0382] When there is an occlusion in Image i and / or Image i - 1, the electronic device can execute S84; when there is no occlusion in Image i and Image i - 1, the electronic device can execute S83.

[0383] S83. Delete the picture corresponding to the image with poor quality according to the quality of Image i and Image i - 1.

[0384] In Figure 4 In S20 of the illustrated embodiment, the quality can include at least one of layout quality, number of characters, and clarity. The layout quality includes at least one of the degree of bending of the pattern, the degree of bending of the paragraph, and the degree of bending of the text line.

[0385] For example, an image with high quality can be an image with high layout quality, a large number of characters, and high clarity, and an image with poor quality can be an image with low layout quality, a small number of characters, and low clarity.

[0386] In Figure 5 In S43 of the illustrated embodiment, the quality can include but is not limited to clarity.

[0387] For example, an image with high quality can be an image with high clarity, and an image with poor quality can be an image with low clarity.

[0388] Thus, the electronic device can delete the picture corresponding to the image with poor quality to retain the picture corresponding to the image with high quality.

[0389] S84. Delete the picture corresponding to the image with a large occlusion amount according to the occlusion amount between image i and image i-1.

[0390] When there is occlusion in both image i and image i-1, the electronic device can determine the image with a small occlusion amount and the image with a large occlusion amount.

[0391] When there is occlusion in one of image i and image i-1, the electronic device can determine the image without occlusion as the image with a small occlusion amount, and the image with occlusion as the image with a large occlusion amount.

[0392] Thus, the electronic device can delete the picture corresponding to the image with a large occlusion amount to retain the picture corresponding to the image with a small occlusion amount.

[0393] In summary, based on the occlusion amount and quality, through the above steps, the electronic device can keep the pictures with high preservation quality and further avoid repeatedly saving the pictures corresponding to the same image.

[0394] Based on some of the foregoing embodiments, the document processing method provided in this application is introduced below.

[0395] Exemplarily, this application provides a document processing method.

[0396] For the sake of convenience of description, the document processing method of this application can be executed by the Figure 2 electronic device in.

[0397] Please refer to Figure 9 , Figure 9 which shows a schematic flowchart of the document processing method provided in an embodiment of this application.

[0398] As Figure 9 shown, the document processing method provided in this application may include:

[0399] S101. Display a first interface, where the first interface is used to display the video during recording.

[0400] Among them, for the specific implementation manner of the first interface, reference can be made to interface 22 in Figures 3B - 3L , and the display interface in Figure 4 , and the description of the display interface in Figure 5 , and interface 22 in Figures 3B - 3L .

[0401] For the specific implementation manner of S101, reference can be made to the description of interface 22 in Figures 3B - 3L , and S11 in Figure 4 , and the description of S31 in Figure 5 , which will not be elaborated here.

[0402] S102. In response to detecting a first document in the first interface, display a first control in the first interface.

[0403] For the format and content of the first document, reference can be made to the description of the format and content of Document 1 above.

[0404] The format of the first document can be PDF format, PPT format, Word format, or long picture format.

[0405] For the specific implementation of the first control, reference can be made to Figures 3D - 3J the control 205 in the interface 22 shown, and Figure 4 the document scanning control in Figure 5 the description of the document scanning control in

[0406] For the specific implementation of S102, reference can be made to the description of the control 205 in the interface 22 shown in 3D- Figure 3J and the S12 in Figure 4 and the description of S32 in Figure 5 which will not be elaborated here.

[0407] S103. In response to a first operation on the first control, determine a recording focal length, which is used to adjust the size of the viewfinder frame in the first interface to match the size of the document border of the first document in the first interface.

[0408] For the specific implementation of the first operation, reference can be made to Figure 3D the operations on the control 205 of the interface 22 shown, and Figure 4 the document scanning control in Figure 5 the description of the operations on the document scanning control in

[0409] For the recording focal length, reference can be made to the recording focal length displayed on the control 203 in the interface 22 shown in 3B- Figure 3L For the specific implementation of S103, reference can be made to

[0410] the control 203 in the interface 22 shown, and Figures 3D - 3G the S13 in Figure 4 and the description of S33 in Figure 5 which will not be elaborated here.

[0411] S104. According to the recording focal length, detect the images within the document border of the first document to obtain a picture set of the first document, where the picture set of the first document includes one or more pictures.

[0412] For the specific implementation of the document border of the first document, reference can be made to Figures 3E - 3J the document border 206 in the interface 22 shown, and Figure 4S14 in Figure 5 the description of S34 in

[0413] For the specific implementation of the picture set of the first document, reference can be made to Figures 3M - 3P the description of interface 23 in

[0414] For the specific implementation of S104, reference can be made to Figures 3G - 3L the interface 22 shown, and 3M - Figure 3P the interface 23 shown, and Figure 4 S14 - S22 in Figure 5 the description of S34 - S46 in , which will not be elaborated here.

[0415] S105. After the video recording ends, generate a second document according to the picture set of the first document.

[0416] Among them, the format and content of the second document can be the same as or different from those of the first document.

[0417] In some embodiments, the second document can be Figure 3R document 2 in interface 24 shown.

[0418] For the document processing method of this application, during the video recording process of the electronic device, in response to detecting the first document, display a first control on the first interface. Thus, the electronic device can obtain the first document in response to the operation of the first control, saving the user's operation time, improving the user experience, and ensuring the diversified services of the electronic device. The electronic device can also determine the recording focal length in response to the operation of the first control, enabling the electronic device to obtain high - definition images, providing data preparation for saving pictures corresponding to high - quality images. The electronic device detects the images within the document border of the document through the recording focal length, and one or more pictures of the document can be obtained. The electronic device can automatically generate a second document corresponding to the first document according to one or more pictures of the first document after the video recording ends, simplifying the user's operation and further improving the user experience.

[0419] In some embodiments, the document processing method of this application may include:

[0420] Step 201: Display a first interface, where the first interface is used to display the video during recording.

[0421] Step 202: In response to detecting the first document in the first interface, display a first control in the first interface.

[0422] Step 203: In response to a first operation on a first control, determine a recording focal length and a recording direction. The recording focal length is used to adjust the size of the viewfinder of the first interface to match the size of the document border of the first document in the first interface, and the recording direction is used to indicate the direction of the viewfinder of the first interface.

[0423] Step 204: Detect the images within the document border of the first document according to the recording focal length and the recording direction, to obtain a picture set of the first document. The picture set of the first document includes one or more pictures.

[0424] Step 205: After the video recording ends, generate a second document according to the picture set of the first document.

[0425] In a feasible implementation manner of step 203, the electronic device determines the recording focal length and the recording direction in response to the first operation on the first control.

[0426] The recording direction can refer to the direction shown by the viewfinder 201 in the interface 22 shown in 3B- Figure 3L The specific implementation manner of step 203 can refer to the control 203 and the viewfinder 201 in the interface 22 shown in

[0427] and S13 in Figures 3E - 3F as well as the description of S33 in Figure 4 which will not be elaborated here. Figure 5

[0428] In a feasible implementation manner of step 204, the electronic device detects the images within the document border of the first document according to the recording focal length and the recording direction.

[0429] The specific implementation manner of step 204 can refer to the control 203 and the viewfinder 201 in the interface 22 shown in Figures 3B - 3J as well as the interface 23 shown in 3M- Figure 3P and S14 - S22 in Figure 4 as well as the description of S34 - S46 in Figure 5 which will not be elaborated here.

[0430] In some embodiments, the document processing method of the present application may include:

[0431] Step 301: Determine the current direction of the viewfinder of the first interface and the direction of the document border of the first document.

[0432] Among them, the specific implementation manner of the current direction of the viewfinder and the direction of the document border can refer to the description of S53 in Figure 6 The specific implementation manner of step 301 can refer to

[0433] as well as the description of S53 in Figures 3D - 3EThe interface 22 shown, and Figure 6 the description of S53 in Figure 6 will not be elaborated here.

[0434] Step 02: When the current direction of the viewfinder in the first interface is inconsistent with the direction of the document border of the first document, display a second control in the first interface.

[0435] Among them, for the specific implementation method of the second control, reference can be made to Figures 3E - 3J the control 207 in the interface 22 shown, and Figure 6 the description of the recommended landscape control or the recommended portrait control in Figure 6 .

[0436] For the specific implementation method of step 302, reference can be made to Figure 3E the interface 22 in Figure 3E , and Figure 6 the description of S57 in Figure 6 will not be elaborated here.

[0437] Step 303: After receiving the second operation on the second control, adjust the current direction of the viewfinder in the first interface to obtain the recording direction; determine the recording focal length according to the recording direction and the ratio between the document border of the first document and the viewfinder of the first interface corresponding to the adjusted direction.

[0438] Among them, for the specific implementation method of the second operation, reference can be made to Figure 3E the operation on the control 207 in the interface 22 shown, and Figure 6 the operation on the recommended landscape control or the recommended portrait control in Figure 6 .

[0439] The above ratio may include at least one of the area ratio, length ratio, width ratio, and perimeter ratio. For example, the above ratio may be the area ratio.

[0440] For the specific implementation method of step 303, reference can be made to Figures 3E - 3F the viewfinder 201 in the interface 22 shown, and Figure 6 the description of S60 - S61 in Figure 6 will not be elaborated here.

[0441] Step 304: When the second operation on the second control is not received after a preset duration, or when the current direction of the viewfinder in the first interface is consistent with the direction of the document border of the first document, determine the current direction of the viewfinder in the first interface as the recording direction; determine the recording focal length according to the recording direction and the ratio between the document border of the first document and the viewfinder of the first interface corresponding to the current direction.

[0442] For the specific implementation method of step 304, reference can be made to Figure 6 S59 and the descriptions of S55 - S56 in Figure 6 will not be elaborated here.

[0443] In some embodiments, the document processing method of the present application may include:

[0444] Step 401: Determine the document border of the first document in the first interface according to the recording focal length and the recording direction.

[0445] Among them, for the specific implementation manner of step 401, reference may be made to Figure 3G the interface 22 shown, and Figure 4 the description of S14 in

[0446] which will not be elaborated here.

[0447] Among them, for the i-th image, reference may be made to Figure 4 the description of image i in S15 of

[0448] For the specific implementation manner of step 402, reference may be made to Figure 4 the description of S15 in

[0449] which will not be elaborated here.

[0450] Among them, for the content change amount, reference may be made to Figure 4 the description of character change and / or layout change in S17 of

[0451] For the specific implementation manner of step 403, reference may be made to Figure 4 the description of S16 - S17 in

[0452] which will not be elaborated here.

[0453] Among them, for the preset change amount, reference may be made to Figure 4 the description of threshold 1 in S17 of

[0454] For the specific implementation manner of step 404, reference may be made to Figure 4 the description of S17 - S21 and S14 in

[0455] Step 405: When the content change amount between the i-th image and the (i - 1)-th image is less than a preset change amount, update i to i + 1, and continue to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends.

[0456] Among them, for the specific implementation manner of step 405, reference can be made to Figure 4 the descriptions of S17 and S14 therein, which will not be elaborated here.

[0457] In some embodiments, the document processing method of the present application may include:

[0458] Step 501: Perform optical character recognition and / or layout analysis on the i-th image and the (i - 1)-th image to obtain the content detection situations of the i-th image and the (i - 1)-th image.

[0459] Step 502: Determine the content change amount between the i-th image and the (i - 1)-th image according to the content detection situations of the i-th image and the (i - 1)-th image.

[0460] Among them, for the specific implementation manners of step 501 and step 502, reference can be made to Figure 4 the description of S16 therein, which will not be elaborated here.

[0461] In some embodiments, the document processing method of the present application may include:

[0462] Step 601: Determine the document border of the first document in the first interface according to the recording focal length and the recording direction.

[0463] Among them, for the specific implementation manner of step 401, reference can be made to Figure 3G the interface 22 shown, and Figure 5 the description of S34 therein, which will not be elaborated here.

[0464] Step 602: Perform multiple optical flow trackings on the feature points of the i-th image within the document border of the first document, where i is a positive integer taking values greater than or equal to 1, and the qualities of the i-th images in the multiple optical flow trackings are different.

[0465] Among them, for the specific implementation manner of step 602, reference can be made to Figure 5 the descriptions of S36 - S38 therein, which will not be elaborated here.

[0466] Step 603: When the number of successful optical flow tracking is greater than or equal to a preset number, adjust the tilt angle of the i-th image corresponding to when the number of successful optical flow tracking is greater than or equal to the preset number; save the adjusted picture corresponding to the i-th image in the picture set of the first document; when the ratio between the number of tracked feature points of the i-th image and the number of tracked feature points of the (i - 1)-th image is less than a preset threshold, update i to i + 1, and continue to execute the step of determining the document border of the first document in the first interface according to the recording focal length and recording direction until the video recording ends; when the ratio between the number of tracked feature points of the i-th image and the number of tracked feature points of the (i - 1)-th image is greater than or equal to the preset threshold, continue to execute the step of performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document until the number of successful optical flow tracking is greater than or equal to the preset number.

[0467] Among them, the preset number can be referred to Figure 5 the description of threshold 2 in S39 of

[0468] For the specific implementation method of step 603, reference can be made to Figure 5 S39 - S45 in

[0469] and the descriptions of S34 and S36, which will not be elaborated here.

[0470] Among them, for the specific implementation method of step 604, reference can be made to Figure 5 the descriptions of S39 and S36 in

[0471] Step 605: When the optical flow tracking is not successful, continue to execute the step of determining the document border of the first document in the first interface until the video recording ends.

[0472] Among them, for the specific implementation method of step 603, reference can be made to Figure 5 the descriptions of S37 and S34 in

[0473] In some embodiments, the document processing method of the present application may include:

[0474] Step 701: Perform the k-th optical flow tracking on the feature points of the i-th image within the document border of the first document, where k is a positive integer taking all values greater than or equal to 1.

[0475] Step 702: Determine whether the k-th optical flow tracking is successful.

[0476] Step 703: When the k-th optical flow tracking is successful, determine the total number of successful optical flow tracking before the k-th optical flow tracking, add one to it, and determine it as the number of successful optical flow tracking; when the number of successful optical flow tracking is greater than or equal to the preset number, adjust the tilt angle of the i-th image in the k-th optical flow tracking; when the number of successful optical flow tracking is less than the preset number, update k to k + 1, and continue to perform the k-th optical flow tracking on the feature points of the i-th image within the document border of the first document until the number of successful optical flow tracking is greater than or equal to the preset number.

[0477] Step 704: When the k-th optical flow tracking is unsuccessful, continue to perform the step of determining the document border of the first document in the first interface according to the recording focal length and recording direction until the video recording ends.

[0478] Among them, the k-th optical flow tracking can refer to Figure 5 the description of the current optical flow tracking in

[0479] Among them, the specific implementation manners of steps 701 - 704 can refer to Figure 5 the descriptions of S36 - S41 in

[0480] In some embodiments, the document processing method of the present application may include:

[0481] Step 801: When the i-th image is the (i - 1)-th image, determine whether there is occlusion in the i-th image and / or the (i - 1)-th image.

[0482] Among them, the specific implementation manner of step 801 can refer to Figure 8 the description of S82 in

[0483] Step 802: When there is occlusion in the i-th image and / or the (i - 1)-th image, according to the occlusion amount between the i-th image and the (i - 1)-th image, save the picture corresponding to the image with a smaller occlusion amount in the picture set of the first document, and delete the picture corresponding to the image with a larger occlusion amount.

[0484] Among them, the specific implementation manner of step 802 can refer to Figure 8 the description of S84 in

[0485] Step 803: When there is no occlusion in both the i-th image and the (i - 1)-th image, according to the quality of the i-th image and the (i - 1)-th image, save the picture corresponding to the image with higher quality in the picture set of the first document, and delete the picture corresponding to the image with lower quality.

[0486] Among them, the specific implementation manner of step 802 can refer to Figure 8 the description of S83 in

[0487] In some embodiments, the document processing method of the present application may include:

[0488] Step 901: Determine the positions of each vertex of the document border of the first document corresponding to the image in the first interface.

[0489] Among them, the positions of each vertex of the document border of the first document in the first interface can be referred to Figures 3G - 3J the positions of the four vertices A, B, C, and D of the document border 206 shown.

[0490] For the specific implementation manner of step 901, reference can be made to Figure 7 the description of S71 in, which will not be elaborated here.

[0491] Step 902: Determine the distance from each vertex of the document border of the first document corresponding to the image from the first interface to the presentation interface of the first document.

[0492] Among them, for the specific implementation manner of step 902, reference can be made to Figure 7 the description of S72 in, which will not be elaborated here.

[0493] Step 903: Adjust the tilt angle of the image according to the position of each vertex in the first interface and the distance from each vertex from the first interface to the presentation interface of the first document.

[0494] Among them, for the specific implementation manner of step 903, reference can be made to Figure 7 the description of S73 in, which will not be elaborated here.

[0495] In some embodiments, the document processing method of the present application may include:

[0496] In response to a third operation on the first control, stop detecting the image within the document border of the first document.

[0497] Among them, for the specific implementation manner of the third operation, reference can be made to Figure 3J the operation on the control 205 in the interface 22 shown, and Figure 4 the document scanning control or cancel document scanning control in, and Figure 5 the operation on the document scanning control or cancel document scanning control in.

[0498] For the specific implementation manner of this step, reference can be made to Figure 3J the operation on the control 205 in the interface 22 shown, and Figure 4 S22 in, and Figure 5 the description of S46 in, which will not be elaborated here.

[0499] In some embodiments, the document processing method of the present application may include:

[0500] Step 1001: Display a third control on a first interface.

[0501] Step 1002: In response to a fourth operation on the third control, start video recording. Alternatively, in response to a fifth operation on the third control, stop video recording.

[0502] Among them, for the specific implementation of the third control, reference can be made to the control 204 in the Figures 3B - 3L shown interface 22, and Figure 4 the video recording control or the video recording stop control in Figure 5 and the description of the video recording control or the video recording stop control in

[0503] For the specific implementation of the fourth operation, reference can be made to the operations on the control 204 in the Figure 3B shown interface 22, and Figure 4 the operations on the video recording control of S11 in Figure 5 and the operations on the video recording control of S31 in

[0504] For the specific implementation of the fifth operation, reference can be made to the operations on the control 204 in the Figure 3K shown interface 22, and Figure 4 the video recording control of S11 in Figure 5 and the operations on the video recording control of S31 in

[0505] Alternatively, for the specific implementation of the fifth operation, reference can be made to the operations on the control 204 in the Figure 3K shown interface 22, and Figure 4 the video recording stop control of S22 in Figure 5 and the operations on the video recording stop control of S43 in

[0506] For the specific implementation of Step 1001 and Step 1002, reference can be made to Figure 4 S22 in Figure 5 and the description of S46 in , which will not be elaborated here.

[0507] In some embodiments, the document processing method of the present application may include:

[0508] Step 1101: Display a second interface for displaying each picture in a picture set of a first document.

[0509] Among them, for the specific implementation of the second interface, reference can be made to Figures 3M - 3P the description of the shown interface 23.

[0510] For the specific implementation of Step 1101, reference can be made to Figures 3M - 3PDescription of the interface 23 shown.

[0511] Step 1102: In response to a sixth operation on the second interface, determine the selected picture in the picture set of the first document.

[0512] Among them, for the specific implementation manner of the sixth operation, reference can be made to Figures 3M - 3N the operations on the controls 2301, 2302, and 2303 in the interface 23 shown.

[0513] For the specific implementation manner of step 1102, reference can be made to Figures 3M - 3N the description of the interface 23 shown.

[0514] Step 1103: Display a fourth control in the second interface.

[0515] Among them, for the specific implementation manner of the fourth control, reference can be made to Figures 3M - 3P the description of the control 209 in the interface 23 shown.

[0516] Step 1104: In response to a seventh operation on the fourth control, generate a second document according to the selected picture.

[0517] Among them, for the specific implementation manner of the seventh operation, reference can be made to Figure 3P the operations on the control 209 in the interface 23 shown.

[0518] For the specific implementation manner of steps 1103 - 1104, reference can be made to Figure 3P the description of the interface 23 in

[0519] In some embodiments, the document processing method of the present application may include:

[0520] Step 1201: Display a fifth control in the second interface.

[0521] Step 1202: In response to an eighth operation on the fifth control, perform at least one of image enhancement, image shadow removal, or image grayscaling on the selected picture.

[0522] Among them, for the specific implementation manner of the fourth control, reference can be made to Figures 3M - 3P the description of the control 208 in the interface 23 shown.

[0523] For the specific implementation manner of the eighth operation, reference can be made to Figure 3O the operations on the control 208 in the interface 23 shown.

[0524] For the specific implementation manner of steps 1201 - 1202, reference can be made to Figure 3O the description of the interface 23 in

[0525] Exemplarily, the present application provides a document processing device, which may include one or more modules for executing the methods in the foregoing embodiments.

[0526] Exemplarily, the present application provides an electronic device, including a processor; when the processor executes computer code or instructions in a memory, the electronic device is caused to execute the methods in the foregoing embodiments.

[0527] Exemplarily, the present application provides an electronic device, including: a memory and a processor; the memory is coupled to the processor, and the memory is used to store program code or instructions; the processor is used to call the program code or instructions in the memory so that the electronic device executes the methods in the foregoing embodiments.

[0528] Exemplarily, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen, and a sensor; the chip system includes: one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the memory and send signals to the processors, and the signals include computer code or instructions stored in the memory; when the processors execute the computer code or instructions, the electronic device executes the methods in the foregoing embodiments.

[0529] Exemplarily, the present application provides a computer-readable storage medium, in which code or instructions are stored, and when the code or instructions run on an electronic device, the electronic device is caused to execute the methods in the foregoing embodiments when implemented.

[0530] Exemplarily, the present application provides a computer program product, which causes an electronic device to implement the methods in the foregoing embodiments when the computer program product runs on a computer.

[0531] In the foregoing embodiments, all or part of the functions may be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer codes or instructions. When the computer program code or instructions are loaded and executed on a computer, all or part of the processes or functions according to the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer code or instructions may be stored in a computer-readable storage medium. The computer-readable storage medium may be any available medium that the computer can access or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0532] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by relevant hardware instructed by a computer program. This program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes various media that can store program codes, such as read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

Claims

1. A document processing method, characterized in that, The method includes: Displaying a first interface for displaying a video during recording; In response to detecting a first document in the first interface, displaying a first control in the first interface; In response to a first operation on the first control, determining a recording focal length and a recording direction, where the recording focal length is used to adjust the size of the viewfinder frame in the first interface to match the size of the document border of the first document in the first interface, and the recording direction is used to represent the direction of the viewfinder frame in the first interface; Detecting images within the document border of the first document according to the recording focal length and the recording direction to obtain a picture set of the first document, where the picture set of the first document includes one or more pictures; After the video recording ends, generating a second document according to the picture set of the first document; The detecting images within the document border of the first document to obtain the picture set of the first document includes: Determining the document border of the first document in the first interface according to the recording focal length and the recording direction; Performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document, where i is a positive integer taking values greater than or equal to 1, and the quality of the i-th image in multiple optical flow trackings is different; When the number of successful optical flow trackings is greater than or equal to a preset number, adjusting the tilt angle of the i-th image corresponding to when the number of successful optical flow trackings is greater than or equal to the preset number; saving the picture corresponding to the adjusted i-th image in the picture set of the first document; when the ratio between the number of tracked feature points of the i-th image and the number of tracked feature points of the (i - 1)-th image is less than a preset threshold, updating i to i + 1, and continuing to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends; when the ratio between the number of tracked feature points of the i-th image and the number of tracked feature points of the (i - 1)-th image is greater than or equal to the preset threshold, continuing to execute the step of performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document until the number of successful optical flow trackings is greater than or equal to the preset number.

2. The method according to claim 1, characterized in that Determining the recording focal length and the recording direction includes: Determining the current direction of the viewfinder frame in the first interface and the direction of the document border of the first document; When the current direction of the viewfinder frame in the first interface is inconsistent with the direction of the document border of the first document, displaying a second control in the first interface; After receiving a second operation on the second control, adjusting the current direction of the viewfinder frame in the first interface to obtain the recording direction; determining the recording focal length according to the recording direction and the ratio between the document border of the first document and the viewfinder frame in the first interface corresponding to the adjusted direction. When the second operation on the second control is not received after a preset duration, or when the current direction of the viewfinder of the first interface is the same as the direction of the document border of the first document, determine the current direction of the viewfinder of the first interface as the recording direction; determine the recording focal length according to the recording direction and the ratio between the document border of the first document and the viewfinder of the first interface corresponding to the current direction.

3. The method according to claim 2, wherein The detecting the images within the document border of the first document to obtain the picture set of the first document includes: Determine the document border of the first document in the first interface according to the recording focal length and the recording direction; Obtain the i-th image within the document border of the first document, where i is a positive integer taking values greater than or equal to 1; Determine the content change amount between the i-th image and the (i - 1)-th image; When the content change amount between the i-th image and the (i - 1)-th image is greater than or equal to a preset change amount, adjust the tilt angle of the i-th image; save the picture corresponding to the adjusted i-th image in the picture set of the first document; update i to i + 1, and continue to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends; When the content change amount between the i-th image and the (i - 1)-th image is less than the preset change amount, update i to i + 1, and continue to execute the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction until the video recording ends.

4. The method according to claim 3, wherein The determining the content change amount between the i-th image and the (i - 1)-th image includes: Perform optical character recognition and / or layout analysis on the i-th image and the (i - 1)-th image to obtain the content detection situations of the i-th image and the (i - 1)-th image; Determine the content change amount between the i-th image and the (i - 1)-th image according to the content detection situations of the i-th image and the (i - 1)-th image.

5. The method according to claim 3, characterized in that The method further includes: When the number of successful optical flow tracking is less than the preset number, continue to execute the step of performing multiple optical flow trackings on the feature points of the i-th image within the document border of the first document until the number of successful optical flow tracking is greater than or equal to the preset number; When the optical flow tracking is not successful, continue to execute the step of determining the document border of the first document in the first interface until the video recording ends.

6. The method according to claim 5, characterized in that The method specifically includes: Perform the k-th optical flow tracking on the feature points of the i-th image within the document border of the first document, where k is a positive integer taking values greater than or equal to 1; Determine whether the k-th optical flow tracking is successful; When the k-th optical flow tracking is successful, the total number of successful optical flow tracking before the k-th optical flow tracking is incremented by one to determine the number of successful optical flow tracking; when the number of successful optical flow tracking is greater than or equal to the preset number, the tilt angle of the i-th image in the k-th optical flow tracking is adjusted; when the number of successful optical flow tracking is less than the preset number, k is updated to k + 1, and the k-th optical flow tracking of the feature points of the i-th image within the document border of the first document is continued until the number of successful optical flow tracking is greater than or equal to the preset number; When the k-th optical flow tracking is unsuccessful, the step of determining the document border of the first document in the first interface according to the recording focal length and the recording direction is continued until the video recording ends.

7. The method according to any one of claims 3-6, characterized in that After saving the picture corresponding to the adjusted i-th image in the picture set of the first document, the method further includes: When the i-th image is the (i - 1)-th image, determine whether there is occlusion in the i-th image and / or the (i - 1)-th image; When there is occlusion in the i-th image and / or the (i - 1)-th image, according to the occlusion amount between the i-th image and the (i - 1)-th image, save the picture corresponding to the image with a smaller occlusion amount in the picture set of the first document, and delete the picture corresponding to the image with a larger occlusion amount; When there is no occlusion in both the i-th image and the (i - 1)-th image, according to the quality of the i-th image and the (i - 1)-th image, save the picture corresponding to the image with a higher quality in the picture set of the first document, and delete the picture corresponding to the image with a lower quality.

8. The method according to any one of claims 3 to 6, characterized in that, Adjusting the tilt angle of any one image includes: Determining the positions of each vertex of the document border of the first document corresponding to the image in the first interface; Determining the distance from each vertex of the document border of the first document corresponding to the image from the first interface to the presentation interface of the first document; Adjusting the tilt angle of the image according to the positions of each vertex in the first interface and the distance from each vertex from the first interface to the presentation interface of the first document.

9. The method according to any one of claims 1 to 6, characterized in that The method further includes: In response to a third operation on the first control, stop detecting the images within the document border of the first document.

10. The method according to any one of claims 1-6, characterized in that The method further includes: Displaying a third control in the first interface; In response to a fourth operation on the third control, start video recording; Alternatively, in response to a fifth operation on the third control, stop video recording.

11. The method according to any one of claims 1 to 6, characterized in that, Generating a second document according to the picture set of the first document includes: Displaying a second interface for displaying each picture in the picture set of the first document; In response to a sixth operation on the second interface, determining the selected picture in the picture set of the first document; Displaying a fourth control in the second interface; In response to a seventh operation on the fourth control, generating the second document according to the selected picture.

12. The method according to claim 11, wherein Before generating the second document according to the selected picture in response to a seventh operation on the fourth control, the method further includes: Displaying a fifth control in the second interface; In response to an eighth operation on the fifth control, perform at least one of image enhancement, image shadow removal, or image grayscale conversion on the selected picture.

13. An electronic device, characterized in that, It includes a processor and a memory, the memory is coupled to the processor, and the memory is used to store computer program code. The computer program code includes computer code or instructions. When the processor executes the computer code or instructions, the electronic device executes the document processing method described in any one of claims 1-12.

14. A chip system, characterized in that, The chip system is applied to an electronic device including a memory, a display screen, and a sensor; the chip system includes: one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the memory and send signals to the processors, and the signals include computer code or instructions stored in the memory; when the processor executes the computer code or instructions, the electronic device executes the document processing method described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, Code or instructions are stored in the computer-readable storage medium. When the code or instructions run on the electronic device, the electronic device executes the document processing method described in any one of claims 1-12.

Images