Document shooting method and related device

Abstract

This application provides a document capture method and related apparatus, applied in the field of terminal technology. The method includes: when the angle between the electronic device and the subject (including text) is greater than a first threshold, and the distance between the camera and the subject is less than a second threshold, in response to an operation on a camera control, obtaining a first image; the first image is obtained by fusing multiple captured first source images, where the multiple first source images contain images with different focus areas; when the angle between the electronic device and the subject is less than or equal to the first threshold, or the distance between the camera and the subject is greater than or equal to the second threshold, in response to an operation on a camera control, obtaining a second image; the second image is obtained by fusing multiple captured second source images, where the focus areas of the multiple second source images are all the same, or the second image is obtained by processing a captured second source image. This improves the user experience.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a document shooting method and related devices. Background Art

[0002] Some electronic devices have a photographing function. Users can use the electronic device to photograph a document so as to view the document photo or extract text content from the document photo on the electronic device.

[0003] However, when a user uses an electronic device to take a photo of a document, there may be a problem that it takes a long time from the user clicking the photo control to the electronic device obtaining the document photo, resulting in a poor user experience. Summary of the invention

[0004] The embodiment of the present application provides a document shooting method and related devices, which are applied to the field of terminal technology. When an electronic device shoots a shooting object containing text, the identified document scene is further judged. When it is judged that the document scene is a scene of shooting a document at a long distance or a scene of shooting a document without tilting, the electronic device processes one or more source images collected after focusing once to obtain a document photo with uniform clarity. When it is judged that the document scene is a scene of shooting a document at a close distance and tilting, the electronic device performs fusion processing on multiple source images collected after multiple focusing to obtain a document photo with uniform clarity. Among them, one focusing can be understood as the position of the focus area remains unchanged when one or more source images are collected, and multiple focusing can be understood as the position of the focus area is changed when multiple source images are collected. In this way, in the scene where the electronic device shoots a document at a long distance or shoots a document without tilting, the time consumption of the position change of the focus area is reduced, and while obtaining a document photo with uniform clarity, the time from when the photo control is clicked to when the electronic device obtains the document photo is shortened, which can improve the user experience. When an electronic device takes a photo of a document at an angle at a close distance, although the position of the focus area needs to be changed, which makes it take longer from the time the photo control is clicked to the time the electronic device obtains the document photo, it reduces the probability of uneven clarity of the document photo caused by taking the document at an angle at a close distance, and can also improve the user experience.

[0005] In a first aspect, an embodiment of the present application provides a document shooting method, the method comprising:

[0006] In the case where the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, a first image is obtained in response to an operation on a photographing control displayed on the electronic device. The first image is obtained by fusing multiple frames of first source images captured by the electronic device, and there are images with different focus areas in the multiple frames of the first source images. In the case where the angle between the electronic device and the photographed object is less than or equal to the first threshold, or the distance between the camera and the photographed object is greater than or equal to the second threshold, a second image is obtained in response to an operation on a photographing control. The second image is obtained by fusing multiple frames of second source images captured by the electronic device, and the focus areas of the multiple frames of second source images are the same, or the second image is obtained by the electronic device processing a captured frame of the second source image.

[0007] In this way, when the document scene is a close-up oblique shooting scene, in response to the operation acting on the camera control, the electronic device can obtain a first image with uniform clarity by fusing the source images obtained by multiple focusing, reducing the probability of the first image obtained due to close-up oblique shooting of the document having some areas clear and some areas blurred, which can improve the user experience. The first image can be a document photo obtained in the embodiment when the document scene is a close-up oblique shooting scene. It is also implemented that when the document scene is a non-oblique shooting scene or the document scene is a long-distance shooting scene, by performing image processing on the source image obtained by one focus, a second image with uniform clarity can be obtained. Compared with multiple focusing, one focus takes less time, which can reduce the time from the operation of the camera control to the acquisition of the second image, thereby improving the user experience. The second image obtained in the scene of non-oblique shooting of the document can be a document photo obtained in the embodiment when the document scene is a non-oblique shooting scene or the document scene is a long-distance shooting scene.

[0008] In a possible implementation, in response to an operation of a photo control on a display of an electronic device, before obtaining a first image, the method further includes: when it is determined that the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, configuring a first photo processing flow. The first photo processing flow is used to instruct the camera of the electronic device to focus multiple times, and to capture one or more frames of images each time it focuses, and is also used to fuse the captured images to obtain a photo. The focus areas of two adjacent focuses are different.

[0009] In this way, when the document scene is a close-up oblique shooting scene, multiple focusing and image acquisition can be performed when an operation acting on the photo control is received, and a document photo with uniform clarity can be obtained by processing the acquired images, thereby reducing the probability of some areas of the document photo being clear and some areas being blurred when the document is photographed at a close distance and obliquely.

[0010] In a possible implementation, determining whether the angle between the electronic device and the photographed object is greater than a first threshold includes: determining whether the absolute value of the difference between the length of a first side of the photographed object and the length of a second side of the photographed object in the preview image is greater than a third threshold, and / or whether the ratio of the length of the first side to the length of the second side in the preview image is less than a fourth threshold. The first side and the second side are at different distances from the camera of the electronic device, the length of the first side is less than the length of the second side, and the first side and the second side have no intersection in the photographed object. When the absolute value of the difference between the length of the first side of the photographed object and the length of the second side of the photographed object is greater than the third threshold, and / or the ratio of the length of the first side to the length of the second side is less than the fourth threshold, it indicates that the angle between the electronic device and the photographed object is greater than the first threshold.

[0011] The ratio of the length of the first side to the length of the second side or the absolute value of the difference between the length of the first side and the length of the second side represents the difference between the length of the first side and the length of the second side. In this way, based on the difference between the length of the first side and the length of the second side, it is possible to determine whether the angle between the electronic device and the document is greater than the first threshold, and further determine whether the document scene is an oblique shooting scene.

[0012] In a possible implementation, the photographed object includes a first vertex, a second vertex, a third vertex, and a fourth vertex. The two endpoints of the first side are the first vertex and the second vertex, respectively, and the two endpoints of the second side are the third vertex and the fourth vertex, respectively. In the coordinate system of the preview window of the electronic device displaying the preview image, the absolute value of the difference between the ordinate of the first vertex and the ordinate of the second vertex is less than a fifth threshold, and the absolute value of the difference between the ordinate of the third vertex and the ordinate of the fourth vertex is less than the fifth threshold. Determining whether the ratio of the length of the first side to the length of the second side in the preview image is less than a fourth threshold includes: determining whether the ratio of the minimum length to the maximum length of the first side and the second side is less than the fourth threshold.

[0013] In this way, based on the comparison of the ratio of the minimum side length to the maximum side length of the first side and the second side with the fourth threshold, it is possible to determine whether the angle between the electronic device and the document is greater than the first threshold, and then determine whether the document scene is an inclined shooting scene.

[0014] In a possible implementation, before determining whether the angle between the electronic device and the photographed object is greater than the first threshold, the method further includes: determining whether the first probability of the continuous m-frame preview image is greater than the sixth threshold. The first probability is the sum of the probabilities that the absolute value of the difference between the lengths of the first sides corresponding to two adjacent frames of preview images in the continuous m-frame preview image is less than the seventh threshold. Determining whether the angle between the electronic device and the photographed object is greater than the first threshold includes: when determining that the first probability is greater than the sixth threshold, determining whether the angle between the electronic device and the photographed object is greater than the first threshold.

[0015] In this way, based on the absolute value of the difference in side lengths between two adjacent preview images in the continuous m frames of preview images, it is possible to accurately determine whether the document scene is stable, and further, based on whether the document scene is stable, it is possible to accurately determine whether the user has the intention to photograph the document. When it is determined that the document scene is stable or the user has the intention to photograph the document, a determination is made as to whether the document scene is an inclined shooting scene, which can reduce the power consumption generated by determining whether the document scene is an inclined shooting scene when the user does not have the intention to photograph the document.

[0016] In a possible implementation, determining whether the distance between the camera of the electronic device and the photographed object is less than a second threshold includes: determining whether the shooting distance corresponding to the focus parameter associated with the preview image is less than the second threshold. Wherein, if the shooting distance corresponding to the focus parameter associated with the preview image is less than the second threshold, it means that the distance between the camera of the electronic device and the photographed object is less than the second threshold.

[0017] In this way, it is possible to determine whether the photographing scene is a close-up shooting scene, so as to determine whether the document scene is a close-up tilted scene.

[0018] In a possible implementation, determining whether the distance between the camera of the electronic device and the photographed object is less than a second threshold includes: determining whether the distance between the camera and the photographed object obtained from a laser ranging device or an ultrasonic ranging device of the electronic device when capturing a preview image is less than the second threshold. Wherein, when the distance between the camera and the photographed object is less than the second threshold when capturing the preview image, it indicates that the distance between the camera of the electronic device and the photographed object is less than the second threshold.

[0019] In this way, it is possible to determine whether the photographing scene is a close-up shooting scene, so as to determine whether the document scene is a close-up tilted scene.

[0020] In a possible implementation, in response to an operation on a photographing control, before obtaining a second image, the method further includes: configuring a second photographing process when it is determined that the angle between the electronic device and the photographing object is less than or equal to a first threshold, or when it is determined that the distance between the camera and the photographing object is greater than or equal to a second threshold. The second photographing process is used to instruct the camera to focus once, collect one or more frames of images, and process the collected images to obtain a photo.

[0021] In this way, when the document scene is a non-inclined shooting scene or a long-distance shooting scene, when an operation acting on the photo control is received, a focus is performed to capture the image, and a document photo with uniform clarity is obtained by processing the captured image. Since multiple focusing is not required, the time from when the photo control is clicked to when the electronic device obtains the document photo is reduced.

[0022] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory. The memory stores computer-executable instructions. The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method of the first aspect.

[0023] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0024] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method of the first aspect.

[0025] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method of the first aspect.

[0026] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A photographing scene diagram provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;

[0029] Figure 3 A software structure block diagram of the electronic device 100 provided in an embodiment of the present application;

[0030] Figure 4 A flowchart of a document shooting method provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of each vertex of a document in a preview image provided by an embodiment of the present application;

[0032] Figure 6 A schematic diagram of a first photo processing flow provided in an embodiment of the present application;

[0033] Figure 7 A module interaction flowchart provided for an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0035] 1. Electronic equipment

[0036] The electronic device of the embodiment of the present application may include a handheld device, a vehicle-mounted device, etc. with an image processing function. For example, some electronic devices are: tablet computers, mobile phones, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices (such as vehicle computers), wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (public land mobile The embodiments of the present application do not limit this.

[0037] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating systems, Unix operating systems, Android operating systems, iOS operating systems, or Windows operating systems. The application layer includes applications such as cameras, browsers, address books, word processing software, and instant messaging software.

[0038] 2. Other terms

[0039] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0040] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0041] In the embodiment of the present application, "at..." may be the instant when a certain situation occurs, or may be a period of time after a certain situation occurs, and the embodiment of the present application does not specifically limit this. In addition, the display interface provided in the embodiment of the present application is only an example, and the display interface may also include more or less content.

[0042] Figure 1 A photographing scene diagram provided by an embodiment of the present application is shown.

[0043] When the electronic device receives an operation to open the camera application, the electronic device may display Figure 1 The interface shown in a. Figure 1The interface shown as a in [description], which interface includes a preview window 101, and a document 102, a control 103, a photographing control 104, a control 105, more controls, a video recording control, a photographing control, a portrait control, a document photographing control, a control 106, a control 107, a control 108, a control 109, a control 110, a control 111, and an identifier 112 displayed in the preview window 101. Among them, the document 102 is a document containing the text "aaaa bbbb Page 1". The control 103 can be a control for switching between the front camera and the rear camera. The control 105 can be a control for opening the gallery application. The gallery application can be referred to as an album. The control 106 can be a control for setting the camera application. The control 107 is a control representing filters. The control 108 can be used to turn on or off the flash. The control 109 can be used to identify whether the photographing scene is a document scene. The control 110 can be a control representing intelligent object recognition. The control 111 can be used to adjust the shooting magnification. The identifier 112 can be used to indicate that the recognized photographing scene is a document scene.

[0044] The document scene can be understood as in a photographing scene, the shooting object containing text is the main shooting subject. The main shooting subject can be understood as the proportion of the area of the shooting object in the document photo or the preview image being greater than the area threshold.

[0045] In Figure 1 On the interface shown as a in [description], the user can perform a click operation on the photographing control 104, and the electronic device can display Figure 1 the interface shown as b in [description]. As Figure 1 shown in the interface shown as b in [description], thumbnails of document photos can be displayed in the control 105, Figure 1 For other content of the interface shown as b in [description], refer to Figure 1 the content of the interface shown as a in [description], which will not be elaborated here.

[0046] As Figure 1 shown in a and b in [description], some electronic devices can, based on the preview image displayed in the preview window, identify that the photographing scene is a document scene. In some implementations, when the electronic device identifies that the photographing scene is a document scene, in response to a click operation on the photographing control, the electronic device will capture at least one image with the focus area in the central area of the preview window and at least one image with the focus area in the edge area of the preview window, and synthesize the captured multiple images to obtain a document photo with relatively uniform clarity. Since the time taken for the electronic device to capture at least one image with the focus area in the central area of the preview window and at least one image with the focus area in the edge area of the preview window is relatively long, the time from when the photographing control is clicked to when the electronic device obtains the document photo is relatively long, resulting in a poor user experience.

[0047] In view of this, an embodiment of the present application provides a document shooting method. When an electronic device shoots a shooting object containing text, the identified document scene is further judged. When it is judged that the document scene is a scene of shooting the document at a long distance or a scene of shooting the document without being tilted, the electronic device processes one or more source images captured after one focusing to obtain a document photo with uniform clarity; when it is judged that the document scene is a scene of shooting the document at a close distance and tilted, the electronic device fuses multiple source images captured after multiple focusing to obtain a document photo with uniform clarity.

[0048] Among them, single focus can be understood as the position of the focus area remains unchanged when collecting one or more source images, and multiple focus can be understood as the position of the focus area changes when collecting multiple source images. The source image can be understood as the image used to obtain the document photo. The photographed object containing text can be a document such as a courier slip, test paper, card, barcode, poster or mobile phone screen. The photographed object containing text can also be, for example Figure 1 The document 102 shown in a in FIG.

[0049] In this way, in the scenario where the electronic device shoots a document at a long distance or shoots a document without tilting, the time consumption of changing the position of the focus area is reduced, and while obtaining a document photo with uniform clarity, the time from when the photo control is clicked to when the electronic device obtains the document photo is shortened, which can improve the user experience. When the electronic device shoots a document at a close distance and tilts, although the position of the focus area needs to be changed, which makes it take a long time from when the photo control is clicked to when the electronic device obtains the document photo, the probability of uneven clarity of the document photo caused by close-range tilted shooting of the document is reduced, which can also improve the user experience.

[0050] This is because, when an electronic device shoots a document at a close distance without tilting, although the depth of field is small, the distance difference between different areas of the document and the camera is small, so that different areas of the document can be in the same depth of field range, and thus the clarity of the captured source image is relatively uniform. The source image captured with a single focus can be processed to obtain a document photo with uniform clarity.

[0051] When an electronic device shoots a document at a long distance, due to the large depth of field, different areas of the document can be in the same depth of field range, so that the clarity of the captured source image is more uniform. The source image captured by focusing once can be processed to obtain a document photo with uniform clarity.

[0052] When an electronic device takes a close-up tilted photo of a document, due to the small depth of field, the distances between different areas of the document and the camera vary greatly, so that different areas of the document are not in the same depth of field range, resulting in uneven clarity of the source image taken, for example, some areas of the source image are clear and some areas are blurred. Therefore, it is necessary to focus and take multiple source images multiple times, and there are images with different clear areas in the multiple source images, and then fuse the multiple source images to obtain a document photo with uniform clarity.

[0053] Combine the following Figure 2 and Figure 3 The hardware and software architecture of the electronic device in the embodiments of the present application is described.

[0054] Figure 2 A schematic structural diagram of an electronic device 100 is shown.

[0055] The electronic device 100 may include a processor 200, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a display screen 194. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0056] Exemplarily, when the electronic device determines that the document scene is a scene of shooting the document at a close distance and tilting, when the electronic device receives an operation on the camera control, the camera 193 may focus multiple times to collect multiple frames of source images. The processor 200 may perform fusion processing on the multiple frames of source images collected by the camera 193 to obtain a document photo with uniform clarity. When the electronic device determines that the document scene is a scene of shooting the document at a long distance or a scene of shooting the document without tilting, when the electronic device receives an operation on the camera control, the camera 193 may focus once to collect one or more frames of source images. The processor 200 may process the one or more frames of source images collected by the camera 193 to obtain a document photo with uniform clarity.

[0057] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0058] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.

[0059] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0060] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the system library, and the kernel layer.

[0061] The application layer may include a series of application packages and may also include a coordinate detection module.

[0062] like Figure 3 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, lock screen and other applications.

[0063] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0064] The application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a camera module, a scene detection module, a coordinate processing module, a tilt judgment module, a fusion judgment module, a shooting distance judgment module and an Android runtime, etc. Figure 3 Only a portion is shown.

[0065] Among them, the scene detection module, the coordinate processing module, the tilt judgment module, the fusion judgment module and the shooting distance judgment module are newly added modules of this solution, and their respective functions are described in the subsequent embodiments. The camera module can use the preview processing flow to process the image collected by the camera to obtain the preview image to realize the display of the preview image. The camera module can also use the photo processing flow to process the image collected by the camera to obtain a photo.

[0066] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0067] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0068] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0069] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0070] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0071] The hardware includes the camera.

[0072] Exemplarily, when the camera application is opened and no operation acting on the photo control is received, the camera module can use the preview processing flow to process the image collected by the camera to obtain a preview image. The camera module can store the obtained preview image in a preview buffer for storing the preview image. The camera module can transmit information indicating that the preview image has been stored to the camera application, the shooting distance judgment module, and the scene detection module respectively. The information indicating that the preview image has been stored includes a preview image identifier.

[0073] When receiving the information indicating that the preview image has been stored, the camera application may obtain the preview image corresponding to the preview image identifier from the preview cache for display.

[0074] When receiving information indicating that the preview image has been stored, the shooting distance judgment module can obtain the preview image corresponding to the preview image identifier from the preview cache to judge whether the shooting scene is a close-up shooting scene. If the shooting scene is judged to be a close-up shooting scene, the information indicating that the shooting scene is a close-up shooting scene is transmitted to the fusion judgment module.

[0075] When receiving information indicating that the preview image has been stored, the scene detection module can obtain the preview image corresponding to the preview image identifier from the preview cache to determine whether the photographing scene is a document scene. When the scene detection module determines that the photographing scene is a document scene, the scene detection module transmits the preview image to the coordinate detection module. The coordinate detection module identifies each vertex of the document in the preview image and obtains the coordinates of each vertex. The coordinate detection module transmits the coordinates of each vertex of the preview image to the coordinate processing module. The coordinate processing module determines whether the document scene is stable based on the coordinates of each vertex of the continuous multiple frames of preview images. When the document scene is determined to be stable, the coordinate processing module transmits the coordinates of each vertex of the preview image to the tilt determination module. The tilt determination module determines whether the document scene is an inclined shooting scene based on the coordinates of each vertex of the received preview image. If the document scene is determined to be an inclined shooting scene, information indicating that the document scene is an inclined shooting scene is transmitted to the fusion determination module. If the document scene is determined to be a non-inclined shooting scene, information indicating that the document scene is a non-inclined shooting scene is transmitted to the camera application, and the camera application configures a second photographing processing flow. The second photo processing flow is used to instruct the camera to focus and capture an image when the electronic device receives an operation acting on the photo control, and to process the captured image to obtain a document photo with uniform clarity.

[0076] When the fusion judgment module receives information indicating that the document scene is an oblique shooting scene and information indicating that the photographing scene is a close-up shooting scene, the fusion judgment module transmits information indicating that the document scene is a close-up oblique shooting scene to the camera application, and the camera application configures a first photographing processing flow. The first photographing processing flow is used to instruct the camera to focus and capture images multiple times when the electronic device receives an operation acting on the photographing control, and to perform fusion processing on the captured images to obtain a document photo with uniform clarity.

[0077] Combine the following Figure 4 The document shooting method provided in the embodiment of the present application is described.

[0078] Figure 4 FIG. 1 shows a flow chart of a document photographing method provided by an embodiment of the present application. Figure 4 As shown, the document shooting method provided in the embodiment of the present application includes:

[0079] S401: The electronic device performs scene detection according to the camera preview stream, and determines whether the photographing scene is a document scene.

[0080] The camera preview stream is a preview image displayed in real time by the electronic device when the camera application is open.

[0081] Exemplarily, in response to the operation of opening the camera application, the electronic device may process the image collected by the camera according to the preview processing flow to obtain a preview stream (or preview image). The preview image may be stored in a preview cache.

[0082] The electronic device obtains a preview image from the preview buffer and extracts features from the preview image. Based on the extracted feature data of the preview image, the electronic device determines whether the photographing scene is a document scene.

[0083] Exemplarily, the electronic device extracts feature data of the photographed object from the preview image, inputs the feature data into a classifier of the electronic device, and obtains an output result corresponding to the feature data. The classifier is a model obtained by training with a data set containing feature data of multiple photographed objects. The classifier can be used to identify the input feature data of the photographed object and obtain an output result indicating the category of the photographed object. The category of the photographed object can be any one of the categories such as document, portrait, cat or dog, sunset, blue sky, flower or snow.

[0084] If the output result obtained indicates that the category of the photographed object is a document, the electronic device determines that the photographing scene is a document scene, and the electronic device may execute S402.

[0085] If the output result obtained indicates that the category of the photographed object is one of portrait, cat and dog, sunset, blue sky, flower or snow, the electronic device determines that the photographing scene is a non-document scene, and the electronic device can execute S412. The non-document scene can be any scene of the portrait scene, cat and dog scene, sunset scene, blue sky scene, flower scene or snow scene.

[0086] For example, if the output result obtained indicates that the category of the photographed object is a portrait, the electronic device determines that the photographing scene is a portrait scene, and the electronic device can execute S412 to configure a photographing processing flow corresponding to the portrait scene.

[0087] Optionally, when the electronic device obtains the preview image from the preview buffer, it can identify whether the photographing scene is a document scene through the image histogram, and if so, execute step S402. If not, obtain another frame of preview image from the preview buffer.

[0088] The image histogram is used to identify whether the current photo-taking scene is a document scene. The algorithm is simple and the recognition speed is fast.

[0089] It can be understood that the electronic device may execute step S401 once each time it obtains a frame of preview image.

[0090] S402: When determining that the photographing scene is a document scene, the electronic device may identify each vertex of the photographed object (or document) in the preview image, and calculate the coordinates of each vertex of the document in the preview image according to the coordinate system of the preview window.

[0091] For example, Figure 5 A schematic diagram showing vertices of a document in a preview image provided by an embodiment of the present application is shown.

[0092] like Figure 5 As shown, the vertices of the document are A, B, C and D respectively.

[0093] The coordinates of each vertex are: A(x 1 ,y 1 ), B(x 2 ,y 2 ), C(x 3 ,y 3 ), D(x 4 ,y 4 ).

[0094] It can be understood that the electronic device may execute step S402 once each time it obtains a frame of preview image.

[0095] S403, the electronic device calculates the length of the first side and the length of the second side of the document in the preview image, and determines whether the document scene is stable according to the length of the first side and / or the length of the second side corresponding to the continuous m frames of preview images. If the document scene is determined to be stable, step S404 is executed. If the document scene is determined to be unstable, step S404 is not executed.

[0096] The document scene is stable, which means that the angle between the electronic device and the document and the distance between the camera and the document do not change. The document scene is stable, which means that the user holds the electronic device steadily without shaking, the document scene does not switch to a non-document scene, and the user has the intention to take a photo of the document.

[0097] Therefore, when the document scene is stable, it is necessary to execute S404 to further determine whether the document scene is an oblique shooting scene.

[0098] The instability of the document scene can be understood as a change in the angle between the electronic device and the document and / or the distance between the camera and the document, which may indicate that the user has no intention of taking a photo of the document. The electronic device does not need to perform subsequent steps to save power consumption of the electronic device.

[0099] For example, the electronic device obtains the coordinates A(x 1 ,y 1 ), B(x2 ,y 2 ), C(x 3 ,y 3 ), D(x 4 ,y 4 ), the first side and the second side corresponding to the preview image are obtained as follows, and the length of the first side and the length of the second side are calculated: Based on the coordinates A(x1, y1), B(x2, y2), C(x3, y3) and D(x4, y4) of each vertex of the preview image, if the electronic device determines that |y 1 -y 2 |<fifth threshold, |y 3 -y 4 |<fifth threshold value, the electronic device obtains that the first side is AB, obtains that the second side is CD, and the electronic device obtains that the length of the first side is the length of AB, and obtains that the length of the second side is the length of CD.

[0100] Among them, the length of AB and the length of CD satisfy the following formula:

[0101]

[0102] In this way, the preview image obtains the coordinates of each vertex of the document in each frame of the preview image, and the lengths of two parallel sides of the document in this frame of the preview image can be obtained.

[0103] When the electronic device obtains the length of the first side and the length of the second side corresponding to m consecutive preview images, the electronic device determines whether the document scene is stable based on the length of the first side and / or the length of the second side corresponding to each preview image in the m consecutive preview images, where m≥2.

[0104] Exemplarily, the electronic device can determine whether the angle between the electronic device and the document and the distance between the camera and the document in the document scene have changed based on the probability that the absolute value of the difference in side lengths of two adjacent preview images in m consecutive preview image frames is less than the seventh threshold.

[0105] For example, the electronic device may determine whether the first probability R1 is greater than a sixth threshold.

[0106] If so, it means that the angle between the electronic device and the document and the distance between the camera and the document have not changed, and the electronic device can execute step S404.

[0107] If not, it means that the angle between the electronic device and the document and / or the distance between the camera and the document has changed, and the electronic device does not execute step S404.

[0108] The first probability R1 satisfies the formula: R1=n1 / (m-1), wherein n1 is the number of times the absolute value of the difference between the first sides of two adjacent preview images in the continuous m-frame preview images is less than the absolute value of the seventh threshold, and m-1 is the number of times the absolute value of the difference between the first sides of two adjacent preview images in the continuous m-frame preview images. The first probability R1 can also be expressed as the sum of the probabilities that the absolute value of the difference between the first sides of two adjacent preview images in the continuous m-frame preview images is less than the seventh threshold.

[0109] Optionally, the electronic device may determine whether the second probability R2 is greater than a sixth threshold.

[0110] If so, it means that the angle between the electronic device and the document and the distance between the camera and the document have not changed, and the electronic device executes step S404.

[0111] If not, it means that the angle between the electronic device and the document and / or the distance between the camera and the document has changed, and the electronic device does not execute step S404.

[0112] Among them, the second probability R2 satisfies the formula: R2=N / [2(m-1)], wherein N=n1+n2, n2 is the number of absolute values ​​of the difference in length of the second sides of two adjacent preview images in the continuous m preview images that are less than the absolute value of the seventh threshold, and 2(m-1) is the sum of the number of absolute values ​​of the difference in length of the first sides of two adjacent preview images in the continuous m preview images and the number of absolute values ​​of the difference in length of the second sides of two adjacent preview images in the continuous m preview images.

[0113] Optionally, the electronic device may determine whether the third probability R3 is greater than a sixth threshold.

[0114] If so, it means that the angle between the electronic device and the document and the distance between the camera and the document have not changed, and the electronic device executes step S404.

[0115] If not, it means that the angle between the electronic device and the document and / or the distance between the camera and the document has changed, and the electronic device does not execute step S404.

[0116] The third probability R3 satisfies the formula: R3=n2 / (m-1).

[0117] It is understandable that if m=2, the electronic device can determine whether the document scene is stable when obtaining two consecutive preview frames. If m=8, the electronic device can determine whether the document scene is stable when obtaining eight consecutive preview frames.

[0118] In this way, based on the absolute value of the difference between the side lengths of two adjacent preview images in the continuous m preview images, it can be accurately determined whether the user intends to photograph the document.

[0119] S404: The electronic device determines whether the document scene is an oblique shooting scene according to the length of the first side and the length of the second side of the preview image. If yes, the electronic device may execute step S408. If no, the electronic device may execute step S409.

[0120] Taking the example of a document containing text and A4 paper as the object of photography, the lengths of the two parallel sides of the document are equal. When the electronic device is parallel to the document, for example, when the angle between the electronic device and the document is zero, the length of the first side and the length of the second side of the document in the preview image obtained by the electronic device are equal. When there is an angle greater than zero and less than 90° between the electronic device and the document, the length of the first side and the length of the second side of the document in the preview image obtained by the electronic device are not equal. The existence of an angle greater than zero and less than 90° between the electronic device and the document indicates that the document scene is an oblique shooting scene. Referring to this principle, the electronic device can determine whether the document scene is an oblique shooting scene according to the length of the first side and the length of the second side of the preview image in the following manner one or manner two.

[0121] Method 1: The electronic device determines whether the absolute value of the difference between the side length of the first side of the document and the side length of the second side of the document in the m-th frame preview image is greater than a third threshold, and / or whether a ratio R4 of the minimum side length to the maximum side length of the first side of the document and the second side of the document in the m-th frame preview image is less than a fourth threshold.

[0122] If the absolute value of the difference between the length of the first side of the document and the length of the second side of the document in the m-th frame preview image is greater than the third threshold, and / or if the ratio R4 of the minimum side length to the maximum side length of the first side of the document and the second side of the document in the m-th frame preview image is less than the fourth threshold, it indicates that the document scene corresponding to the m-th frame preview image is an inclined shooting scene, and the electronic device can execute step S408.

[0123] Wherein, R4 satisfies the formula R4=min{side length of the first side, side length of the second side} / max{side length of the first side, side length of the second side}. For example, the four vertices of the document in the preview image of the mth frame are A m , B m , C m and D m , the first side corresponding to the preview image of the mth frame is A m B m , the second corresponding to the preview image of the mth frame becomes C m D mFor example, R4 satisfies the formula: R4=min{A m B m Length, C m D m length} / max{A m B m Length, C m D m length}.

[0124] The document scene being an oblique shooting scene can be understood as an angle between the electronic device and the shooting object (or document) being greater than a first threshold.

[0125] If the absolute value of the difference between the length of the first side of the document and the length of the second side of the document in the m-th frame preview image is less than or equal to the third threshold, and / or if the ratio R4 of the minimum side length to the maximum side length of the first side of the document and the second side of the document in the m-th frame preview image is greater than or equal to the fourth threshold, it indicates that the document scene corresponding to the m-th frame preview image is a non-tilted shooting scene, and the electronic device can execute step S409.

[0126] The document scene being a non-inclined shooting scene can be understood as an angle between the electronic device and the shooting object (ie, the document) being less than or equal to a first threshold.

[0127] Method 2: The electronic device determines whether a ratio R5 of the minimum side length to the maximum side length of the first side and the second side corresponding to the (m-1)th frame preview image and the (m)th frame preview image is less than a fourth threshold.

[0128] If the ratio R5 of the minimum side length to the maximum side length of the first side and the second side corresponding to the m-1th frame preview image and the mth frame preview image is less than the fourth threshold, it indicates that the document scene corresponding to the mth frame preview image is an oblique shooting scene, and the electronic device can execute step S408.

[0129] If the ratio R5 of the minimum side length to the maximum side length of the first side and the second side corresponding to the m-1th frame preview image and the mth frame preview image is greater than or equal to the fourth threshold, it indicates that the document scene corresponding to the mth frame preview image is a non-inclined shooting scene, and the electronic device can execute step S409.

[0130] For example, the four vertices of the document in the preview image of the mth frame are A m , B m , C m and D m , the first side corresponding to the preview image of the mth frame is A m B m , the second corresponding to the preview image of the mth frame becomes C m D m , the four vertices of the document in the preview image of the m-1th frame are A m-1, B m-1 , C m-1 and D m-1 , the first edge corresponding to the preview image of the m-1th frame is Am-1 B m-1 , the second corresponding to the preview image of the m-1th frame becomes C m-1 D m-1 For example, R5 satisfies the formula: R5=min{A m B m Length, C m D m Length, A m-1 B m-1 Length, C m-1 D m-1 length} / max{A m B m Length, C m D m Length, A m-1 B m-1 Length, C m-1 D m-1 length}.

[0131] Exemplarily, the fourth threshold value may be a positive number less than 1, or may be equal to 1. The third threshold value may be 0. The third threshold value may also be greater than 0.

[0132] The document scene being an oblique shooting scene can be understood as the angle between the electronic device and the document being greater than a first threshold. The document scene being a non-oblique shooting scene can be understood as the angle between the electronic device and the document being less than or equal to the first threshold.

[0133] Optionally, when the electronic device completes step S402 to obtain the coordinates of each vertex of the document in the preview image, the electronic device may execute this step when calculating the length of the first side and the length of the second side of the document in the preview image, so that the configuration of the first photo processing flow can be completed as soon as possible afterwards.

[0134] S405: When the electronic device obtains the preview image from the preview buffer, the electronic device may obtain a focus parameter of the preview image.

[0135] Exemplarily, when the electronic device obtains the preview image from the preview buffer, the electronic device may also obtain a motor focus code corresponding to the preview image identifier from the buffer storing acquisition information of the preview image.

[0136] It can be understood that, each time the electronic device obtains a preview image frame from the preview cache, this step can be performed once to obtain the motor focus code corresponding to each preview image frame.

[0137] Optionally, the electronic device may also perform this step when determining that the document scene is an oblique shooting scene according to the length of the first side and the length of the second side of the mth frame preview image to obtain the motor focus code of the mth frame preview image.

[0138] S406: The electronic device obtains a shooting distance corresponding to the motor focus code.

[0139] Exemplarily, an object distance mapping table is stored in the electronic device. The object distance mapping table includes a correspondence between the motor focus code and the shooting distance. The electronic device can search the object distance mapping table for the shooting distance corresponding to the motor focus code obtained in step S405.

[0140] It can be understood that the electronic device can execute this step once each time it obtains a motor focus code corresponding to a frame of preview image.

[0141] S407: The electronic device determines whether the photographing scene is a close-up photographing scene according to the shooting distance corresponding to the preview image. If yes, the electronic device may execute step S408. If no, the electronic device may execute step S411.

[0142] Exemplarily, the electronic device determines whether the shooting distance obtained in step S406 is less than a second threshold. The second threshold may be the maximum distance between the camera and the shooting object when the shooting object is the shooting subject. The second threshold may be any value in the range of 30 cm to 40 cm. For example, the second threshold may be 30 cm, 40 cm, or 35 cm.

[0143] If the shooting distance is less than the second threshold, it means that the shooting scene corresponding to the preview image is a close-up shooting scene, and the electronic device may execute step S408.

[0144] If the shooting distance is greater than or equal to the second threshold, it means that the shooting scene corresponding to the preview image is a long-distance shooting scene, and the electronic device can execute step S411.

[0145] It can be understood that the electronic device may execute this step once each time it obtains a shooting distance corresponding to a frame of preview image.

[0146] S408: When the electronic device determines that the document scene corresponding to the m-th frame preview image is an oblique shooting scene, and determines that the photographing scene corresponding to the m-th frame preview image is a close-up shooting scene, the electronic device configures a first photographing processing flow (pipeline).

[0147] Exemplarily, the photo processing flow may include image acquisition, image front end (IFE), Bayer processing segment (BPS), algorithm processing and photo storage.

[0148] When the electronic device determines that the document scene corresponding to the m-th frame preview image is an oblique shooting scene, and determines that the shooting scene corresponding to the m-th frame preview image is a close-up shooting scene, the electronic device determines that the document scene corresponding to the m-th frame preview image is a close-up oblique shooting scene, and the electronic device obtains configuration information of each step of the first shooting processing flow from a preset configuration file, and configures each step of the shooting processing flow to obtain the first shooting processing flow.

[0149] The configuration information corresponding to the image acquisition step may include configuration parameters such as resolution, the number of frames acquired by the camera focusing once, the number of camera focusing times k, and the position of the focus area for each focus, where k≥2. Optionally, the position of the focus area for each focus can be replaced by the focus distance for each focus. It should be understood that when the camera performs k focus acquisition images according to the configuration parameters corresponding to the image acquisition step, the focus areas of two adjacent focuses are different, or the focus distances of two adjacent focuses are different.

[0150] The configuration information of the front-end processing may include parameters required for color correction, down-sampling, and demosaicing statistics, etc., which are required for the front-end processing.

[0151] The configuration information of the Bayer processing may include parameters for bad pixel removal, phase focusing, demosaicing, downsampling, high-dynamic range (HDR) processing, and Bayer hybrid noise reduction processing, etc., which are required for the Bayer processing.

[0152] The configuration information corresponding to the algorithm processing steps may include a multiple frame noise reduction (MFNR) algorithm, a depth of field fusion algorithm, a text super resolution (SR) enhancement algorithm, and an effect sharpening debugging algorithm. The depth of field fusion algorithm may include image registration, ghost detection, clarity detection, image fusion, and full depth of field output.

[0153] The configuration information of the photo storage may include the photo storage path.

[0154] The first photographing process can be used to instruct the camera of the electronic device to focus multiple times (or k times), and to collect one or more frames of images each time it focuses, and to fuse the collected images to obtain a photo. The focus areas of two adjacent focusing operations are different. Figure 6 For ease of understanding, Figure 6 The embodiments are described below.

[0155] The different focus areas of two adjacent focusing operations can be understood as different positions of the clear areas in the two adjacent focusing operations. By fusing multiple frames of images with different clear areas, a document photo with uniform clarity can be obtained. In this way, when the document scene is a close-up tilted shooting scene, when the electronic device receives an operation of clicking the photo control, the electronic device can execute the first photo processing flow to obtain a document photo with uniform clarity, thereby reducing the probability of having some areas of the document photo being clear and some areas being blurred when the document is shot at a close distance and tilted.

[0156] Exemplarily, when the electronic device completes the configuration of the first photographing process, the electronic device can maintain the flag as an identifier indicating that the photographing process is the first photographing process, so that when the electronic device determines again that the document scene is a close-up tilted shooting scene, the identifier of the flag can be read. If the identifier indicating that the photographing process is the first photographing process is read, no processing is performed to reduce power consumption. If the identifier indicating that the photographing process is the second photographing process is read, the first photographing process is configured.

[0157] S409: When the electronic device determines that the document scene is a non-tilted shooting scene, the electronic device configures a second photo processing flow.

[0158] Exemplarily, when the electronic device determines that the document scene is a non-tilted shooting scene, the electronic device obtains configuration information of each step of the second photographing processing flow from a preset configuration file and configures each step of the photographing processing flow to obtain the second photographing processing flow.

[0159] The difference between the configuration information of each step of the second photographing process and the configuration information of each step of the first photographing process is that: in the configuration information of each step of the second photographing process, the configuration information corresponding to the image acquisition step includes configuration parameters such as resolution, the number of frames acquired by the camera focusing once, the number of camera focusing times h, and the position of the focus area, where h=1; the configuration information corresponding to the algorithm processing step does not include the depth of field fusion algorithm. The other configuration information of each step of the second photographing process is the same as the other configuration information of each step of the first photographing process.

[0160] The second photo processing flow can be used to instruct the camera to focus once, collect one or more frames of images, and process the collected images to obtain photos. In this way, when the document scene is a non-tilted shooting scene, when the electronic device receives an operation of clicking the photo control, the electronic device can execute the second photo processing flow, and can obtain a document photo with uniform clarity without multiple focusing, thereby reducing the time from when the photo control is clicked to when the electronic device obtains the document photo.

[0161] Exemplarily, when the electronic device completes the configuration of the second photographing process, the electronic device can maintain the flag as an identifier indicating that the photographing process is the second photographing process, so that when the electronic device determines again that the document scene is a non-tilted shooting scene, the identifier of the flag can be read. If the identifier indicating that the photographing process is the first photographing process is read, the second photographing process is configured. If the identifier indicating that the photographing process is the second photographing process is read, no processing is performed to reduce power consumption.

[0162] S410: When the electronic device obtains the preview image from the preview buffer, the electronic device may obtain the shooting distance when the preview image is collected from the laser distance measuring device or the ultrasonic distance measuring device of the electronic device. The electronic device executes step S407 according to the obtained shooting distance.

[0163] Optionally, the electronic device may also perform this step when determining that the document scene is an inclined shooting scene based on the length of the first side and the length of the second side of the preview image of the mth frame, and obtain the shooting distance when capturing the preview image of the mth frame by using a laser ranging device or an ultrasonic ranging device of the electronic device.

[0164] S411: When the electronic device determines that the photographing scene is a long-distance photographing scene and a document scene according to the preview image, the electronic device configures a second photographing processing flow.

[0165] Exemplarily, when the electronic device determines that the photographing scene is a long-distance shooting scene and that the photographing scene is a document scene based on the preview image, the electronic device obtains configuration information of each step of the second photographing processing flow from a preset configuration file and configures each step of the photographing processing flow to obtain the second photographing processing flow.

[0166] It can be understood that when the electronic device determines that the photographing scene is a long-distance photographing scene and determines that the photographing scene is a document scene based on the preview image, it indicates that the document scene is a long-distance photographing scene.

[0167] In this way, when the document scene is a long-distance shooting scene, when the electronic device receives the operation of clicking the photo control, the electronic device can execute the second photo processing flow, and a document photo with uniform clarity can be obtained without multiple focusing, thereby reducing the time from when the photo control is clicked to when the electronic device obtains the document photo.

[0168] Exemplarily, when the electronic device completes the configuration of the second photographing process, the electronic device can maintain the flag as an identifier indicating that the photographing process is the second photographing process, so that when the electronic device determines that the photographing scene is a long-distance shooting scene and that the photographing scene is a document scene based on the preview image, the identifier of the flag can be read. If the identifier indicating that the photographing process is the first photographing process is read, the second photographing process is configured. If the identifier indicating that the photographing process is the second photographing process is read, no processing is performed to reduce power consumption.

[0169] S412: When the electronic device determines that the photographing scene is a non-document scene, the electronic device configures a photographing processing flow corresponding to the non-document scene.

[0170] Taking the non-document scene as a portrait scene as an example, illustratively, when the electronic device determines that the photographing scene is a portrait scene, the electronic device obtains the configuration information of each step of the photographing processing flow corresponding to the portrait scene from a preset configuration file and configures each step of the photographing processing flow to obtain the photographing processing flow corresponding to the portrait scene.

[0171] When the camera application completes the configuration of the photographing process flow corresponding to the portrait scene, the camera application may maintain a flag bit as an identifier indicating that the photographing process flow is a photographing process flow corresponding to the portrait scene.

[0172] In this way, when the electronic device determines that the photographing scene is a portrait scene, the identification of the flag bit can be read. If the identification indicating that the photographing process is a photographing process corresponding to the portrait scene is read, no processing is performed. If the identification indicating that the photographing process is a photographing process corresponding to the portrait scene is not read, the photographing process corresponding to the portrait scene is configured.

[0173] It can be understood that when the electronic device determines that the photographing scene is one of the scenes such as cat and dog scene, sunset scene, blue sky scene, flower scene or snow scene, the specific implementation principle of the electronic device configuring the photographing processing flow corresponding to the non-document scene is similar to the specific implementation principle of the electronic device configuring the photographing processing flow corresponding to the portrait scene when the electronic device determines that the photographing scene is a portrait scene, and will not be repeated here.

[0174] In this embodiment, when the document scene is a close-up oblique shooting scene, the electronic device can execute steps S401-S408 to complete the configuration of the first photo processing flow, or the electronic device can execute S401-S404, S410, S407 and S408 to complete the configuration of the first photo processing flow, so that when an operation acting on the photo control is received, multiple focus acquisition images are performed, and the acquired images are processed to obtain document photos with uniform clarity, thereby reducing the probability of some areas of the document photo being clear and some areas being blurred when the document is photographed at a close distance and obliquely.

[0175] When the document scene is a non-inclined shooting scene, the electronic device can execute steps S401-S404 and S409 to complete the configuration of the second photo processing flow, so that when an operation acting on the photo control is received, it can focus and capture the image once, and obtain a document photo with uniform clarity by processing the captured image. Since multiple focusing is not required, the time from when the photo control is clicked to when the electronic device obtains the document photo is reduced.

[0176] When the document scene is a long-distance shooting scene, the electronic device can execute steps S401, S405-S407 and S411 to complete the configuration of the second photo processing flow, so that when an operation acting on the photo control is received, it can focus and capture the image once, and obtain a document photo with uniform clarity by processing the captured image. Since there is no need for multiple focusing, the time from when the photo control is clicked to when the electronic device obtains the document photo is reduced.

[0177] Alternatively, when the document scene is a long-distance shooting scene, the electronic device can also execute steps S401, S410, S407 and S411 to complete the configuration of the second photo processing flow, so that when an operation acting on the photo control is received, it can focus and capture the image once, and obtain a document photo with uniform clarity by processing the captured image. Since there is no need for multiple focusing, the time from when the photo control is clicked to when the electronic device obtains the document photo is reduced.

[0178] When the photo-taking scene is a non-document scene, the electronic device can execute steps S401 and S412 to complete the configuration of the photo-taking processing flow corresponding to the non-document scene, so that when an operation acting on the photo-taking control is received, the photo-taking processing flow corresponding to the non-document scene is executed to obtain photos with better imaging quality corresponding to the non-document scene.

[0179] It is understandable that in some electronic devices, when the camera focuses once, the camera can capture multiple frames of images or only one frame of image. The embodiment of the present application does not limit the number of frames of images captured each time the camera focuses.

[0180] For example, Figure 6 A schematic diagram of a first photo processing flow provided in an embodiment of the present application is shown.

[0181] In the case where the document scene is a close-up tilted shooting scene, when the electronic device receives an operation acting on the photo control, the electronic device can perform the following operations: Figure 6 The first photo processing flow is shown in FIG. Figure 6 As shown, the first photo processing flow executed by the electronic device includes:

[0182] S601: Image acquisition.

[0183] Exemplarily, in response to an operation of clicking a photo control, the camera focuses multiple times to capture images.

[0184] S602: Front-end processing (IFE).

[0185] Exemplarily, the electronic device obtains an image captured by a camera, and performs color correction, downsampling, de-mosaicing and other processing on each frame of the image using configured parameters to obtain an image processed by the front end.

[0186] S603: Bayer Processing (BPS).

[0187] Exemplarily, the electronic device uses the configured parameters to perform bad pixel removal, phase focusing, demosaicing, downsampling, high dynamic range (HDR) processing, and Bayer hybrid noise reduction processing on the image processed by the front end to obtain a Bayer processed image.

[0188] S604: Algorithm processing.

[0189] Exemplarily, the electronic device performs multi-frame noise reduction algorithm processing, depth of field fusion algorithm processing, text super-resolution (SR) enhancement algorithm processing and effect sharpening debugging algorithm processing on the Bayer-processed images corresponding to each of the multiple frames of images acquired by the camera through multiple focusing operations to obtain a document photo.

[0190] S605: Photo storage.

[0191] Exemplarily, the electronic device stores the document photo in a storage location specified in the album.

[0192] It is understandable that, when the document scene is a non-tilted shooting scene or a long-distance shooting scene, the electronic device can execute the second shooting process when receiving an operation acting on the shooting control. Figure 6 The specific implementation principle of the first photo processing flow shown is similar and will not be repeated here.

[0193] Figure 7A module interaction flow chart provided by an embodiment of the present application is shown. Figure 2 and Figure 7 , the document shooting method provided in the embodiment of the present application is described.

[0194] like Figure 7 As shown, the method includes:

[0195] S701: In response to a user opening a camera application, the camera module processes an image captured by the camera using a preview processing flow to obtain a preview image. The camera module stores the preview image in a preview buffer and transmits information indicating that the preview image has been stored to the scene detection module.

[0196] S702: The scene detection module obtains a preview image from the preview buffer, and determines whether the photographing scene is a document scene according to the preview image.

[0197] If yes, execute steps S703a and S703c.

[0198] If not, the electronic device executes step S703b.

[0199] In this step, the scene detection module obtains a preview image from the preview cache and determines whether the photo scene is a document scene based on the preview image. Figure 4 The specific implementation principle of S401 in the embodiment is similar and will not be repeated here.

[0200] S703a: When it is determined that the photographing scene is a document scene, the scene detection module transmits the preview image to the coordinate detection module.

[0201] S704: The coordinate detection module may identify each vertex of the document in the received preview image, and calculate the coordinates of each vertex of the document in the preview image according to the coordinate system of the preview window. The coordinates of the vertices of the document may be referred to as document coordinate points.

[0202] The specific implementation principle of this step is Figure 4 The specific implementation principle of S402 in the embodiment is similar and will not be repeated here.

[0203] S705. The coordinate detection module transmits the coordinates of each vertex of the document in the preview image to the coordinate processing module.

[0204] S706. The coordinate processing module calculates the length of the first side and the length of the second side of the document in the preview image. The coordinate processing module determines whether the document scene is stable according to the length of the first side and / or the length of the second side corresponding to the continuous m frames of preview images.

[0205] The specific implementation principle of this step is Figure 4The specific implementation principle of S403 in the embodiment is similar and will not be repeated here.

[0206] When the coordinate processing module determines that the document scene is stable, the coordinate processing module executes step S707.

[0207] When the coordinate processing module determines that the document scene is unstable, the coordinate processing module may transmit information indicating that the state of the photographing scene is unstable to the tilt determination module, so that the tilt determination module does not execute step S708.

[0208] S707, the coordinate processing module transmits the length of the first side and the length of the second side corresponding to the m-th frame preview image to the tilt determination module. Alternatively, the coordinate processing module transmits the length of the first side and the length of the second side corresponding to the m-1-th frame preview image, and the length of the first side and the length of the second side corresponding to the m-th frame preview image to the tilt determination module.

[0209] When the tilt determination module receives the length of the first side and the length of the second side, the tilt determination module may execute step S708.

[0210] S708: The tilt determination module determines whether the document scene is a tilted shooting scene according to the received length of the first side and the received length of the second side.

[0211] If yes, execute step S709a.

[0212] If not, execute step S709b.

[0213] The specific implementation principle of the tilt determination module for determining whether the document scene is a tilted shooting scene according to the received length of the first side and the length of the second side can be found in Figure 4 S404 in the embodiment will not be described in detail here.

[0214] S709a: When the tilt determination module determines that the document scene is a tilted shooting scene, the tilt determination module transmits information indicating that the document scene is a tilted shooting scene to the fusion determination module.

[0215] S710: When the camera module obtains a preview image and stores the preview image in a preview buffer, the camera module transmits information indicating that the preview image has been stored to the shooting distance determination module.

[0216] Optionally, S710 may be executed concurrently with S701, so that the electronic device executes S714 as quickly as possible, so as to quickly complete the configuration of the first photographing processing flow.

[0217] S711. When receiving information indicating that the preview image has been stored, the shooting distance determination module obtains the shooting distance corresponding to the preview image.

[0218] The specific implementation principle of this step can be found in Figure 4 The specific implementation principles of S405-S406 in the embodiment are not repeated here.

[0219] The specific implementation principle of this step can also be seen Figure 4 The specific implementation principle of S410 in the embodiment will not be repeated here.

[0220] S712: The shooting distance determination module determines whether the photographing scene is a close-range shooting scene according to the shooting distance corresponding to the preview image.

[0221] If yes, execute step S713a.

[0222] If not, execute step S713b.

[0223] The specific implementation principle of the shooting distance judgment module for judging whether the shooting scene is a close-up shooting scene according to the shooting distance corresponding to the preview image can be found in Figure 4 The specific implementation principle of S407 in the embodiment will not be repeated here.

[0224] S713a: When it is determined that the photographing scene is a close-up photographing scene, the photographing distance determination module transmits information indicating that the photographing scene is a close-up photographing scene to the fusion determination module.

[0225] S714. When the fusion judgment module obtains the information indicating that the document scene is an oblique shooting scene corresponding to the m-th frame preview image, and obtains the information indicating that the photographing scene is a close-up shooting scene corresponding to the m-th frame preview image, the fusion judgment module transmits the information indicating that the document scene is a close-up oblique shooting scene to the camera application.

[0226] S715a: When the camera application receives information indicating that the document scene is a close-up oblique shooting scene, the camera application configures a first photo processing flow.

[0227] The specific implementation principle of this step is Figure 4 The specific implementation principle of S408 in the embodiment is similar and will not be repeated here.

[0228] In the case where the document scene is a close-up tilted shooting scene, when the electronic device receives an operation of clicking the photo control, the electronic device can execute the first photo processing flow to obtain a document photo with uniform clarity. In this way, the probability of a document photo obtained when shooting a document at a close distance tilted having some areas being clear and some areas being blurred can be reduced.

[0229] S703b. When it is determined that the photographing scene is not a document scene, for example, when it is determined that the photographing scene is a non-document scene, the scene detection module transmits information indicating that the photographing scene is a non-document scene to the camera application, and the camera application executes step S715b.

[0230] S715b: When receiving information indicating that the photographing scene is a non-document scene, the camera application configures a photographing process corresponding to the non-document scene. Figure 4 The specific implementation principle of S412 in the embodiment will not be repeated here.

[0231] S709b. When the tilt determination module determines that the document scene is not a tilted shooting scene, the tilt determination module transmits information indicating that the document scene is a non-tilted shooting scene to the camera application, and the camera application executes step S715c.

[0232] S715c: When the camera application receives information indicating that the document scene is a non-tilted shooting scene, the camera application configures a second photo processing flow. Figure 4 The specific implementation principle of S409 in the embodiment.

[0233] S703c. When it is determined that the photographing scene is a document scene, the scene detection module transmits information indicating that the photographing scene is a document scene to the camera application, so that the camera application executes step S715d.

[0234] S713b. When it is determined that the photographing scene is not a close-up photographing scene, the photographing distance determination module transmits information indicating that the photographing scene is a long-distance photographing scene to the camera application, and the camera application executes step S715d.

[0235] S715d: When the camera application receives information corresponding to the preview image indicating that the shooting scene is a long-distance shooting scene and information corresponding to the preview image indicating that the shooting scene is a document scene, the camera application configures a second shooting process. Figure 4 The specific implementation principle of S411 in the embodiment.

[0236] In this embodiment, when the document scene is a close-up oblique shooting scene, the electronic device can configure a first photo processing flow to perform multiple focus capture images when receiving an operation acting on a photo control, and obtain a document photo with uniform clarity by processing the captured images, thereby reducing the probability of having some areas of the document photo being clear and some areas being blurred when the document is photographed at a close distance and obliquely.

[0237] When the document scene is a non-inclined shooting scene or a long-distance shooting scene, the electronic device can configure a second photo processing process to focus and capture an image when an operation acting on the photo control is received, and obtain a document photo with uniform clarity by processing the captured image. Since multiple focusing is not required, the time from when the photo control is clicked to when the electronic device obtains the document photo is reduced.

[0238] When the photo-taking scene is a non-document scene, the electronic device can configure a photo-taking processing flow corresponding to the non-document scene, so as to execute the photo-taking processing flow corresponding to the non-document scene when receiving an operation acting on the photo-taking control, and obtain photos with better imaging quality corresponding to the non-document scene.

[0239] Optionally, in the software architecture of some electronic devices, the scene detection module, the coordinate processing module, the tilt judgment module, the fusion judgment module and the shooting distance judgment module can all be located in the hardware abstraction layer (HAL) layer in the software architecture of the electronic device. The functions of the scene detection module, the coordinate processing module, the tilt judgment module, the fusion judgment module and the shooting distance judgment module are the same as those shown in the above embodiment, and will not be repeated here.

[0240] The document photographing method proposed in this embodiment is described below in conjunction with some embodiments.

[0241] The present invention provides a method for photographing a document, the method comprising:

[0242] In the case where the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, a first image is obtained in response to an operation on a photographing control displayed on the electronic device. The first image is obtained by fusing multiple frames of first source images captured by the electronic device, and there are images with different focus areas in the multiple frames of the first source images. In the case where the angle between the electronic device and the photographed object is less than or equal to the first threshold, or the distance between the camera and the photographed object is greater than or equal to the second threshold, a second image is obtained in response to an operation on a photographing control. The second image is obtained by fusing multiple frames of second source images captured by the electronic device, and the focus areas of the multiple frames of second source images are the same, or the second image is obtained by the electronic device processing a captured frame of the second source image.

[0243] The photographed object including text may be the document in the above embodiment.

[0244] The angle between the electronic device and the photographed object being greater than the first threshold can be understood as an angle between the electronic device and the photographed object being greater than the first threshold. The angle between the electronic device and the photographed object being greater than the first threshold can indicate that the photographing scene is a document scene, and the document scene is an oblique photographing scene.

[0245] The angle between the electronic device and the photographed object being less than or equal to the first threshold can be understood as the angle between the electronic device and the photographed object being less than or equal to the first threshold. The angle between the electronic device and the photographed object being less than or equal to the first threshold can indicate that the photographing scene is a document scene, and the document scene is a non-tilted photographing scene.

[0246] The distance between the camera of the electronic device and the photographed object being less than the second threshold may indicate that the photographing scene is a document scene and the document scene is a close-range photographing scene.

[0247] If the distance between the camera and the photographed object is greater than or equal to the second threshold, it can be indicated that the photographing scene is a document scene, and the document scene is a long-distance photographing scene.

[0248] If the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, it can be indicated that the document scene is a close-range oblique shooting scene.

[0249] The first source image may be an image used to obtain a document photo when the document scene is a close-up oblique shooting scene. The second source image may be an image used to obtain a document photo when the document scene is an oblique shooting scene or a close-up shooting scene.

[0250] The first image can be Figure 4 In the embodiment, the document photo is obtained when the document scene is a close-up tilted shooting scene. The second image can be Figure 4 In the embodiment, the document photo is obtained when the document scene is a non-inclined shooting scene or the document scene is a long-distance shooting scene.

[0251] In this embodiment, in response to the operation of the photo control on the display of the electronic device, the specific implementation principle of obtaining the first image is shown in Figure 6 In the embodiment, the electronic device executes the first photographing process to obtain the specific implementation principle of the document photo. In response to the operation acting on the photographing control, the specific implementation principle of obtaining the second image is shown in Figure 6 In the embodiment, the electronic device executes the second photographing process to obtain the specific implementation principle of the document photo.

[0252] In this embodiment, the specific implementation principle of determining that the photographing scene satisfies that the angle between the electronic device and the photographing object including text is greater than the first threshold, and the distance between the camera of the electronic device and the photographing object is less than the second threshold can be seen in Figure 4 The specific implementation principle of the embodiment for determining that the document scene is a close-range oblique shooting scene.

[0253] In this embodiment, the specific implementation principle of determining that the photographing scene satisfies that the angle between the electronic device and the photographed object is less than or equal to the first threshold, or the distance between the camera and the photographed object is greater than or equal to the second threshold can be found in Figure 4 The specific implementation principle of the embodiment for determining that the document scene is a close-range oblique shooting scene.

[0254] Optionally, the electronic device may also obtain the tilt angle of the electronic device from a gyroscope sensor of the electronic device, and may obtain the distance between the camera and the photographed object from a distance sensor. When the tilt angle of the electronic device obtained is greater than a first threshold, and the distance between the camera and the photographed object obtained from the distance sensor is less than a second threshold, a first image is obtained in response to an operation on a photographing control displayed by the electronic device. When the tilt angle of the electronic device obtained is less than or equal to the first threshold, or the distance between the camera and the photographed object obtained from the distance sensor is greater than or equal to a second threshold, a second image is obtained in response to an operation on the photographing control.

[0255] This embodiment can achieve that when the document scene is a close-up tilted shooting scene, in response to the operation on the photo control, the electronic device can obtain a first image with uniform clarity by fusing the source images obtained by multiple focusing, reducing the probability of the first image obtained due to close-up tilted shooting of the document having some areas clear and some areas blurred, thereby improving the user experience. This embodiment can also achieve that when the document scene is a non-tilted shooting scene or the document scene is a long-distance shooting scene, a second image with uniform clarity can be obtained by performing image processing on the source image obtained by one focusing. Compared with multiple focusing, one focusing takes less time, which can reduce the time from the operation of the photo control to the acquisition of the second image, thereby improving the user experience.

[0256] Optionally, in response to an operation of a photo control on a display of an electronic device, before obtaining the first image, the method further includes: when it is determined that the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, configuring a first photo processing flow. The first photo processing flow is used to instruct the camera of the electronic device to focus multiple times, and to capture one or more frames of images each time it focuses, and is also used to fuse the captured images to obtain a photo. The focus areas of two adjacent focuses are different.

[0257] For example, see Figure 4 In an embodiment, when the document scene is a close-range oblique shooting scene, the electronic device can execute steps S401-S408 to complete the configuration of the first photographing processing flow, or the electronic device can execute S401-S404, S410, S407-S408 to complete the configuration of the first photographing processing flow.

[0258] In this way, when the document scene is a close-up oblique shooting scene, multiple focusing and image acquisition can be performed when an operation acting on the photo control is received, and a document photo with uniform clarity can be obtained by processing the acquired images, thereby reducing the probability of some areas of the document photo being clear and some areas being blurred when the document is photographed at a close distance and obliquely.

[0259] Optionally, determining whether the angle between the electronic device and the photographed object is greater than a first threshold value includes: determining whether the absolute value of the difference between the length of the first side of the photographed object and the length of the second side of the photographed object in the preview image is greater than a third threshold value, and / or whether the ratio of the length of the first side to the length of the second side in the preview image is less than a fourth threshold value. Wherein, the first side and the second side are at different distances from the camera of the electronic device, the length of the first side is less than the length of the second side, and the first side and the second side have no intersection in the photographed object. When the absolute value of the difference between the length of the first side of the photographed object and the length of the second side of the photographed object is greater than the third threshold value, and / or the ratio of the length of the first side to the length of the second side is less than the fourth threshold value, it indicates that the angle between the electronic device and the photographed object is greater than the first threshold value.

[0260] For example, the subject is Figure 5 Taking the document 102 in the embodiment as an example, the first side may be AB in the embodiment, and the second side may be CD in the embodiment. The first side and the second side may also be two sides parallel to AB in the document 102, and the first side has intersections with the AC side and the BD side, respectively, and the second side has intersections with the AC side and the BD side, respectively.

[0261] Since the ratio of the length of the first side to the length of the second side is smaller, it can be indicated that the electronic device is tilted more when shooting the document, so that the angle between the electronic device and the document is larger, which may make the angle between the electronic device and the document greater than the first threshold. The first threshold may be greater than or equal to 0°. The greater the absolute value of the difference between the length of the first side and the length of the second side, it can be indicated that the electronic device is tilted more when shooting the document, so that the angle between the electronic device and the document is greater than the first threshold.

[0262] Exemplarily, taking the first threshold as 0°, the third threshold as 0, and the fourth threshold as 1, if the ratio of the length of the first side to the length of the second side is less than 1, it means that the electronic device is tilted when photographing the document, and the angle between the electronic device and the document is greater than 0°. If the absolute value of the difference between the length of the first side and the length of the second side is greater than 0, it means that the electronic device is tilted when photographing the document, and the angle between the electronic device and the document is greater than 0°.

[0263] The ratio of the length of the first side to the length of the second side or the absolute value of the difference between the length of the first side and the length of the second side represents the difference between the length of the first side and the length of the second side. In this way, based on the difference between the length of the first side and the length of the second side, it is possible to determine whether the angle between the electronic device and the document is greater than the first threshold, and further determine whether the document scene is an oblique shooting scene.

[0264] Optionally, the photographed object includes a first vertex, a second vertex, a third vertex and a fourth vertex. The two endpoints of the first side are the first vertex and the second vertex, respectively, and the two endpoints of the second side are the third vertex and the fourth vertex, respectively. In the coordinate system of the preview window of the electronic device displaying the preview image, the absolute value of the difference between the ordinate of the first vertex and the ordinate of the second vertex is less than a fifth threshold, and the absolute value of the difference between the ordinate of the third vertex and the ordinate of the fourth vertex is less than a fifth threshold. Determining whether the ratio of the length of the first side to the length of the second side in the preview image is less than a fourth threshold includes: determining whether the ratio of the minimum length to the maximum length of the first side and the second side is less than the fourth threshold.

[0265] For example, the subject is still Figure 5 Taking the document 102 in the embodiment as an example, the first vertex may be A in embodiment 5. The second vertex may be B in embodiment 5. The third vertex may be C in embodiment 5. The fourth vertex may be D in embodiment 5. The ratio of the minimum side length to the maximum side length of the first side and the second side may be R4 in embodiment 4.

[0266] In this way, based on the comparison of the ratio of the minimum side length to the maximum side length of the first side and the second side with the fourth threshold, it is possible to determine whether the angle between the electronic device and the document is greater than the first threshold, and then determine whether the document scene is an inclined shooting scene.

[0267] Optionally, before determining whether the angle between the electronic device and the photographed object is greater than the first threshold, the method further includes: determining whether a first probability of the continuous m-frame preview images is greater than a sixth threshold. The first probability is the sum of probabilities that the absolute value of the difference between the lengths of the first sides corresponding to two adjacent frames of preview images in the continuous m-frame preview images is less than the seventh threshold. Determining whether the angle between the electronic device and the photographed object is greater than the first threshold includes: determining whether the angle between the electronic device and the photographed object is greater than the first threshold when determining that the first probability is greater than the sixth threshold.

[0268] In this embodiment, the first probability is, for example, Figure 4 The first probability R1 in the embodiment. In this way, based on the absolute value of the difference between the side lengths of two adjacent preview images in the continuous m frames of preview images, it is possible to accurately determine whether the document scene is stable, and then based on whether the document scene is stable, it is possible to accurately determine whether the user has the intention to shoot the document. When it is determined that the document scene is stable or the user has the intention to shoot the document, it is determined whether the document scene is an inclined shooting scene, which can reduce the power consumption generated by determining whether the document scene is an inclined shooting scene when the user does not have the intention to shoot the document.

[0269] Optionally, determining whether the distance between the camera of the electronic device and the photographed object is less than a second threshold value includes: determining whether the shooting distance corresponding to the focus parameter associated with the preview image is less than the second threshold value. Wherein, if the shooting distance corresponding to the focus parameter associated with the preview image is less than the second threshold value, it means that the distance between the camera of the electronic device and the photographed object is less than the second threshold value.

[0270] In this embodiment, the focus parameter associated with the preview image may be a motor diagonal code. In this way, it is possible to determine whether the photographing scene is a close-up shooting scene, so as to determine whether the document scene is a close-up tilted scene.

[0271] Optionally, determining that the distance between the camera of the electronic device and the photographed object is less than a second threshold value includes: determining whether the distance between the camera and the photographed object when the preview image is captured obtained from a laser ranging device or an ultrasonic ranging device of the electronic device is less than the second threshold value. Wherein, when the distance between the camera and the photographed object is less than the second threshold value when the preview image is captured, it indicates that the distance between the camera of the electronic device and the photographed object is less than the second threshold value.

[0272] In this way, it is possible to determine whether the photographing scene is a close-up shooting scene, so as to determine whether the document scene is a close-up tilted scene.

[0273] Optionally, in response to an operation on a photo control, before obtaining a second image, the method further includes: configuring a second photo processing flow when it is determined that the angle between the electronic device and the photographed object is less than or equal to a first threshold, or when it is determined that the distance between the camera and the photographed object is greater than or equal to a second threshold. The second photo processing flow is used to instruct the camera to focus once, capture one or more frames of images, and process the captured images to obtain photos.

[0274] For example, when the document scene is a non-tilted shooting scene, the electronic device may execute Figure 4 Steps S401-S404 and S409 in the embodiment complete the configuration of the second photo processing flow. When the document scene is a long-distance shooting scene, the electronic device can execute Figure 4 In the embodiment, steps S401-S401, S405-S407 and S411 complete the configuration of the second photo processing flow. Alternatively, in the case where the document scene is a long-distance shooting scene, the electronic device can also perform Figure 4 Steps S401-S401, S410, S407 and S411 in the embodiment complete the configuration of the second photo processing flow.

[0275] In this way, when the document scene is a non-inclined shooting scene or a long-distance shooting scene, when an operation acting on the photo control is received, a focus is performed to capture the image, and a document photo with uniform clarity is obtained by processing the captured image. Since multiple focusing is not required, the time from when the photo control is clicked to when the electronic device obtains the document photo is reduced.

[0276] The document shooting method of the embodiment of the present application has been described above, and the device for executing the above document shooting method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above document shooting method.

[0277] The document shooting method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal devices can refer to the above related descriptions, which will not be repeated here.

[0278] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0279] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0280] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0281] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0282] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0283] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0284] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A document shooting method, characterized in that: include: The electronic device performs scene detection according to the camera preview stream and determines that the photographing scene is a document scene; When the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, in response to an operation on a photographing control displayed by the electronic device, obtaining a first image; The first image is obtained by fusing multiple frames of first source images acquired by the electronic device, and the multiple frames of first source images include images with different focus areas; When the angle between the electronic device and the photographed object is less than or equal to the first threshold, or the distance between the camera and the photographed object is greater than or equal to the second threshold, a second image is obtained in response to an operation acting on the photo-taking control; the second image is obtained by fusing multiple frames of second source images captured by the electronic device, and the focus areas of the multiple frames of second source images are the same, or the second image is obtained by processing a captured frame of the second source image by the electronic device.

2. The method according to claim 1, characterized in that Before obtaining the first image in response to the operation of the photo-taking control displayed on the electronic device, the method further includes: When it is determined that the angle between the electronic device and the photographed object including text is greater than a first threshold, and the distance between the camera of the electronic device and the photographed object is less than a second threshold, a first photographing processing flow is configured; the first photographing processing flow is used to instruct the camera of the electronic device to focus multiple times, and to capture one or more frames of images each time it focuses, and is also used to fuse the captured images to obtain a photo; the focus areas of two adjacent focuses are different.

3. The method according to claim 2, characterized in that The determining whether the angle between the electronic device and the photographed object is greater than the first threshold comprises: Determine whether an absolute value of a difference between a length of a first side of the photographed object and a length of a second side of the photographed object in the preview image is greater than a third threshold, and / or whether a ratio of a length of the first side to a length of the second side in the preview image is less than a fourth threshold; Among them, the first side and the second side are respectively at different distances from the camera of the electronic device, the length of the first side is smaller than the length of the second side, and the first side and the second side have no intersection in the photographed object; when the absolute value of the difference between the length of the first side of the photographed object and the length of the second side of the photographed object is greater than a third threshold, and / or the ratio of the length of the first side to the length of the second side is less than a fourth threshold, it indicates that the angle between the electronic device and the photographed object is greater than the first threshold.

4. The method according to claim 3, characterized in that The photographed object includes a first vertex, a second vertex, a third vertex and a fourth vertex; two endpoints of the first side are the first vertex and the second vertex respectively, and two endpoints of the second side are the third vertex and the fourth vertex respectively; In a coordinate system of a preview window in which the electronic device displays the preview image, an absolute value of a difference between a ordinate of the first vertex and a ordinate of the second vertex is less than a fifth threshold, and an absolute value of a difference between a ordinate of the third vertex and a ordinate of the fourth vertex is less than the fifth threshold; Determining whether a ratio of the length of the first side to the length of the second side in the preview image is less than a fourth threshold includes: It is determined whether the ratio of the minimum side length to the maximum side length of the first side and the second side is less than the fourth threshold.

5. The method according to claim 4, characterized in that Before determining whether the angle between the electronic device and the photographed object is greater than the first threshold, the method further includes: Determine whether a first probability of the continuous m frames of preview images is greater than a sixth threshold; the first probability is the sum of probabilities that the absolute value of the difference between the lengths of the first sides corresponding to two adjacent frames of preview images in the continuous m frames of preview images is less than a seventh threshold; The determining whether the angle between the electronic device and the photographed object is greater than the first threshold comprises: When it is determined that the first probability is greater than the sixth threshold, it is determined whether the angle between the electronic device and the photographed object is greater than the first threshold.

6. The method according to any one of claims 2 to 5, characterized in that: The determining that the distance between the camera of the electronic device and the photographed object is less than a second threshold value includes: Determine whether the shooting distance corresponding to the focus parameter associated with the preview image is less than the second threshold; wherein, if the shooting distance corresponding to the focus parameter associated with the preview image is less than the second threshold, it means that the distance between the camera of the electronic device and the photographed object is less than the second threshold.

7. The method according to any one of claims 2 to 5, characterized in that: The determining that the distance between the camera of the electronic device and the photographed object is less than a second threshold value includes: Determine whether the distance between the camera and the photographed object when capturing the preview image obtained from the laser ranging device or the ultrasonic ranging device of the electronic device is less than the second threshold; wherein, when the distance between the camera and the photographed object is less than the second threshold when capturing the preview image, it indicates that the distance between the camera of the electronic device and the photographed object is less than the second threshold.

8. The method according to any one of claims 1 to 5, characterized in that Before obtaining the second image in response to the operation on the photographing control, the method further includes: When it is determined that the angle between the electronic device and the photographed object is less than or equal to the first threshold, or when it is determined that the distance between the camera and the photographed object is greater than or equal to the second threshold, a second photographing processing flow is configured; the second photographing processing flow is used to instruct the camera to focus once, capture one or more frames of images, and process the captured images to obtain photos.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images