Image processing method and device and electronic equipment

Abstract

The invention discloses an image processing method and device and electronic equipment, and belongs to the technical field of image processing. The image processing method comprises the following steps: receiving a first input under the condition that a preview image is displayed on a shooting preview interface and a frame is marked on a first object in the preview image; in response to the first input, acquiring multiple frames of shot images, and performing image fusion processing on the multiple frames of shot images to obtain a fused image; in preview images displayed in the shooting preview interface, a first preview image associated with the first shot image and having the same shooting timestamp is searched, and the first shot image is a reference frame in the multiple frames of shot images subjected to fusion processing; and cutting the corrected fusion image according to the first coordinate information of the frame of the identifier in the first preview image to obtain a corrected image of the first object.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device, electronic device, and readable storage medium. Background Art

[0002] As the shooting functions of electronic devices continue to improve, electronic devices can crop and correct the image of the target object to restore the true state of the target object. In the related art, the electronic device displays a shooting preview interface, identifies the area where the target object is located in the shooting preview interface, and when the user triggers the shooting function, shoots the target object displayed in the shooting preview interface, and then crops and corrects the captured image according to the pre-identified area. Since there is a difference in the field of view of the image displayed in the shooting preview interface and the captured image obtained by shooting, the captured image is cropped and corrected by identifying the area in the shooting preview interface, resulting in a low accuracy rate of cropping and correcting the target object, and the cropped and corrected image cannot display the complete target object, and displays other objects except the target object. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image processing method, device and electronic device, which can solve the problem of low accuracy in cropping and correcting objects displayed in captured images.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] In a case where a preview image is displayed on the shooting preview interface and a frame is marked for a first object in the preview image, receiving a first input;

[0006] In response to the first input, a plurality of frames of captured images are acquired, and the plurality of frames of captured images are subjected to image fusion processing to obtain a fused image;

[0007] Searching for a first preview image associated with the same shooting timestamp as the first shot image in the preview images displayed on the shooting preview interface, where the first shot image is a reference frame in the multiple shot images to be fused;

[0008] The fused image is cropped and rectified according to the first coordinate information of the frame identified in the first preview image to obtain a rectified image of the first object.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0010] A first receiving module, configured to receive a first input when a preview image is displayed on a shooting preview interface and a frame is marked for a first object in the preview image;

[0011] An image fusion module, configured to obtain a plurality of frames of captured images in response to a first input, and perform image fusion processing on the plurality of frames of captured images to obtain a fused image;

[0012] A search module, used to search for a first preview image with the same shooting timestamp as the first shot image in the preview images displayed on the shooting preview interface, where the first shot image is a reference frame in the multiple shot images to be fused;

[0013] The cropping and correction module is used to crop and correct the fused image according to the first coordinate information of the frame marked in the first preview image to obtain a corrected image of the first object.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method of the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method of the first aspect.

[0018] In an embodiment of the present application, a preview image is displayed on a shooting preview interface and a frame is marked for a first object in the preview image, which is beneficial for a user to expect an image to be captured through the preview image; a plurality of frames of captured images are acquired by setting, and image fusion processing is performed on the plurality of frames of captured images to obtain a fused image, so as to improve the image quality of the fused image; a first preview image associated with the same shooting timestamp as the first captured image is searched, and the fused image is cropped and corrected according to the first coordinate information of the frame marked in the first preview image to obtain a corrected image of the first object, thereby achieving consistent recognition of the first object in the fused image and the first object in the preview image, improving the accuracy of cropping and correcting the first object displayed in the captured image, eliminating the step of identifying the first object in the captured image after the image is captured, and improving the efficiency of generating the corrected image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of an image processing method provided in an embodiment of the present application;

[0020] Figure 2 This is one of the partial flow charts of the image processing method provided in one embodiment of the present application;

[0021] Figure 3 This is one of the schematic diagrams of changes in the display interface provided in one embodiment of the present application;

[0022] Figure 4 This is one of the partial flow charts of the image processing method provided in one embodiment of the present application;

[0023] Figure 5 This is the second schematic diagram of the change of the display interface provided in an embodiment of the present application;

[0024] Figure 6 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0029] The image processing method, device, electronic device, readable storage medium and computer program product provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0030] See also Figure 1, an embodiment of the present application provides an image processing method, the method comprising:

[0031] S110, receiving a first input when a preview image is displayed on a shooting preview interface and a frame is marked on a first object in the preview image;

[0032] The shooting preview interface is an interface used to view the shooting scene or picture in real time during the photography or video recording process, so that users can instantly view the shooting effect, shooting parameters, and composition. When the electronic device is in shooting preview mode, the camera module obtains the preview image at a preset frame rate and displays the preview image on the shooting preview interface for users to browse.

[0033] When one or more objects are displayed in the preview image, the pre-stored object detection algorithm can identify the first object in the preview image, and generate a border surrounding the first object based on the position of the identified first object in the preview image. The border can be a regular polygon or an irregular line frame formed along the edge of the first object. When the user controls the camera module to move, the displayed preview image changes, the position of the first object in the preview image changes, and the corresponding identified border changes as well.

[0034] The first object can be a piece of paper or text on a screen; it can also be static or dynamic objects such as clothes, utensils, household appliances, plants, animals, etc. Exemplarily, when the first object is text, the pre-stored document detection algorithm recognizes the text in the preview image, marks a quadrilateral frame for the text, and the text surrounded by the frame can be converted into editable electronic text through the text optical character recognition (Optical Character Recognition) technology. Exemplarily, when the first object is clothing, the pre-stored clothing detection algorithm recognizes the clothing in the preview image, marks an irregular frame for the clothing, and the clothing surrounded by the frame can be retrieved through image comparison technology to obtain clothing for sale similar to the clothing. Exemplarily, when the first object is an animal, the pre-stored animal detection algorithm recognizes the animal in the preview image, marks an irregular frame for the animal, and the animal surrounded by the frame can be retrieved through image comparison technology to obtain an animal encyclopedia introduction similar to the animal.

[0035] When the electronic device displays a preview image and marks a frame for a first object in the preview image, a first input is received, and the first input may be a single click, double click, long press, etc. input made to a control, image, frame, etc. displayed on the shooting preview interface, or a specific sliding gesture made, a voice command issued, etc. The first input is used to instruct the camera module to shoot an image.

[0036] S120, in response to the first input, acquiring multiple frames of captured images, and performing image fusion processing on the multiple frames of captured images to obtain a fused image;

[0037] In response to the first input, the camera module is activated to shoot and generate multiple captured images. The preview image is an image generated by the camera module using a first field of view for real-time display and preview by the user, and the captured image is an image that is captured and saved by the camera module using a second field of view after the user observes and browses the image. The first field of view angle of the preview image and the second field of view angle of the captured image may be different, so that the size and displayed content of the preview image and the captured image generated at the same time are different.

[0038] Image fusion processing can fuse multiple frames of captured images into a single image, so that the synthesized single image scene has a more comprehensive and clear description. By performing image fusion processing on multiple frames of captured images, a fused image is obtained to reduce or avoid blurring of the image used for subsequent cropping correction due to device shaking or movement of the first object when capturing the image.

[0039] See also Figure 2 In S120, the camera module can be mobilized to shoot and generate n frames of preview images 21, and m frames of captured images 22 can be captured, and then the m frames of captured images 22 can be subjected to image fusion processing to obtain a fused image 23, in which the first object is displayed. Based on the clarity of the image, the captured images with a clarity greater than a preset value can be screened from the m frames of captured images 22 as the first captured images; based on the position of the first object in the captured image 22, the first captured images whose position in the captured image 22 is within a preset range can be screened from the m frames of captured images 22. The m frames of captured images 22 can be images of continuous frames or non-continuous frames. Optionally, m is 4 or 8, and 4 or 8 frames of captured images are obtained for image fusion processing to obtain a fused image, so as to avoid excessive calculation of image fusion processing.

[0040] S130, searching for a first preview image associated with the same shooting timestamp as the first shot image in the preview images displayed on the shooting preview interface, where the first shot image is a reference frame in the multiple shot images to be fused;

[0041] When displaying a preview image in the shooting preview interface, several frames of preview images are cached, and the shooting timestamp of the preview image is the time when the camera module generates the preview image. The shooting timestamp of the shot image is the time when the camera module generates the shot image. The first shot image and the first preview image are associated with the same shooting timestamp, indicating that both are generated by the camera module at the same time.

[0042] S140: Crop the rectified fused image according to first coordinate information of the frame identified in the first preview image to obtain a rectified image of the first object.

[0043] The first coordinate information may be a coordinate with the first size information of the first preview image as a reference system. The first coordinate information may represent the position of the frame in the first preview image. In S110, the frame may be displayed in the shooting preview interface based on the first coordinate information. Cropping and correcting the fused image using the first coordinate information includes but is not limited to cropping the fused image and correcting the perspective relationship of the first object. Cropping the fused image may delete other objects except the first object in part of the fused image. Correcting the perspective relationship of the first object may restore the display of the first object to a display from a frontal perspective.

[0044] See also Figure 3 , among which, Figure 3 As shown in (a), a text 31 is displayed in the first preview image, the text 31 is recognized, and a frame 32 is marked on the text 31. Figure 3 As shown in (b), the fused image is cropped and corrected according to the first coordinate information of the border 32 to obtain a corrected image 33 showing the text 31.

[0045] In the image processing method provided in the present application, a preview image is displayed in a shooting preview interface and a frame is marked for a first object in the preview image, which is beneficial for a user to expect an image to be captured through the preview image; a plurality of frames of captured images are acquired by setting, and the plurality of frames of captured images are subjected to image fusion processing to obtain a fused image, so as to improve the image quality of the fused image; a first preview image associated with the same shooting timestamp as the first captured image is searched, and the fused image is cropped and corrected according to the first coordinate information of the frame marked in the first preview image to obtain a corrected image of the first object, thereby achieving consistent recognition of the first object in the fused image and the first object in the preview image, improving the accuracy of cropping and correcting the first object displayed in the captured image, eliminating the step of identifying the first object in the captured image after the image is captured, and improving the efficiency of generating the corrected image.

[0046] In some embodiments, after S130, the following steps are included:

[0047] S210, cropping and correcting the first preview image according to the first coordinate information to obtain a playback image showing the first object;

[0048] S220: Display a thumbnail of the echoed image.

[0049] The echo image is an image used for the user to view and understand the approximate state of the corrected image before obtaining the corrected image of the first object. The echo image cuts and corrects the first preview image according to the first coordinate information. The corrected image requires first performing image fusion processing on multiple frames of captured images to obtain a fused image, and then cutting and correcting the fused image according to the first coordinate information. Therefore, compared with generating a corrected image, generating an echo image involves less calculation and takes less time to generate an echo image. Through S210, an echo image can be generated for the user to view. Please refer to Figure 4 Among the multiple preview images 21, the first preview image 21a is selected for cropping correction to obtain an echo image 34 showing the first object.

[0050] In S220, a thumbnail of the echoed image may be displayed in a preset area of ​​the shooting preview interface or the shooting album interface for the user to browse.

[0051] The step of performing image fusion processing on the multiple frames of captured images to obtain a fused image in S120 and the step of searching for a first preview image associated with the same capture timestamp as the first captured image in S130 can be performed simultaneously to improve the efficiency of generating the corrected image.

[0052] In some embodiments, after S220, the method further includes:

[0053] receiving a second input;

[0054] In response to the second input, the echo image is displayed.

[0055] The second input may be an input to the thumbnail of the echo image. For example, a click, a long press, a two-finger move, etc., on the thumbnail of the echo image. The echo image may be displayed in a size adapted to the display screen, or the echo image may be displayed according to the pixel size of the original echo image. Compared with the thumbnail of the echo image, the displayed echo image has a larger size and clarity.

[0056] For example: See Figure 5 ,like Figure 5 As shown in (a), a preview image 41 and a thumbnail 43 of the echo image are displayed in the shooting preview interface. The preview image 41 is marked with a frame 42, and a click input on the thumbnail 43 of the echo image is received, as shown in FIG. Figure 5 As shown in (b), an echo image 44 is displayed.

[0057] In some embodiments, after S130, the method further includes:

[0058] S410, cropping and correcting the first preview image according to the first coordinate information to obtain a playback image showing the first object;

[0059] S420, displaying the echo image;

[0060] After S140, it also includes:

[0061] When the echo image is displayed, the correction image is switched to be displayed.

[0062] In some embodiments, after the echo image is generated, the echo image may be directly displayed. In other embodiments, after the echo image is generated, a thumbnail of the echo image may be directly displayed, and the echo image may be displayed according to the received second input.

[0063] After S140 , it means that the corrected image has been generated. When the echo image is displayed, the echo image is closed and the corrected image is displayed, so that the user can browse a clearer corrected image with more accurate cropping and correction.

[0064] Optionally, after S140, the step further includes: deleting the echoed image.

[0065] In some embodiments, before S110, the steps include:

[0066] S510, acquiring a preview image, identifying a first object displayed in the preview image, and obtaining first coordinate information of a frame of the first object;

[0067] S520: Display the preview image on the shooting preview interface at a preset frame rate, and mark a frame in the preview image according to the first coordinate information.

[0068] See also Figure 2 In 510, the camera module obtains a preview image 21 at a preset frame rate. Each time a preview image 21 is obtained, a first object in the preview image is identified by a pre-stored object detection algorithm, and a frame surrounding the first object is generated, that is, first coordinate information of the frame surrounding the first object is generated.

[0069] In 520, the preview image is displayed on the shooting preview interface at a preset frame rate, so that the displayed preview image is adapted to the displacement of the camera module operated by the user. While displaying the preview image, a frame is drawn in the preview image according to the first coordinate information, so that the user can browse the preview image marked with the frame to determine that the shooting angle of the camera module can capture an image showing the first object.

[0070] In some embodiments, after S510, the following steps are included:

[0071] S610, at least one second preview image, a shooting timestamp of the second preview image, and first coordinate information of a frame identified in the second preview image are stored in association, and the preview images displayed on the shooting preview interface include the second preview image;

[0072] The at least one second preview image includes the first preview image; S130 includes:

[0073] S620: Search, according to the shooting timestamp of the first shot image, at least one stored second preview image for a first preview image having the same shooting timestamp.

[0074] The camera module generates multiple preview images. After performing the first object recognition processing on the preview images, the multiple preview images are displayed in sequence in the shooting preview interface. The second preview image is one of the multiple preview images displayed in the shooting preview interface, that is, the second preview image is one of the multiple preview images generated by the shooting module.

[0075] Based on a preset screening rule, j second preview images can be screened out from i preview images for cache processing. The preset screening rule can be setting preview images separated by a preset number of frames as second preview images; obtaining the clarity of the first object displayed in the preview image, and setting the preview image with a clarity greater than a preset clarity value as the second preview image.

[0076] The shooting timestamp of the second preview image is the time when the second preview image is generated. The first coordinate information of the frame marked in the second preview image is the first coordinate information of the frame of the first object displayed in the second preview image. The second preview image, the shooting timestamp, and the first coordinate information are stored in association to facilitate querying the first coordinate information consistent with the position of the first shot image through the shooting timestamp.

[0077] Multiple frames of preview images and multiple frames of captured images are generated by the camera module in sequence along the time sequence. Therefore, the shooting timestamps of each preview image are different, and the shooting timestamps of each captured image are different. The preview images or each captured image can be distinguished by the shooting timestamp. At the same time, the relative position of the camera module and the first object remains fixed, that is, when the preview image and the captured image with the same shooting timestamp are generated, the relative position of the camera module and the first object is consistent. Therefore, the first preview image with the same shooting timestamp is searched in at least one stored second preview image, and the first coordinate information generated based on the first preview image can be applied to the first captured image, thereby eliminating the need to identify the first object in the first captured image and generate the calculation of the border of the first object.

[0078] In some embodiments, S120 includes:

[0079] S710, acquiring multiple frames of captured images, and selecting a first captured image from the multiple frames of captured images;

[0080] S720: Taking the first captured image as a reference frame, performing image fusion processing on multiple frames of captured images to obtain a fused image.

[0081] The multiple frames of captured images all have preview images with the same shooting timestamp, but the preview image with the same shooting timestamp as the captured image is not necessarily set as the second preview image for caching. Therefore, the captured image corresponding to the second preview image for caching can be selected from the multiple frames of captured images as the first captured image. The one with the highest definition of the first object can also be selected from the multiple frames of captured images as the first captured image.

[0082] Acquire feature information of other images except the first captured image in the multiple frames of captured images, and adjust the first captured image according to the feature information so that the obtained fused image is clear.

[0083] In some embodiments, S140 includes:

[0084] S810, converting the first coordinate information into second coordinate information that conforms to the size information of the first captured image according to the size information of the first preview image and the size information of the first captured image;

[0085] S820: Crop the corrected fused image according to the second coordinate information to obtain a corrected image of the first object.

[0086] The size information of the multiple preview images may be the same. Optionally, the size information of the preview images matches the size information of the display screen of the electronic device. The size information of the multiple captured images may be the same, and the fused image obtained based on the first captured image is consistent with the size information of each captured image. Optionally, the size information of the captured image may be 1080*1080 pixels, 4032*3024 pixels, 3264*2448 pixels, etc. The size information of the captured image may be larger than the size information of the preview image so that the captured image can display more details of the object.

[0087] Since the size information of the preview image is different from the size information of the first captured image, the first coordinate information of the frame of the first object in the first preview image is directly used to crop the fused image, which will cause the cropped corrected image to be too small. Therefore, the first coordinate information of the frame is converted to the same reference system as the size information of the first captured image to obtain the second coordinate information, and then the fused image is cropped and corrected according to the second coordinate information, so that the corrected image can restore the shape of the first object with high accuracy.

[0088] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0089] See also Figure 6 , Figure 6is a structural diagram of an image processing device provided by another embodiment of the present application. Figure 6 As shown, the image processing device may include:

[0090] A first receiving module 601 is used to receive a first input when a preview image is displayed on a shooting preview interface and a frame is marked for a first object in the preview image;

[0091] An image fusion module 602 is used to obtain multiple frames of captured images in response to the first input, and perform image fusion processing on the multiple frames of captured images to obtain a fused image;

[0092] A search module 603 is used to search for a first preview image with the same shooting timestamp as the first shot image in the preview images displayed on the shooting preview interface, where the first shot image is a reference frame in the multiple shot images to be fused;

[0093] The cropping and correction module 604 is configured to crop and correct the fused image according to the first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

[0094] In an optional example, the image processing apparatus further includes:

[0095] A first echo image generating module, used for cropping and correcting the first preview image according to the first coordinate information to obtain an echo image showing the first object;

[0096] The first display module is used to display a thumbnail of the echoed image.

[0097] In an optional example, the image processing apparatus further includes:

[0098] A second receiving module, used for receiving a second input;

[0099] The second display module is used to display an echo image in response to a second input.

[0100] In an optional example, the image processing apparatus further includes:

[0101] A second echo image generating module, used for cropping and correcting the first preview image according to the first coordinate information to obtain an echo image showing the first object;

[0102] A third display module, used for displaying the echo image;

[0103] The image processing device also includes:

[0104] The fourth display module is used to switch to display the corrected image when displaying the echo image.

[0105] In an optional example, the image processing apparatus further includes:

[0106] An object recognition module is used to obtain a preview image, recognize a first object displayed in the preview image, and obtain first coordinate information of a frame of the first object;

[0107] The fifth display module is used to display the preview image on the shooting preview interface at a preset frame rate and mark a frame in the preview image according to the first coordinate information.

[0108] In an optional example, the image processing apparatus further includes:

[0109] an associated storage module, used to associate and store at least one second preview image, a shooting timestamp of the second preview image, and first coordinate information of a frame identified in the second preview image, wherein the preview image displayed on the shooting preview interface includes the second preview image;

[0110] The at least one second preview image includes the first preview image; the search module 603 is used to search for the first preview image with the same shooting timestamp in the at least one stored second preview image according to the shooting timestamp of the first shot image.

[0111] In an optional example, the image fusion module includes:

[0112] A selection unit, used for acquiring a plurality of frames of captured images, and selecting a first captured image from the plurality of frames of captured images;

[0113] The fusion unit is used to perform image fusion processing on multiple frames of captured images with the first captured image as a reference frame to obtain a fused image.

[0114] In an optional example, the cropping correction module 604 includes:

[0115] a coordinate conversion unit, configured to convert the first coordinate information into second coordinate information that conforms to the size information of the first captured image according to the size information of the first preview image and the size information of the first captured image;

[0116] The cropping and correction unit is used to crop and correct the fused image according to the second coordinate information to obtain a corrected image of the first object.

[0117] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), an ATM or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0118] The extended reality device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0119] The image processing device provided in the embodiment of the present application can achieve Figures 1 to 5 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0120] Alternatively, if Figure 7 As shown, an embodiment of the present application further provides an electronic device 100, including a processor 101 and a memory 102, wherein the memory 102 stores programs or instructions that can be executed on the processor 101, and when the program or instructions are executed by the processor 101, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0121] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0122] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0123] The electronic device 1000 includes but is not limited to components such as a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810.

[0124] Those skilled in the art will appreciate that the electronic device 1000 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0125] The processor 810 is used to receive a first input when a preview image is displayed on the shooting preview interface and a frame is marked for a first object in the preview image;

[0126] The processor 810 is used for acquiring a plurality of frames of captured images in response to a first input, and performing image fusion processing on the plurality of frames of captured images to obtain a fused image;

[0127] The processor 810 is used to search for a first preview image associated with the same shooting timestamp as the first shot image in the preview images displayed on the shooting preview interface, where the first shot image is a reference frame in the multiple shot images to be fused.

[0128] The processor 810 is configured to crop and correct the fused image according to first coordinate information of a frame identified in the first preview image to obtain a corrected image of the first object.

[0129] Optionally, the processor 810 is configured to crop and correct the first preview image according to the first coordinate information to obtain an echo image showing the first object;

[0130] The processor 810 is configured to display a thumbnail of the echoed image.

[0131] Optionally, the processor 810 is configured to receive a second input;

[0132] The processor 810 is configured to display an echo image in response to a second input.

[0133] Optionally, the processor 810 is configured to crop and correct the first preview image according to the first coordinate information to obtain an echo image showing the first object;

[0134] The processor 810 is used to display the echo image;

[0135] The processor 810 is used to switch to displaying the corrected image when displaying the echo image.

[0136] Optionally, the processor 810 is configured to obtain a preview image, identify a first object displayed in the preview image, and obtain first coordinate information of a frame of the first object;

[0137] The processor 810 is used to display the preview image on the shooting preview interface at a preset frame rate, and mark a frame in the preview image according to the first coordinate information.

[0138] Optionally, the processor 810 is used to store at least one second preview image, a shooting timestamp of the second preview image, and first coordinate information of a frame identified in the second preview image in association with each other, and the preview image displayed on the shooting preview interface includes the second preview image;

[0139] The processor 810 is configured to search, according to the shooting timestamp of the first shot image, for a first preview image having the same shooting timestamp in the at least one stored second preview image.

[0140] Optionally, the processor 810 is used to acquire multiple frames of captured images, and select a first captured image from the multiple frames of captured images;

[0141] The processor 810 is used to perform image fusion processing on multiple frames of captured images with the first captured image as a reference frame to obtain a fused image.

[0142] Optionally, the processor 810 is configured to convert the first coordinate information into second coordinate information that conforms to the size information of the first captured image according to the size information of the first preview image and the size information of the first captured image;

[0143] The processor 810 is configured to crop and correct the fused image according to the second coordinate information to obtain a corrected image of the first object.

[0144] It should be understood that in the embodiment of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and at least one of other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0145] The memory 809 can be used to store software programs and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 809 may include a volatile memory or a non-volatile memory, or the memory 809 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0146] The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 810.

[0147] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0148] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0149] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0150] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0151] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0152] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0154] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: include: In a case where a preview image is displayed on a shooting preview interface and a frame is marked for a first object in the preview image, receiving a first input; In response to the first input, a plurality of frames of captured images are acquired, and the plurality of frames of captured images are subjected to image fusion processing to obtain a fused image; Searching for a first preview image with the same shooting timestamp as the first shot image in the preview images displayed on the shooting preview interface, wherein the first shot image is a reference frame in the multiple shot images to be fused; The fused image is cropped and corrected according to the first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

2. The image processing method according to claim 1, characterized in that: After searching for the first preview image associated with the same shooting timestamp as the first shot image, the method further includes: Cropping and correcting the first preview image according to the first coordinate information to obtain an echo image showing the first object; A thumbnail of the echoed image is displayed.

3. The image processing method according to claim 2, characterized in that: After displaying the thumbnail of the echoed image, the method further includes: receiving a second input; In response to the second input, the echo image is displayed.

4. The image processing method according to claim 1, characterized in that: After searching for the first preview image associated with the same shooting timestamp as the first shot image, the method further includes: Cropping and correcting the first preview image according to the first coordinate information to obtain an echo image showing the first object; displaying the echoed image; After the fused image is cropped and corrected according to the first coordinate information of the frame identified in the first preview image to obtain a corrected image showing the first object, the method further includes: When the echo image is displayed, the corrected image is switched to be displayed.

5. The image processing method according to claim 1, characterized in that: In the case where the preview image is displayed on the shooting preview interface and a frame is marked for the first object in the preview image, before receiving the first input, the method includes: Acquire a preview image, identify a first object displayed in the preview image, and obtain first coordinate information of a frame of the first object; The preview image is displayed on the shooting preview interface at a preset frame rate, and the frame is marked in the preview image according to the first coordinate information.

6. The image processing method according to claim 5, characterized in that: The method of acquiring a preview image, identifying a first object displayed in the preview image, and obtaining first coordinate information of a frame of the first object includes: storing at least one second preview image, a shooting timestamp of the second preview image, and first coordinate information of the frame identified in the second preview image in association with each other, wherein the preview image displayed on the shooting preview interface includes the second preview image; The at least one second preview image includes the first preview image; and searching for the first preview image associated with the first captured image with the same capture timestamp includes: According to the shooting timestamp of the first shot image, the first preview image having the same shooting timestamp is searched in the at least one stored second preview image.

7. The image processing method according to claim 1, characterized in that: The step of acquiring multiple frames of captured images and performing image fusion processing on the multiple frames of captured images to obtain a fused image includes: Acquire multiple frames of captured images, and select the first captured image from the multiple frames of captured images; Taking the first captured image as a reference frame, the multiple frames of captured images are subjected to image fusion processing to obtain a fused image.

8. The image processing method according to claim 1, characterized in that: The step of cropping and correcting the fused image according to the first coordinate information of the frame identified in the first preview image to obtain a corrected image showing the first object includes: converting the first coordinate information into second coordinate information that conforms to the size information of the first captured image according to the size information of the first preview image and the size information of the first captured image; The fused image is cropped and corrected according to the second coordinate information to obtain a corrected image showing the first object.

9. An image processing device, characterized in that: include: A first receiving module, configured to receive a first input when a preview image is displayed on a shooting preview interface and a frame is marked for a first object in the preview image; An image fusion module, configured to obtain a plurality of frames of captured images in response to the first input, and perform image fusion processing on the plurality of frames of captured images to obtain a fused image; A search module, configured to search for a first preview image associated with the same shooting timestamp as the first shot image, wherein the plurality of frames of shot images include the first shot image, and the preview images displayed on the shooting preview interface include the first preview image; The cropping and correction module is used to crop and correct the fused image according to the first coordinate information of the frame marked in the first preview image to obtain a corrected image showing the first object.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

Images